Yeah I think depends on the fidelity of the world model.
If you count “impales person with blue stick while distant asteroid is in position X Y Z” then yes, there are probably infinite, so it depends how you class a discrete method
I’m not sure I believe this anymore. Maybe around 2 years ago, I think.
But the trajectory, of everything it seems, has completely changed. And I have no optimism left.
The world would need to possibly take up arms against its leadership to recover from where it is right now. The problem is, I know the masses are pacifist and reasonable, and it’s the insane few that are either running everything or have all the money. That dreads me, because just as I wouldn’t take up arms, nobody else will either, so who will stop the descent?
You got voted down, but your sentiment is reasonable. Times are pretty rough right now, politicians seem worse than ever, and people seem to be more ready to hate each other than ever. But the trajectory is controlled by individuals.
Heinlein wrote this in a dark period in history too. The writing of it was resistance against the trajectory of his times. Belief is a choice. Action is a choice. If you don't feel optimism, that's OK to recognize. But then make a choice: to believe something good about your neighbor, to honor something noble about someone you disagree with, and to act as if we each can do better than societal trends seem to indicate.
Most who are running a successful (or) profitable startup know that sales is a day and night job in most industries, and that it’s completely unrealistic not to have separate cofounders for development and sales.
Sure, as a total fluke in a very specific industry, software might sell itself. Everywhere else, and especially in physical product and deep tech, someone has to fully own the sales side of the business including the really time intensive stuff like website, email marketing, conferences, merch, sales meetings and trade shows. Heck, even viable solo founder companies would probably sell a magnitude harder with a sales co-founder.
Anecdotally my company picked up a few big distributor contracts for software tools and deep tech products, stuff people want. The products are done, high quality, but they don’t sell themselves - we have to actively put effort into outreach for every sale. Think about that in the context of your own startup. Building the thing is not the end hurdle, it’s kind of like a dependency to selling the thing, which creates revenue.
I have an extensive sales background and also an entry level software developer that worked for years at a coworking office while getting my solo miniSaas off the ground.
I tried over and over to explain to other founders that they REALLY should think about sales early. I'd try to say "Ok imagine your product is done magically. It works great. Now what?"
And it was interesting to see them struggle to think of what they would actually do if it was done.
You’ll rarely be too early. If you’re building some people want/need, they will be happy to hear what you’re working towards and potentially give feedback.
If you have customers that are interested, and you end up burning the relationship, there are usually other customers that are interested. The real risk when founding a startup is that you end up building something nobody wants, because the need is made up. Once you have ~10 or so customers, you can be reasonably sure that there are more than 10 or so customers, because markets are almost never that small or that specialized.
(The exception is companies that are glorified consultancies, eg. you have just one customer that needs a very specific service done. But these are eliminated once you've triangulated enough to find 10 or so customers. At that point your product is general enough that it will no longer appeal to the companies that just want custom development outsourced.)
You can definitely do certain kinds of sales too early. Early sales should be website and brand creation, and talking to people about what you’re building to see if they’re interested, not necessarily closing deals. I’d say close 1 or 2 pilot deals (for pennies and/or free is good) and then pause it there, so that you have only a reasonable amount of pressure to build and steer a high quality product.
Otherwise you’ll be under such stress that you’ll become reactive and lose your Birds Eye View, and that’s a slow way to sink.
My flippant answer is ask Bill Gates about that. It seemed to work out.
But in reality, as long as you're somewhat honest in the sales process, it'll still work out better to talk to potential customers as early as possible. Maybe the dogma has changed, but doesn't YC say to talk to your early users/customers as early as possible, and if you dont have anyone like that to talk to, how are you even building this thing you have no connection to?
There's no other leading metric that will tell you (directionally) whether you are doing a good job. Did they buy it? Did the conversion get made? Is it easier or harder to sell based on what you made/how you made it?
Yes, you can have whales that fuck you up, or bad feedback, or whatever, but at the end of the day a lot more folks end up siloing worrying about initial impressions, run out of money, build something nobody cares about anyway, and then they get to START at "let's go build what people actually will buy"
delay building as much as possible - unless you're doing something completely innovative such as inventing the iPhone.
research markets, research TAMs (they can be too big or too small - which is fatal) - read books about selling. reach out to potential customers in your target markets that match the TAM you want.
The reason this cost so much is because the AI has the ridiculous goal of getting identical assembly output.
The agents would have had to mess around with compiler versions, optimisation options, and the phase of the moon as well.
If you just went for functional equivalence, it would probably cost 10x or 100x less tokens.
Another false economy was using Sonnet instead of a more intelligent model like Sol 6.1 (1), which would have cost more per token, but is 100x or so better at reverse engineering and coding and therefore can chew through the source code much quicker and make fewer mistakes, meaning less work needing to be scrapped.
In my testing doing a similar task, I ran multiple sonnet for weeks and burnt through ~$1000 in tokens to get 20% completion and output that was pretty bad. After switching to Sol 6.1, it finished the whole task in around 2 days, cost around $50, and it did it with zero supervision and a single /goal.
(1): struggle to use Opus for reverse engineering, too many safeguards. OAI has virtually none, and uses way less tokens so is more economical.
Identical output isn’t ridiculous. It’s pretty much a requirement to ensure the game actually is the same in every way. These decomp projects are pitched as a high performance alternative to emulation.
No one is going to use it if it’s a kind of close but not really reimplementation.
Testing functional equivalence is also pretty much impossible. How would you for example test the new one has exactly the same bugs which haven’t been discovered yet. Or doesn’t introduce new ones? This stuff matters for speed runners.
It’s ridiculous because something as simple as the compiler picking different registers is going to make zero functional difference but give a false negative on assembly compare.
Yet the C code can’t pick what registers to use, so the poor agent is probably shuffling the code around randomly for hours or days until it matches.
That’s probably why the agent dropped down into inline assembly in the first place (the author complained about this), because I bet it’s thinking trace was that this is futile.
Compilers themselves are not even deterministic and running them multiple times creates different assembly.
> Yet the C code can’t pick what registers to use, so the poor agent is probably shuffling the code around randomly for hours or days until it matches.
Humans too do that for matching decompilation. Or at least I imagine so, given that I personally refuse to do that.
People have different goals and will use different techniques to achieve them. The video game reverse-engineering/decompilation community isn't a hive-mind, I went ahead and created ghidra-delinker-extension because I had my own ideas on how to do that.
Isn't MW2 rooted in id tech engines? Last I checked there was raw assembly inside the code base. I would have started with quake 2/3 whatever source code and tried to match that up with Claude first - some of it has to have survived (though I doubt much)
How do you know the original code didn't use some undefined variable on stack, but happened to consistently work because something else spilled a register containing "suitable" value?
I think you're overlooking an important point here. While the token consumption might feel extremely high, I think it's a bit deceiving.
Due to this strict harness, we were able to use extremely dumb models efficiently.
I think at least 200-300B of tokens (maybe even more) were processed by Luna.
Given that Astra for example is 100x more expensive per token than Luna, a high token consumption doesn't necessarily mean high cost.
So these 300B Luna tokens would equate to 3B Astra tokens and roughly 15B Sol tokens.
Unfortunately a huge part of the logs is lost, so I can't do a detailed analysis on which models were used, how much they cost, etc.
> Due to this strict harness, we were able to use extremely dumb models efficiently.
With all due respect I seriously doubt that. What are you doing that's different? Almost every test between just Opus and Opus conducting cheaper agent swarms ends up with Opus getting better results at a lower cost. People set up harnesses just like yours and see this.
I tried giving glm 5.3 ghidra. I found that it frequently did better just reading the actual assembly, where needed writing decompilers in real time, in the terminal.
The reasoning showed that it was able to write ghidra as needed and indeed where necessary simply read assembly tokens.
Trying to think about ghidra probably confused it.
I also think that there is a gradient of losing intelligence in refusals, opposed to a simple boolean refuse. I suspect that using an abliterated model would boost the intelligence way up.
I don’t recommend always using Astra for RE but in my experience I’d use Sol as minimum. As of recently, perhaps upgraded Terra is good value, but in general it seems like you want the smartest model that is economical to run 24/7 and Astra isn’t economical.
Identical assembly output is the only way to get functional equivalence, otherwise you're getting equivalence in some percentage of situations, but not 100%. Your bug is my feature etc.
Is a code comment and ledger the best way? Should the agent just fill out a form or something and attach it to the sub issue. This is how the hospital would work.
It often does that too, but we keep the ledger and the code comment as well, to ensure that if it doesn't get resolved by the sibling, that it's not lost. If the sibling does resolve it, it's removed from the ledger and the code.
There are Pros and Cons. Cons: generated passwords have less flexibility, not being able to add custom fields (ie backup codes or PIN), can’t store key files (unless you copy and paste contents into Notes section, which doesn’t redact it), no independent master vault password like 1Password.
Pros: it’s free and will probably be free forever, can store and generate TOTP codes, biometric unlock, very easily syncs between devices, great integration on macOS and iPhone/Pad, easy sharing of password to family members without them needing a subscription.
I have been a 1PW user for 10+ years but earlier this year started the long transition to save $10/month for the rest of my life which turns out to be a non-insignificant amount.
i think i'm making this connection because i used to roll my eyes at people a bit older than me who started buying vinyls. "just stream your music, unc."
then i started buying physical movies and had a, "ah shit, guess that's me now..."-moment :D
wonder what the next step will be. "jake actually bought a PC again instead of using a cloudbox like everyone else"(hope not).
If you count “impales person with blue stick while distant asteroid is in position X Y Z” then yes, there are probably infinite, so it depends how you class a discrete method
reply