In your code, this makes the model subtly devolve into certain patterns and stereotypes that aren't best practices, it's just a randomly frozen trajectory that is extremely hard to change zero-shot, without running a second pass. In other words, it's the same problem as in writing. Moreover, this is harmful for software design, where the ability to recognize the uncertainty matters. This is the reason everybody does the same thing. Frontend design also suffers from that.
>If you need more creativity it can come from the user or the harness.
Not fully possible. You can randomize the prompt all you want but you'll always get repetitive patterns and same choices in the same generation. This failure mode makes the model learn a very rigid mapping between this input idea and that single output idea (at best! at worst it's multiple input ideas -> single output idea).
> In your code, this makes the model subtly devolve into certain patterns and stereotypes that aren't best practices, it's just a randomly frozen trajectory that is extremely hard to change zero-shot, without running a second pass.
But in practice, who uses models “zero-shot”? It is standard for coding agents to review their own code even before returning to the user — so a zero-shot agent invocation isn’t a zero-shot model invocation. And then many users will get a different run of the agent to review the change a(frequently even a different model being invoked by a different harness-e.g. I personally mix Claude, Copilot and Cursor)
> This is the reason everybody does the same thing.
If you have a human-in-the-loop reviewing the agent output, different humans will have different opinions on what the agent should have done instead, and they’ll tell the agent to redo its output in different ways. And then they tire of repeating themselves, and it becomes guidelines in AGENTS.md.
Additionally, a coding agent asked to add a feature to a large code base will frequently check for the code base’s existing design and coding patterns, and so will end up producing very different code than it would have given a blank slate.
Yes, obviously external sources like human input, random generators, tools will bring variation across a single generation, but there's not remotely enough semantic entropy in them, unless you make meaningful decisions each little step. Longer reasoning and unattended agentic loops and self-reviews won't introduce variation, the intuition is wrong in this case. Anything isn't explicitly conditioned by external sources will be sampled from model's knowledge space, and if it's constrained by this effect compared to the post-training set distribution you get reduced capabilities. You can observe it in the form of everything looking the same in practice. It's a bit of a butterfly effect, some training quirk in a model somewhere is essentially shaping the entire world.
If you fix mode collapse, the effect on software engineering use case would be neutral at worst, or an improvement at best. It's not a verifiable task and has plenty of room for subjective judgement which benefits from semantic variety. For the same reason greedy decoding (temp 0) isn't used even for coding, it leads to degraded and pathologic outputs.
Right, and I definitively don't want to guess at what I might want. A "round trip to the user" is much better than getting bogged down implementing something I didn't ask for and don't want.
Which is good is you're developing a tool to turn natural language into functional code. But it's bad if you need a trillion dollars because you promised everyone a magic wishing genie.
It’d be a waste of tax payer money. The companies who were hacked are just directly receiving restitution behind closed doors. Unless they are unhappy there’s no reason to bring the courts into it.
I completely agree that proper safeguards were not in place. But response is happening just not in public. Companies work problems out between their own lawyers all the time because it’s much cheaper, faster and easier to control the PR around.
Criminal prosecution can be done without needing the victims to call for it, and in this case I would argue it's necessary, as it will help ensure companies don't treat their act of gross negligence as "business as usual".
What LLMs are you using? Just simply telling Sol 6.1, Opus 5.5, Fable or Astra to performance test something will get you a solid improvement in poorly optimized code.
Giving it a specific plan will get you a solid test harness.
And setting up an autoresearch system and running it overnight will get you expert level optimizations if you set the metric up right.
Performance testing has always had problems with isolation, mocking, covariance of services etc. I’ve just spent two weeks driving down latency across our framework, and autoresearch was definitely not a viable path. Most of these loops have these logarithmic, non-step change curves.
Just like with actual research, the real benefits came from me asking why is it like X and not like Y?
The original performance testing framework Claude created to bench our different versions against did not even mock high variance provider calls…
yes but you can use an llm to do the grunt work which still speeds up the process, you don't need to be like "computah speed this up", you can be like "computah, write me a profiling script using lldb to inspect this one hotpath and look for X, Y, and X" etc etc.
For programmers this is the equivalent of when accountants got spreadsheets. We can just tell the machine what to do without manually performing all the steps.
Some programmers enjoyed the code writing more than the problem solving though.
> Some programmers enjoyed the code writing more than the problem solving though.
Programmers enjoy problem solving and coding, but wishing to have problems solved isn't the same thing and only satisfying if it wasn't something that was going to get done otherwise. Wishing products into existence is more of a PM dream
Coding is an extremely verifiable and loopable task, like math (in fact, all of the math that these models has done has been through the lens of Lean, which is itself just coding). I am talking about their capabilities in tasks that are more general, the execution of which represent the vast majority of economic value generation in the world.
I don’t want creative code. I want boring code following generally agreed upon best practices.
If you need more creativity it can come from the user or the harness.