Awesome. I remember thinking last year that software is about to get a lot more buggy now that people are offloading more to AI, but tools like this give me hope that perhaps the future of software actually looks a lot more safe.
Anecdotally in my company we have observed a far lower incident and outage rate since we moved to complete Agentic coding.
The extra velocity has meant we move considerably faster than the product team, so a lot of that extra dev capacity has been channeled into improving our tests, verifications and automated review processes - so hardly anything is slipping through the gaps.
This is the opposite of what I expected when we started working this way. One of the concerns we as a team raised was that it would lead to more bugs and lower quality but after 6 months the opposite appears to be true
How often do you need to change the actual library which does the encryption for the signal protocol? Adding features to the app does not require changes to the encryption.
Spending time upfront designing is sound advice for any software engineer. LLMs can also save time here as well, if you're disciplined enough to not proceed until the design is polished.
Given how many security vulnerabilities SOTA models are able to catch in software we've considered some of the most mature ever written, they can most certainly be more reliable in certain contexts and when used certain ways.
They're sure useful to find some vulnerabilities not previously noticed, but that doesn't make them write code better than humans.
Making them review LLM-generated code will probably catch some errors and vulnerabilities, but could well leave there insanely dangerous hallucinated code.
Are you working only with exceptionaly (like world class) good engineers that happen to write code by hand ? Because otherwise I'd find your claims dubious.
Meanwhile I've been dealing with so much human slop (including mine) for the last 25 years that what comes out of LLMs in 2026 is an absolute relief.
My claims? Do I live on a different universe? However bad "human slop" can be, an honest human is not going to write the completely senseless, security-shattering code that an LLM can output..!
You don't see what's going to happen if you don't even have a look at it?
You clearly must have been extremely lucky in your work environment then. As humans we've been tossing bugs and security issues for decades.
Current LLM output is so much better than what I or my colleagues used to write, on every levels. Better readability when properly steered, better performance, better security, everything. Frankly this year has been a humbling experience.
> Do I live on a different universe ?
I guess...
> You don't see what's going to happen if you don't even have a look at it?
This whole discussion is about not looking at it, did you see the initial post?
You are aware that howwer much better to your hellish experience the average LLM output is, it's going to spit brain-dead, extremely dangerous nonsense sometimes?
> it's going to spit brain-dead, extremely dangerous nonsense sometimes
Again, in my experience it's very rare (and rarer than at the beginning of this year). What was your experience with these tools ?
I'm not OP, and I'm not defending the point of not looking at the code at all or not doing any review. I do think checking each line is definitely not necessary and the amount of verification should be adapted to your domain.
However rare it might be, do you agree that it justifies checking all of the code?
How can you verify the code if you don't verify "each line"?
You can maybe check them with less care, yes.
But even for the most frivolous domains, in most cases malicious software can cause a lot of damage (for on example on older systems), so it is irresponsible to not check it, in my opinion.
No, definitely not. We've all been using libraries for decades with very real (and not rare at all) human-made bugs in them and almost never verified the code by reading each and every line. That would be impractical in most cases, even if it may be required in a few rare cases. What we do is test, to very various degrees depending on the domain.
There'd be a debate to have about why LLMs are threatening to some engineers while they are empowering to others. I'm having a blast producing the best output I ever done while some of my colleagues are suffering to the point of thinking about changing careers. There must be deep psychological reasons to this.
Libraries are (or used to be) written by humans, Jodie, which if honest don't add obviously devastating code.
For open source libraries there's a chance that some of its users reviewed the code, for commercial libraries there's the legal risks holding them from being excessively careless.
LLM code instead can only be looked upon by you.
And, until a short while ago we were worrying of how to reduce the risks placed by libraries, with dependency cooldowns etc. Now we decided that we can ship random code seen by absolutely no one?
> There'd be a debate to have about why LLMs are threatening to some engineers
You're claiming that if an engineer doesn't like LLMs it's because he feels threatened by them?
There definitely are at least two very different groups of software developers: one that enjoys writing code, and the mechanicality and predictability of it, and one that maybe doesn't love developing, and for sure cares more about making a product, and maybe about the money that you can earn from it.
This is the purely psychological difference, but then there's a group of developers who feels the responsibility for what they do, realizes what effects their software can have and does what it can to protect its users; and another group that doesn't care too much, and considers their user's fault to decide to use their software, and to possibly harm other people through its use.
And, if your capabilities are limited, or your company forces you to go too fast, for sure the average LLM output can be better than your average code.
Not all LLMs fans are like that, of course, many just have misconceptions and don't realize the risks.
I think an isolated and ambiguous sentence like the one you provided with zero surrounding context is perhaps a bad example. English can be very ambiguous. It is also possible to write unambiguous English, as far as deciphering intention is concerned. I think LLMs have shown us just how far English can go as a medium to get quite good results. And as others mention, there are other tools, like compiled programming languages and static analysis tools and unit tests, that can do more of the heavy lifting on our behalf.
Nice work. I unfortunately wouldn't be able to use it for programming because I find the parenthesis and square bracket look too similar, and it would definitely increase extraneous cognitive load.
I think the OP was talking about robotics specifically and not tasks like the one you described. Nobody can argue that for tasks like your binder replacement cost savings are enormous.
No, they were carrying on the analogy about artisanal home builders being replaced by mythical robots that do it perfectly.
In this analogy, we human software devs are the artisanal homebuilders and the fancy LLMs are the robots.
And yes, there's quite a cost to build all the datacenters for all the AI that Big Tech believes that people will want. But as you mention, the cost savings are enormous for even despite that, and will make a much larger set of problems accessible to real software solutions, that previously either stayed on paper, in peoples' heads, or maybe an Excel sheet somewhere.
I agree with this to a large extent. This new technology is lowering the barrier to meaningfully work with existing tools. It's increasing the magnitude of jostling that may very well bump us out of a local maximum sooner than before.
I would argue that all of the problems we have to solve are novel and tricky. The rest we've already solved but constantly reinvent (not novel) or already solved and just need to piece together like Legos (not tricky).
Forgive my ignorance, since I am only superficially knowledgeable about this, but how are you certain that it "completely forgot" as opposed to the knowledge still being encoded but 800 inferences was not enough to probabilistically trigger it? Like hunting down a race condition that happens rarely enough that it may not surface in 800 runs of the program.
The study was entirely about the probabalistic triggering. That's what I was after. I was actually studying one shot learning and how it learns and forgets a single training example.
More detail:
Trained like ~500 runs.
Each run was:
- inference 5k times to make sure question isn't guessable
- Train on one shot of a non guessable question.
- inference 800 times for a % remembered
Then do this 200 times per run:
- train one step on a different normal dataset
- inference 800 times on the Q
reply