It's not too dissimilar to life as a Principal level engineer pre-AI. You technically oversee the work of many engineers on their own systems, but you couldn't possibly review all the code yourself. You appreciate that human code review reduces, but does not eliminate, the number and severity of bugs shipped to production. You don't have the low-level details in your head, just the high-level ones. You design how all these engineers interact with each other, without stepping on any toes.
Time will tell whether we were right to move from a system where we trusted humans to understand the low-level system, to one where we trust fancy calculators. But so far, it's working.
There seems to be disturbingly little objective evidence of that. There's certainly a lot of code being written but how much actual value that code is adding is another question. With the scale of resources that have been pumped into AI-supported software development over the past few years where are all the new applications and the quantum leaps in quality and performance in the applications we already had? If anything I seem to be seeing more anecdotes going the opposite way so far.
Anthropic's recent blog post about how they replaced Git should be a massive wakeup call. A trillion-ish dollar company (exact number unimportant here), the vast majority of its value coming from its intellectual property, replacing the storage layer that is an essential part of its system of record for that intellectual property, written pretty much entirely with its in-house Kool-Aid.
Disturbingly little evidence? I would hardly call that "disturbingly little"...
(Apologies if I'm misreading your reply. I'm assuming it was sarcasm but Poe's Law might apply here.)
This does feel a lot like the "Agile era" that was the last silver bullet that was supposed to fix software development forever and the "Cloud era" that was supposed to fix infrastructure and deployments.
Agile had a few good ideas. Code reviews and automated tests are commonly used today.
It had a few plausible ideas that it took too far. We are better at recognising the value of incremental development over big up-front design today - despite the ironic clash this seems to be having with agentic workflows - but almost everyone hates stand-ups and lots of projects have gone off the rails by breaking things down far too much and never committing to anything and so never getting anywhere.
And it had plenty of ideas that almost all of us have thrown out as unpleasant and/or unworkable. Pair programming comes to mind immediately.
What didn't really change was the fundamental nature of software development. Did we adopt a few useful ideas and get a bit more efficient and improve quality a bit? Certainly. Did we get orders of magnitude better at anything and give up all the tried and tested ways of working in favour of radical new ones that never quite felt right? No.
On our current trajectory I'm expecting a broadly similar outcome for LLMs. They're now very good at some things. But they are still pretty bad at other things that also matter - and some of the reasons they are bad at them are an inherent part of how they work and therefore probably can't be overcome (at least not with incremental improvements on the technology that still basically work the same way). So I am expecting to see a period of consolidation with a lot of organisations pulling back from their dramatic leaps into the unknown and downsizing of dev teams once the big AI companies start squeezing financially and the problems of developer burnout and stagnating skills and low morale that we're already seeing after just a few months become endemic. And then as I suggested in another comment I think it's also increasingly possible that we'll see developers unionise and/or push for professional regulation and licensing so they can resist the pressure to just ship whatever their agent wrote without even bothering to check it but still being responsible when inevitably things then go wrong.
If AI progress stops right now or stalls out soon sure, but if we really do have something significantly better coming, up to and including AGI, I think none of this will be an issue.
Well yes - we never know what the future holds and some new development or discovery could always change the trajectory completely. Personally I prefer to deal in what I can see and validate rather than hope and hype so until we see the actual developments or discoveries I'm cautious about assuming what will become possible with hypothetical improvements.
In particular I don't see why we should assume that the capabilities of AI programming tools will continue to improve at the same pace. There is no obvious source of new and high quality data for anyone to slurp up to train the next generation of models. Much of the change over the past year hasn't come from the incremental improvements in the models anyway. In fact I've even seen a few comments online arguing that some of the newer models had regressed in some respects. It has been the shift towards using agents and automating the incremental refinement of work in progress instead of waiting for human intervention that has been the big change and I don't think we really know yet whether the problems we're already seeing with that approach can be resolved effectively.
Yeah no one can predict the future but betting on progress tends to be a better heuristic with technology, especially in AI given the rate of current progress. And as far as I know the availability of (human generated) training data isn't a strict requirement for further progress given test time scaling. The way I see it is we're still running these models on hardware mostly designed before the LLM revolution so we'll be seeing massive gains in, at the very least, the speed and cost of the current frontier models in the coming years which will make all sorts of things possible. I also think it's more likely than not they will keep improving in capability at some rate and the things software engineers are complaining about now will be seen by the labs and worked on deliberately if simply scaling the models itself isn't enough.
The assumption was that some engineers knew how each part of the system worked, or at least the system was comprehensible because it was created by humans for other humans. There were comments explaining difficult places, etc. With AI this assumption is broken. Now nobody understands how things work, and it cannot be understood in principle because the system is a mountain of slop created by some alien intelligence.
I'm a staff engineer on team of 20+ engineers working in a single codebase. We're all pretty heavy users of AI, like most engineers today.
I worry about a lack of understanding creating risk, and we are creating tools and conventions to address this risk.
But I don't yet see evidence that we're losing touch with what we're building.. yes, code is shipping faster, but I'm honestly just not seeing this "sleep-walk-engineering" issue in reality.
Code isn't shipping so fast that the engineer doesn't understand the intent and general implementation details of the PR.
PRs are required to be small (400ish lines), and have tests, and good PR descriptions must be present. And two humans must approve the PR.
The time it takes to understand, verify and validate the change is fast, often as fast as or faster than the time it takes for AI to implement.
As it stands right now, our level of understanding, PLUS our new verification tools... is all we seem to need.
We're not like the hyperscalers, but we're considering how some of their approaches might fit into our workflow.
And much like Anthropic, we aren't seeing bugs increase with the rate of code increase.
Engineers are still able to understand enough of what they need to.
Well, the fancy calculators do. Yes, they still need guidance to keep from going off the rails. But, at least from personal anecdotal experience, I need to go spelunking into the codebase less and less over time. The agents and models really did just start to become that good.
Time will tell whether we were right to move from a system where we trusted humans to understand the low-level system, to one where we trust fancy calculators. But so far, it's working.