Previously my main concern was with LLM's writing worse code. Now my concern is the ability to write better code than I was ever able to write before -- considering every possible potential bug and race condition and error state and what have you -- but at what cost? How much more time do you have to spend researching, designing, making decisions around every possible edge case and state?
Before, you couldn't even attempt to balance quality with speed, because you couldn't see the quality in advance. You had to wait for bug reports to come in. Now, you're confronting quality up-front in a much deeper way. Which allows for much more reliable software, but then you have to decide how much effort is worth it and how much isn't?
This keeps bringing me back to John Backus’s Turing award lecture/paper, “Can Programming be Liberated from the Von Neumann Style?”
In it he pointed out a huge deficiency of most programming languages that still haunts us: they’re wickedly difficult to reason about. So proving important properties about programs is all but infeasible.
So we try to handle it empirically with unit tests, but that has its limitations.
LLMs can kinda sorta do it analytically, but that isn’t completely satisfying, either. The results are probabilistic, and they are pretty bad at inferring intent which can be a source of both type I and type II errors. But still, it would be silly to throw out a layer of Swiss cheese for having holes.
But I sure do wish we could have a plausible version of Backus’s original vision of a language that lets us formally declare (a useful subset of) the specification and then statically verify that the program conforms to it.
I mean, it depends on how you use an LLM like Claude.
You can not know what you're doing and use it to write slop.
Or you can set up your project with best practices in every way with full design documents and code that flows from design and full rigor around everything.
I do it the second way. And so yes, I find myself being confronted with so many edge cases that I just never thought about before. And I'd say it about doubles the time it takes me to deliver stuff, and I'm not always sure it's worth it.
I don't intend to argue that your experience is invalid. if 99% of people don't use ai in your way then you must understand your argument is parallel to that of gun control. anyway the edge cases you are unexpectedly encountering through ai, may I ask on average how many of them would be career ruining (or life endangering) as a percentage?
99% of people? Gun control? Career ruining? Life endangering?
You speak in hyperbolics, my friend. You may be lacking some realistic perspective here.
Plenty of people who are using LLM's within technical organizations, as opposed to script kiddies vibe coding, are encountering the same kind of issue. And people spend lots of time fixing bugs in existing code anyways. This is just a simple question of newly-visible tradeoffs.
your account has been here since 2011. unless the account got sold off, i take it that you're my elder and therefore are deserving of respect. so respectfully I ask, if you made a comment opening multiple avenues of discussion, and somebody shut them all down without acknowledgement, how you would react personally?
Previously my main concern was with LLM's writing worse code. Now my concern is the ability to write better code than I was ever able to write before -- considering every possible potential bug and race condition and error state and what have you -- but at what cost? How much more time do you have to spend researching, designing, making decisions around every possible edge case and state?
Before, you couldn't even attempt to balance quality with speed, because you couldn't see the quality in advance. You had to wait for bug reports to come in. Now, you're confronting quality up-front in a much deeper way. Which allows for much more reliable software, but then you have to decide how much effort is worth it and how much isn't?