I think its more likely that llms got better at using harnesses, which help force them to make sure everything compiles/runs before passing back to the user. This caused people to form habits around this where they slowly stopped checking the output.
So people are super impressed because things just work, and they do.. Until they dont.
Just because things compile doesnt mean they're going to hold up, and most of this slop wouldn't, ive seen it first hand and you can also tell by the increased amount of outtages across different platforms.
We’ve been two years away from nuclear fusion for 50 years. Sure, at some point, maybe we will genuinely be two years away, but without a crystal ball it’s not much use to keep saying it.
It was crap, it's good now is the positive equivalent of the boy who cried wolf. We've been hearing this bullshit so, so many times that no sane person would care to listen without some serious hard evidence and no your zero user vibe slop repo launched yesterday or "I'm so much more productive…dicking around the codebase but zero useful features to be seen by the user" doesn't cut it.
Ok, so NOW that the models are awesome, i assume you will drop a lot of impressive software, correct? You were not able to do that a year ago because they were bad, however, that bottleneck is gone NOW, so NOW you will deliver, right?
It's so obviously true until the next model comes out. Or until a non-American model comes out matching it, then it's "no one could using AI tools for coding until this new GPT/Claude model came out"
Could you share something that we can replicate, where SOTA models and harnesses, i.e. a Claude, does not work where say a junior or medior engineer new to your team would solve the problem?
Do people expect an eternal junior engineer in their teams? Even with a recent hire, there’s a deadline to the onboarding and hand holding process. Generally after three months, you were expected to deliver with no oversight other than peer reviews. It was a good deal for the team. Spend three months teaching, and get a productive team member for two years or more.
reply