Occasionally it takes me at most 10 minutes to figure out that a solution the LLM took an hour to churn out is not up to par and I need to throw it away entirely.

There are diminishing returns, but half of the rate you mentioned would already help me keep focus.

You’re thinking about this the wrong way. Yes, any individual developer doesn’t need this much power all the time. But any individual developer would like to get an answer quickly, which this does. You share the speed and capacity of this over a whole organization. Any one developer only uses a fraction of its capabilities at any given time. Getting an “instantaneous” answer is quite helpful. Think of it like a shared compiler; a developer is not constantly compiling, but having an “instantaneous” compiler is quite useful.

most of the time developers arn't looking for answers. They want to implement some feature. And I'd hazard most developers arn't using these things as a crafter implementer but a frankenstein of architect, implementer, tester, builder and deployers.

And the dev, if they're sane, wants to actually evaluate what it's doing in those arenas. Speed might be a psychology killer then. Actually being able to see the development process as it does it's good think and it's wrong think is a benefit for local development.

I don't think speed of tokens is all that magical if there's a huge imbalance of ability to "human in the loop" it's processs.

Code is an answer to a coding question or direction. Getting an answer quickly is always better than getting an answer slowly. You can evaluate the answer at your leisure. The human can take as long as the human wants. Meanwhile, the infrastructure is generating answers quickly for other people. Faster is always better, even with a human in the loop.

No, faster is not always better, there's a reason why we have cliches and literal nursery rhymes about it.

You're larping too, can we bring NFTs back so the the dunning kruger vibe coder commenters on the internet find something else to fixate on?

1. You don’t know anything about me, so don’t presume.

2. Which nursery rhyme? The tortoise and the hare is a fable, not a nursery rhyme. Even if you think the lesson was “faster is not always better,” which it wasn’t, faster is sometimes better. And in particular, when a machine is involved, all things being equal, faster is better. Would you rather use a fast compiler or a slow compiler, all things being equal except cost to keep the analogy with the original? Compiler A produces the same binary with the same optimization levels a compiler B, but in 1/10th the time. Assuming you can afford it or someone above you in your organization is willing to buy it for you, you take compiler A. If you can’t afford it, you lust after it. Why, because a slow compiler results in a slow iteration loop no matter what. With an instantaneous compiler, the loop is as slow or fast as the human part of it. You want to think some more, rearchitect this or that? Fine. Your code is always ready to run.

I get the sense that you’re anti-AI. That’s fine. I’m not sold myself, though I’m increasingly trending that way. For those that choose to use AI, I submit that a fast AI, all things equal (same output), is better than a slow AI for the same reason that a fast compiler is more preferable. Note that I’m not saying that faster overall development at all costs is preferable, or that we should just throw everything at the AI and stop thinking.

speed gets you to have flow though, rather than breaking your concentration with various waits that aren't actually times where you can stop and think because its 7s surrounded by tiring reading.

a milisecond response, that you can then spend 30s focused on is better.

its the same as a build tool. just slow enough build tools really suck, compared to either an instant build or a 30min build

evaluating the areas isnt the same as watching the llm harnes say "thinking" between submitting a request and getting a response

I thiml of it as driving. If youre in kansas and on q highway with no features, go ahead, speed away. But the reality is, a human in the loop system needs a variable speed car. Like all driving, sure, some will go faster.

But the opinion is about psychology and expectations.

If you expect the singularity, then of course this sounds crazy. To christians, ghosts also sound crazy. It dont mean a think if yoire biased to expect AGIM

At those token rates you could treat the LLM as a compiler and rebuild your software from scratch every week. That would mean the end for AI slop because people who spend the effort to build a process that creates high quality software from scratch can take full advantage of the one shot performance of an LLM without having to release the process itself.

Youre completely missing my argument. Speed is not the bottleneck, its the review burden.

You need human review when working on anything not table stakes. You cannot write shitty code for an avionics package on an airplane under the assumption that if we encounter bugs, its no big deal because we can rewrite it super fast..

It seems like all these bullish takes on AI are coming from people who think all software runs in a browser, and simply exists to take stripe payments or serve ads. Most software is actually very high stakes and lives depend on it and have high compliance burdens.

Just because it can "one shot" (which actually means 90 tools calls and a whole lot of brute forcing against tests/linter/compiler, and possibly reward hacking) to something that runs, does not mean it will continue to run well.

People treat llm made software like disposable code. You're the only one using it, typically on a single machine with no real world multi-user testing, but for some reason we derive from the fact it worked well the 3 times we used it, on a single machine with a single user that it is evidence of llm solving programming. Its bogus.

You can lessen the human review burden by pitting multiple agents against each other. Ask 3 agents to implement the same solution, then set them up against each other in differential tests. This can find bugs in them or clarify the spec. When tokens get cheap it is worth having multiple replicas, taking advantage of different programming languages, libraries and agents. You don't keep one version, you have a population, and it tests itself. It's easier to implement again than to imagine testing scenarios anyway.