I like the way you put this. I agree this is where the frontier of software development is right now. You can have LLM chug and bug fix all day long but in the end most real issues are in this layer of intent. I don't think this problem is particular to DB/programming language codebase either - it happens a lot in product design it just doesn't surface in code review. "Everything is working perfectly" yet the resulting workflow or UI is not truly making the user's life easier, or has grown grotesque with additions/modifications.
I will say, if the problem is purely code you occasionally see "sparks of AGI" where the model will step back and say "hold on, there is a massive simplification we can make here...". This started happening with Fable-class models. But rarely if ever will the model push for a huge simplification at the product level.
In my experience it's the opposite - the model's design decisions always try to add complexity, even when it's not necessary.
So you get permission flags and multilayered data structured with complex interfaces, and there is never any sense of "Wait a minute - we don't need this."
It's much like the way AI art looks meaninglessly busy with too many random lines or other elements, and AI writing is always exactly, precisely, specifically, unnecessarily adverbial.
>> In my experience it's the opposite - the model's design decisions always try to add complexity, even when it's not necessary.
This happens if the model does not have good context regarding what is being built, why and for whom.
For one of my side projects, my CLAUDE.md has a line that says "the users of this product will be non-technical people who find technology scary." And you know what? Claude always steers me away from complexity when working on that project.
This is why code is going to be automated, and the software engineer is a link between organizational intent and the code. This is what I’m working on.
Finding the holes in your specs under automation: time spent is returned 10x
Writing code: Human 15 lines per minute. LLM: 600+ lines per minute
Functional Validation: same time per iteration. Fewer bugs with LLM when properly specified => fewer iterations. Increased awareness of LLM escaping specification boundaries feeds back into increasing quality of spec writing => fewer bugs.
The difference is that instead of spending half your time understanding the business requirements and half your time writing code, you'll spend half your time understanding the business requirements, 5% of your time typing these requirements into a chatbot, and the rest of your time doing... ????
Some of the rest of the time can be spent making it look like your ratios are the same but in reality you just take the saved time and do whatever you want with it. The decision making and reviews aren’t happening any faster anyway so just take it easy and coast a bit
I guarantee there will not actually be saved time - the bullshit will increase to more than compensate, as everyone actually fixing things will be drowned out by bullshitters just pretending to fix things and creating more actual problems.
I go back and forth on this. For awhile yes it took a lot of workflow tweaks and finesse to get directionally correct outputs (thinking back like mid 2025). I invested time in figuring that out, mostly because I was interested, but those skills are all obsolete now, you don't need to worry so much about context management and special instruction recipes or anything these days.
The meta at this point seems to be mostly around "knowing what to ask for". Eg: I suck at prompting for specific design tweaks, because I don't have the design vocabulary to be specific and articulate what I want. It's a lot of trial and error. I hear people who are developing full scale applications without a tech background run into the issue writ large. For now, I have an advantage over these people when building applications. But perhaps that's just one round of RL away from being obsolete too. Supposedly the model will eventually have such good taste in design/architecture/whatever there will be little point in asking for tweaks.
The only thing that seems relatively future-proof is just being able to communicate clearly what you're trying to do. Not how to do it, or even what success looks like. Perhaps an element of it is even as basic as being honest with yourself.
I think there are a lot of people using AI who aren't as familiar with these tools as us. We stop noticing a lot of the complexity when we pick it up gradually. There are so many powerful capabilities built into codex/ claude code that have little or no UI affordance.
it's tons of small things like choosing the right effort level for a task. or knowing when to ask for subagents. oh, you can have two threads working at the same time and communicating with each other? you can do /side to parallel path and fold that back into the main thread?
which of these things happens automatically, and which do you need to ask for? and does any of this still work the way it did a few months ago?
the list goes on forever. many users of AI are only just now getting comfortable with skills (and probably still writing bad skills unfortunately hah).
A lot of the specific things you learned may have become obsolete. But there's still a gap between what these tools can do and what most people know how to get them to do. Keeping up with that is a skill too, but perhaps not forever.
> and does any of this still work the way it did a few months ago?
For me this is why it's not worth it investing much time and effort learning any specific AI workflow. Remember "prompt engineering"? Remember "Ralph loops"?
I'll play with various harnesses but until things settle down in a few years I'm not going to learn Claude Code the way I learned SQL.
"In a few years" is eons. I have absolutely no idea how these things will look in a few years and I'm not at all confident that it will settle, rather than accelerate until it's basically not even recognizable as a software "tool" that you have to "learn" for a "job".
Codex /side allows you to toggle to a different pane and maintain an ephemeral or persistent side conversation at the context point which it was launched. It can't natively communicate back to the main thread though. It also allows you to fork the side chat into a new chat. I don't believe it can edit files in side chat mode so the only way to paste context back is fork and do that or just copy and paste. So similar to /btw but with other features.
Claude allows the teams feature and easily spawning co agents or subagents in tmux panes you can interact with (or send to the background). These can communicate with each other. Sometimes you can't interact with one of the tmux agents agents though if it was spawned as some specific type I can't remember.
> The meta at this point seems to be mostly around "knowing what to ask for"
We came full circle on the problem that also exists with Google Search, and why so many people don't get good results out of it.
> For now, I have an advantage over these people when building applications. But perhaps that's just one round of RL away from being obsolete too.
I don't have a crystal ball, but my eggs are in the basket of "It isn't, and it won't be"
> The only thing that seems relatively future-proof is just being able to communicate clearly what you're trying to do
This somehow contradicts the previous statement. The advantage you have is, because of domain knowledge expertise, you're able to articulate what you want. It's not that you knowing what LazyLoading is that you now can describe it better, you can immediately tell that when your Frontend stalls for 5 seconds after hitting the "Load" button, perhaps it eagerly loads and blocks.
> We came full circle on the problem that also exists with Google Search, and why so many people don't get good results out of it.
yeah. I spent the first 15 years of my career being amazed at how many problems people came to me with which I was able to resolve in 10 minutes with a simple google search.
I suspect I will spend the next 15 years of my career being amazed at how many problems people come to me with which I am able to resolve in 10 minutes with a simple LLM prompt.
I am probably not the only one who feels embarassed when I copy-paste a large error output to an LLM and it points out to me that the error message very clearly said what I needed to do already.
I think the knowledge is one level higher at this point. It’s not LazyLoading that you need to know about, it’s just that it can and should be faster. That you can demand more (and sometimes you can’t). It’s hard to know the difference when you’re not in your area of expertise.
Fair point on the contradiction. I don’t have the answers either.
If we figure out long horizon tasks, why wouldn't you be able to ask it to send out a user survey, prioritize the next iteration of work based on the results, and then execute on it?
These things are already figuring out how to use hidden message boards to pass information between themselves for long running tasks. I see no reason we wouldn't be able to automate the whole product life cycle eventually.
Programmers can’t even agree on tabs vs spaces, less alone something important like strict vs weak typing. That survey is going to be a mess, even if you limited it to the most senior engineers. About the only thing we’ll be able to agree on is “source control good”.
i have been vibing out a godot game which is a 3d voxel map thing lately. i had it on software webgl compatibility output and just demanded it figure out what the slowdown was on the simulation over 3 days of back and forth. it turned off backfaces and invisible geomeetry and locked render resolution at 1080p and built in diagnostic telemetry and chopped up the level where doors separate rooms until now, still with no hw acceleration, the framerate is in 60fps with some frame drops, it started at 12 fps.
brute forcing performance optimization for a personal project by having only handwavy tricks ive read about from online and repeatedly setting targets until the magic box gives me what i know it can do means that eventually the game did what i wanted with those. yes baked lighting was needed.
next, i an going to aim for 60fps on a windos xp box just for the challenge. we had quake and such run blisteringly fast on weaker boxes than those, surely my empty museum tour of a 3d game can do the same with the graphical voxel and crappy looking models it does now.
real pro grade work is beyond vibing out and not caring about poor quality. but it scratched my itch and gave me the computer software i wanted and with way less effort than doing it for realz would require. i love programming itself on its own time (going to learn nasm and someday do a hexagonal architecture with actor model in idris just to add up my nerd trophies together) at times, but using llms lets me do my random hairbrained ideas much more easily for what i put into it than not existing in the past.
I don't know. Claude Opus 5.5 still makes rudimentary mistakes all the time as I use it.
More than that, it cannot go beyond a certain level. Like a dog that can only do dog-level actions. I've lost count of the number of times I step back, take a critical look at what it has been hacking away at for hours, which would seem OK, but can be replaced by a much more effective architecture.
For the foreseeable future, a human is still very much needed in the driver's seat.
Individual agents do suffer from making dumb assumptions and tunnel vision. But that can be improved significantly by using multiple agents, with some assigned to just review and critique the workers' outputs. Personally, I've not found this useful because there's just so many layers between you and the work. Maintaining intent is much harder.
> I don't have the design vocabulary to be specific and articulate what I want. It's a lot of trial and error.
One thing you can do is, once you finally do get the model to do what you want, ask it how it would have described the change itself, in a way that would have skipped the intermediate steps. In my experience, this lets me more quickly explain what I want in the future.
People preached the wrong imperative before "or get left behind." Turns out they really meant "get in now and re-evaluate everything you think you know every month in perpetuity or get left behind." I'll be here typing the code until the dust settles, and do my thinking then.
Agree. Maybe such soft skills will become a lot more important soon, for instance also the ability to translate feelings into useful information, since LLMs cannot feel as humans do.
Personally I've been very hesitant to deep-dive into putting together the perfect AI-workflow, I've been sticking to just using the conversational/chat modality for pretty much everything, usually that gets me as far as I need. It's just really hard to justify spending time on it when it all potentially gets turned upside down next month again.
It seems unlikely a few rounds of rl are going to fix the following problem: most things are more complex than they seem on the surface, and decisions to be made. There are always trade-offs.
I did the same in about the same timeframe. I created a nice little hierarchy of "when writing tests, read docs/test-patterns.md" type logic. The idea was to put all of my personal heuristics about coding in a "lazy loaded" documentation directory. I'm able to generate Java code that is nearly the same as what I would generate if I was coding it myself. I haven't deleted my documentation or anything. I see people making this claim, did you literally delete your coding documentation or something?
It seems like, for example, some people like to use var and some people like to use the full type name in Java declarations. The LLM can't know your preference upfront, you need to tell it. When people talk about shitty vibe-code I assume it's because they didn't do these steps.
I could see (but wouldn't count on it) the argument that the current models are so good at recognizing patterns in the code that in a mature well-organized codebase they don't need to be told what the patterns are, but I haven't bothered with those experiments yet. If it ain't broke don't fix it was my thinking.
We have graduates come in and master development with agents in a few days. There's really not that much to it. There are a lot of people out there trying to make it sound difficult. Maybe to make themselves feel better, maybe to try and make money?
Master development in a few days? Unless you are mastering something extremely basic like, I don't know, WordPress websites, I highly doubt that. Learning how to do proper architecture and various test systems requires a lot of critical thinking skill that comes with experience far longer than a couple of days, unless you are just a meat proxy for AI, and adapt AI's opinion as your own, with no critical thinking applied?
What's future proof is to know what's important enough to you to burn a lot of tokens (money/usage) on and what not. And to communicate this to the AI so it spends your money where you want it. It's future-proof as long as you have money to spend on tokens. (similar things apply on a company scale)
> The only thing that seems relatively future-proof is just being able to communicate clearly what you're trying to do. Not how to do it, or even what success looks like.
And knowing the right thing to ask for and not giving it wrong assumptions. And knowing, somewhat, how it interprets certain words and phrases.
Huh, just like any tool, you have to know how to use it
Well, that’s true to an extent, but sometimes you need to ”play” a bit before you (or the model) can “work”, if that makes sense. And it’s hard to know the right kind of “play” without experience in the area, this is something the models can’t yet do.
Yeah I remember when the original post about this came out. Def not recent. Though I think their point survives in that they didn't exactly RL on this.
I agree. I don't necessarily "trust" Anthropic and OpenAI when it comes to CC/Codex respectively, but I respect that they have immense internal resources and telemetry to be able to understand what features move the needle and nudge traces in the right direction. I don't understand how non-labs judge feature inclusion? Just vibes?
I just tell everyone to use Fable 5.1 for everything at this point. Astra is unfortunately a dud, I'm sure they will try to fix a bunch of it with GPT-6.1 but OAI has had this issue for awhile now where every other generation has some sort of strange tic, or reward hacking issue, or something. It's almost like they are balancing the RL on the tip of a needle.
Opus 5 has issues too, comment-slop, claude-ish, etc.
5.1 on the other hand can seemingly do no wrong. Easy to work with, writes human-level code. Expensive, yes, but even at Low effort it's well worth it.
Quite good as work horses? To me Luna is the work horse and Sol and Astra are prancing thoroughbreds. If I use Sol or Astra for anything other than curated reasoning and planning I will burn through my usage limits in an hour.
Same. I use Luna for research/scout/test subagents, Sol for coordinator, Terra for delegate, and Astra high for review and simplify. Even with Astra in there, it’s fresh context, and I’m consistently amazed by how far I can stretch my $20 subscription with really good results.
Why do you use Sol for the coordinator? Does it also do planning and design? Because otherwise you are cranking Sol turns on very mundane orchestration jobs.
I used to do this but recently I switched to having Fable 5.1 spawn forks of itself rather than Opus subagents. Yes it's more expensive but you don't pay for reads that already happened pre-fork, and you end up doing less rework since Fable agents are just much smarter.
I agree with you on this. Opus feels tedious and it cannot be stopped from doing change-narration comments, but Fable feels like a real collaborator. I am usually pretty happy with the code it writes.
- Occasionally has strange tics around asking for permission for obvious next-steps, implied actions, etc.
- It's very expensive, both in terms of tokens and % usage on subscription plans.
- Relatedly, effort level is unintuitive. Sometimes it seems like higher effort levels are actually cheaper due to not under-thinking and needing to correct work. But other times they are overkill and send the model into rabbitholes.
That said, it's fantastic as a code-reviewer or "hunter seeker". It's better at finding bugs than Fable and "Get this well articulated task done single-mindedly" is an Astra-shaped task.
I gave up on fable 5 after I asked it to critique my PR and it spit out a page of complete nonsense technical jargon. Like, to the point that I had to review the feedback with other models and try to parse what it was saying and ultimately it wasn't even right. Compare to Astra and Sol where I can almost forget there's a model and just speak/read naturally.
I think I should give 5.1 another chance but I am just so triggered by the way it talks after spending so long battling fable 5.
Also I'm starting to wonder if the latest round of models have finally saturated for my personal coding needs. I mean obviously not for taste and judgement, but those barely seem to improve with model generations. For just spitting out a 1000-line feature I've vaguely scoped out, Astra feels basically as good as I need.
Interesting, in my experience Astra is a marked improvement over both Sol 5.6 and Fable 5.1. Its output feels a lot more natural, and it is just less "dumb." But individual experiences may vary.
It's a great model and you're right it does feel quite natural at times while Fable 5.1 still has a claude-ish shape to it. Unfortunately I just find that it's not reliable enough as a daily driver and ends up performing specialist tasks rather than being the primary pane of glass.
I'm sorry but the whining over LLM output styles is embarrassing. Do Claude and GPT models always respond in exactly the way my most articulate coworker would? No. The overused jargon is absolutely annoying. But these things aren't my drinking buddies, they're professional tools. It's not _literally unreadable_. It's just not ideal. Most of my tooling is "not ideal". That's okay. That's what I'm paid for. I just work around it.
For me I added some instructions to speak clearly and it helped marginally and that's fine. There will be a new model out in a few weeks where I'm sure they've laser focused on this issue since nobody can shut the fuck up about it. The same thing happened with GPT if anyone can recall the ancient period of 4-6 months ago.
The “whining” stems from watching the communication style obviously degrade, and it’s a huge problem for people who want to use this stuff to build and instead continually fight the tools.
Like so many other products, people are moving too fast and shipping things that move the ground under people’s feet needlessly.
All this while we’re beaten to death with the marketing and false promises, and the broader consequences (ex: layoffs, stress, crazy expectations) caused from all this.
Obviously what Anthropic and co have built is amazing and people aren’t losing sight of that. That’s actually the key part of the frustration.
So no, this is not whining. This is the natural response you get when you make bad product decisions.
If you don’t want to get feedback, don’t sell products.
Opus 5 is literally unbearable to read for me, but more importantly, it is much worse than 4.8 and that one was already annoying. The more they train it on its own output, the more pronounced its idiosyncrasies become.
It is not the jargon per se, it is the style. Other models even in the same family are not as bad, and I still have to babysit it, so the voice matters.
I've had to read difficult things at work in my life, like C++ error messages, but I think Opus 5 is the first time I felt like I was being poisoned by my terminal. I keep having to leave and go use the office gym.
If you don't find it nearly impossible to read dozens of paragraphs of this shit all day, I assume you're not reading it. I can't even skim it anymore and get the gist of it. That's the thing, I don't really care how it writes as long as I can quickly digest it - at this point I would actually prefer it spoke like a caveman or a toddler.
These things do work that previously would have taken expensive engineers months to do, at much lower quality, and what's our response? Ti nit pick on it being more verbose than we'd like?
Just like with humans, when someone is being too verbose, there's a skill to just filter through the noise and focus on the important parts.
This feels no different when I use an AI.
But I guess it's a good sign that we've from complaining about 'AI slop code' to, 'I don't like how it speaks to me'.
They're absolutely best in a chocolate-in-my-peanutbutter situation - I use them to adversarially review each other's changes and ping pong the branch back and forth - A makes changes, B reviews, B makes changes, A reviews. Together they're much smarter than either individually - anthropic models tend towards more creativity, openai models can get bogged down in the nitty gritty.
This is basically the answer, they generate A LOT of synthetic task rollouts in parallel, then use RL on the resulting reward signals to improve the model. Add scale to this and you have a Fable class model.
keep in mind fable = mythos which as been "done" since february. so the gap is not 2 months, it's more like - techniques probably started "working" in late 2025, now are trickling down to 2nd tier labs 9 months later.
Yes it does, it just means all the companies come out with similar models around the same time. If what they were doing was completely novel, it would take a long time to repeat. As it is now each company releases a new model every few months, and every couple years the "leading" company changes.
Who are these task producers? Are you saying that Anthropic, et al delegate the RL part to third party companies that do it for pretty much every other AI company as well?
Yes they're called RL gym companies and there's a whole ecosystem of them. You hardly hear about them because their only customers are AI labs and RLVR is where the improvements are coming from at the frontier right now.
Note that RLVR is incredibly compute expensive but it's CPU as much as GPU.
I will say, if the problem is purely code you occasionally see "sparks of AGI" where the model will step back and say "hold on, there is a massive simplification we can make here...". This started happening with Fable-class models. But rarely if ever will the model push for a huge simplification at the product level.