wait until CTOs are forced to stop pretending the world is all sunshine and rainbows and that much of their tech stack has adversaries fingerprints on it
AI is as good as the prompt. It's a simple as that. If you don't foresee everything and anything that can go wrong AI isn't not going to magically fill in the blanks and save you. So in projects that already had good test discipline, well written specs and tickets and very clear expectations, coding guidelines and documentation it is relatively easy to fit an AI in for guided work. You still need to check up but chances are good that it will usually work as expected.
But any kind of mistake, any kind of ambiguity and you'll be in for a wild ride. It usually takes a while for the rot to set in. I've had to revert - more than once - to many commits prior to undo a bunch of work that had me completely puzzled as to what the reason that particular fork was taken to begin with. Usually I end up figuring it out but definitely not always. In some cases this kind of wild modification fed back on itself until the situation was beyond rescue. I'm a bit quicker with such reverts now than in the past, for the first year or so I would try to save what could be saved, now I just restore and try again.
But if you're not sharp when evaluating what gets done there is a good chance that you'll miss it and then it can pop up way later.
What I've noticed as correlation (but I'm not sure if that is the root cause) is that non-local effects and stack effects are particularly hard as is everything that requires a working model of the world.
edit: this comment seems to make people uncomfortable.
I disagree that the prompt is the most important thing, I'd say other things like the model you use, a clean codebase, actually reading code and using discretion about which changes are accepted are far more important if you're using AI for work that needs to be correct. LLMs still make significant mistakes in design especially.
Harness, prompt etc etc are all peripheral IMO, though people like to think their super long prompt, custom harness and MakeNoMistakes.md skills are what contributes to their special skill with AIs, but really models are just improving.
You can just trust the AI to generate a perfect answer I suppose (and why are humans slowing it down at all if so?), but if you don't expect that yet you have to put in the work.
I think I noticed claude uses the shape and style of the existing code base to generate the new code. When reviewing claude's output, it looks familiar and the patterns are similar to what I would write.
I'm not sure who is studying this yet, but there has to be a statistical relationship between three factors of
1. intelligence of the human: the design patterns and architecture is already modelled in the mind, steering the AI becomes easy to know what to say next
2. quality of the codebase: the LLM can pick up the adjacent patterns and write code that looks and works cleanly.
3. quality of the prompt: the prompt keeps the AI focused and ignores the trivial side effects; keep the AI wandering away from critical path work; sometimes the AI can't make the decision what call site is next
In summation, what if claude is just a mirror pointing back at us? It certainly would explain wildly divergent opinions of opus 5.5
There are definitely nuances, and yes of course the prompt must make some difference, but I'm not convinced it makes as big a difference as some think, particularly a long prompt or complex setup of harnesses etc surrounding the LLM model.
To me the order of importance would be:
1. The model - if you use a terrible model you'll get bad output
2. The codebase - this definitely forms the model for what claude outputs to some extent.
3. The human operator, what they accept and reject etc.
4. Things like prompts, skills, harnesses etc. The quality of the prompt.
So I think our positions are really just a different ordering of what is important rather than fundamentally opposed.
Re Claude being a mirror, if it is it's a particularly distorting mirror which often changes things in subtle ways and is determined to make people feel smart and addicted to its output - it has been trained to do so after all. I really don't think a mirror is the best analogy, because it's too independent and seductive to be a mirror.
There's a good example of the sort of thing they produce without good guidance as a dead comment on this thread as a reply to yours. Encouraging, positive and superficially plausible, but without any actual meaning or intention. The equivalent of that in code is what we need to be wary of.
I think the environment you set up for the agents is just as important as the prompt. I've been working full time on a claude code developed project for the last 10 months, and it seems to become better and better at developing the right thing. It has the tools to research whatever context it needs, test it's output in a relevant way, etc. It doesn't by itself suggest complete refactoring, or big performance optimizations, so that's something you have to prompt for, but it will collect all the evidence and context to make a proposal, implement it, and test it.
Every single case of AI slop code I've encountered in a shipped product is a result of a complete lack of proper, professional workflow on the part of the human who decided to ship it. People are using AI as a reason to continue to justify their lack of responsibility for what it is they're doing.
And on the other side, I have seen AI produce exceptionally efficient, productive and effective code - the opposite of slop - when it is treated like a junior developer who will do any idiot thing, except when it is mentored by an actually experienced, professional developer.
The source of slop in AI code is not AI, its not the training data - its in the actual workflow of the human involved in its production.
The most effective use of AI is in an environment which already produces high quality software through proven tooling and methodology. If your workflow is broken, your AI isn't going to fix that - except by way of giving you a lot of slop to push through whatever pipes you're already not getting flow through.
Two tracks: code I don't care about where AI can be used and core stuff that I do care about and that needs to be production grade (real time stuff). The core stuff I write myself, the auxiliary stuff is almost all automated. This doesn't get me to 'maximum speed' but it means I keep total control over what I produce and AI can review it, but read only, no changes and it is set up just to flag things, not to suggest particular fixes.
Good one, that seems about right by my read of things.
Review is more important now than ever. My workflow has 3 review stages .. after analysis/specs/design, just prior to starting development (intake), and in the quality stages of testing and so on.
Well, there's also an after-release review, but the sales/marketing guys do that as part of their followups.
The code AI generates is fine, it's the context on a codebase that's often missing. Generating duplicate code because they didn't see that a Class already existed for the same thing. Or having no clue that changing a method will fix A but often missing that it will break B.
Use an iPad. For the price of a duo you can purchase an iPad Air and an iPhone, even more screens, you can run even more apps next to each other. And the iPad can extend its screen to a normal display if you want to run even more apps.
You are completely ignoring the importance of pocketability and having something with you all the time. My old M1 iPad Pro remains severely underused eating dust because I'm never carrying that thing around with me, including in my work bag.
I had a Lenovo thinkpad with OLED before switching to a MacBook Pro (first M1 Max, now M5 max). I don’t really mis the OLED display, it’s not that big of a difference (according to my 55 year old eyes…) the thinkpad also had a touchscreen, never used it, only by mistake. I would not pay extra for a touchscreen or OLED on a laptop. The photos on the apple website are not convincing, playing a keyboard on a vertical screen looks very uncomfortable.
Also don’t understand the iPhone duo, I’d rather have a normal iPhone and an iPad Air. Cheaper, and better for mobile use and for couch media consumption.
I think the real weakness of democracy is that governments can’t make difficult changes. The current government wanted to but both the right and the left did not accept those changes. Similar story as France, a big need to change, but everybody expects others to pay, not themselves.
Today, it’s always either the fault of the other party (who never had a real chance of making big changes), the rich (but if you tax them too much they leave and you’re left with no economy, see France), or the foreigners.
It's not just democracy, but also huge population growth and population clustering. People moving from the country to the city as jobs increasingly moved into the city.
And yet we can and we should still make distinctions when we talk about those subjects
Estimates for the death toll of US and British strategic bombing, for instance, range from 300,000 to 600,000 in Europe and around 330,000 in Japan (including the atomic bombings)
Now compare that with 12–17 million civilians killed deliberately by Nazi Germany and the around 6 millions killed deliberately by imperial Japan
reply