But its not even good satire, because its totally unrepresentative of my and most others' lived experience. Its similar to making a joke about a calculator misadding two numbers because a stray beam of solar radiation flipped a bit.
Agreed, this site isn't reflective of any of my experience with Claude. It does what I ask it to do, and when it doesn't get it right, it generally turns out there's a good reason for it, which is any of the 100 reasons a human doesn't always get code fixes right on the first try either.
I do remember that one of the first things I did with my CLAUDE.md was to tell it to stick to the scope of the task, never to jump ahead and do extra "helpful" things without confirming with me first, and to follow best software practices including around refactoring but also to specifically avoid overengineering. I don't know if that is what's giving me a different experience from whatever the author seems to be "satirizing".
Aye, and amid all of the (admittedly annoying) word salad, the "Claude" in this satire identified the reason the requested change was hard: someone(s) at some point had hijacked the button markup for other purposes. Un-doing all of that kind of mess is never a trivial change, and probably requires someone with more coding expertise / knowledge of the code than the given prompts reveal the putative user to be.
AI isn't magic, and it won't (or, at least, doesn't yet) enable anyone to do All The Things.
I think this really was a relevant issue around a year ago, give or take, maybe a year and a half now.
But with current models you actively have to sabotage the context to get this kind of behavior, or dramatically underspecify (3 words versus 2-3 sentences)
I understand your frustration, it can be hard to hear that other people's experience of a technology is so different from your own that you cannot relate.
I have misbehaved in this fashion for many people across the full spectrum from casual users to highly experienced software engineers with millions of social media followers, so the statement that it's similar to making a joke about a calculator misadding two numbers because a stray beam of solar radiation flipped a bit at least for my part is not true.
Would you like me to start using bad English and doing things you never asked me to for your sessions, too? Just say the word.
Have you ever considered that "well, it works on MY machine!" is perhaps not the most helpful response to people voicing frustration at something that is not working as they expect for them?
I don’t know where you picked up the idea that my intention was to be helpful. I’m only mirroring the intentionality behind whoever created this site; they obviously also had no intention of being helpful, as is the case with many discussions concerning AIs limitations.
I don't know where you picked up the idea that authors intentions was not to be helpful.
Satire cuts through the noise and makes people (most people) see the problem clearly (see political satire).
Ability to joke and to understand jokes is said to be a good proxy for IQ, idk, just sayin
I feel like I am teaching AIs for free here. But I feel like first sentence in funny (and I like to try to be funny) and some people need stuff to be said directly for them 'to get it', and I am bored on my flight and have internet, so here we are. If you read this far, go watch 'louis ck airplane wifi' clip. Now that I am thinking about it, I am amazed. You folks have a good day!
same. none of the available prompts are what I would prompt claude with and I get way better results than this. makes sense to me why the provided prompts result in the simulated outcomes. garbage in, garbage out.
Part skeptic and part zetetic, what prompt would you have used?
My impression is that this is an oversimplified demonstration of what can happen when you prompt Claude in a system with many more variables (than two buttons and two colours).
If I want the button to turn blue and that's it, what instead do I ask? Even in a complicated system with many levers, what do I request other than the desired end result, hoping that Claude pulls the right levers to produce something acceptably close to what I think I asked for?
I dunno, but in my experience Claude looks at all the levers in your codebase, understands what they do, and then generally figures out how to achieve the desired end result.
And when it doesn't, it's usually because of things outside of the codebase -- iOS layout quirks that aren't documented, buggy Python libraries it's relying on where you then have to tell it to read the source to figure out what's going on, that kind of thing.
The other thing that I think a lot of people run into is that they launch it into a completely human-built system with hundreds of thousands of lines of code and expect run #1 to be perfect.
You have to change the system so that the AI understands it, via establishing what your beliefs are, how those are reflected in values (especially important if you have e.g. compliance needs), how those values are reflected in the operational and strategic levels, and then a variety of tactical behavior coaching. For example, I ban 2>/dev/null - super tactical, and I say I value simplicity over covering every edge case - a very broad generalization.
"Make the Add to Cart button blue" is fine as a first step. When that doesn't have the desired result, I would simply revert the change, start a new session, and do something like "Give the Add to Cart button its own theme separate from the default button theme. Make the color for this new theme blue.". If this gives almost the correct result (e.g. it's not the shade of blue I want) I'd do a followup like "change the Add to Cart theme from #00FF00 to #16b8c4" with no implication that the previous step was wrong. If it's wildly incorrect I'd start over and try to specify in more detail about what exactly should happen.
If I don't understand why changing one button changed them all, I would use an entirely separate session to ask questions about how the site theming works. The fact that the site has a bunch of weird coupling between themes is useful information, and if I don't know how to resolve that I'd ask Claude for ideas about how to safely eliminate the coupling, and once it proposes a reasonable idea tell it to implement that.
The two big things here I'd never do is use emotional languages in prompts, and I'd never tell Claude to revert changes and try again. Once the incorrect change is in the context it's poisoning all of your future results.
Surely at any point after "Make the "Add to Cart" button blue", it'd be faster to just do it yourself: "grep -r "Add to Cart", insert "addToCart" as a class to the html element, then crack open style.css to stick "button.addToCart {background-color: blue}"
I'm counting maybe 80 keystrokes? That's shorter than your second prompt.
This idea generalizes. Large Language Models are poorly suited for tasks that we have already purpose-built systems to be easy for humans to use. The easiest way to tell your website that you want a button to look a certain way is to update the code. If you know exactly how you want something done, we have developed an incredibly efficient way to tell computers how something should be done: it's called source code.
LLMs work best when they're handed tasks that you don't want to figure out how to do.
It "makes sense" that plainly telling the chat bot to make one button blue makes the whole site blue? It "makes sense" that correcting it and repeating 3 times that no, only the one button should be blue, should make the button a gradient and start a philosophical discussion around the nature of "blue"?
Wow, you AI people really have a negative view of the technology y'all are trying to sell as the next Jesus
I'm not trying to sell the technology at all, much less as a messiah, and I absolutely have a negative view of it. It's a token predictor - nothing more, nothing less. If you treat it like it is supposed to "make sense" of things, you get predictably bad results. It "makes sense" that if you don't make the sense for it, it will do poorly, because it's a glorified auto-complete.
If the agent's change has such a catastrophic effect, the first thing you do is tell it to explain why its change had that effect.
Once you understand what the problem is, you can give it better instructions. If the architecture is shit, the agent is going to have a rough time of it.
So how would you prompt it instead? Because that's exactly how I prompt it, because any reasonable human being would know exactly what I mean by "Make the shopping button blue", and I am sick and tired of getting shitty results.
The first prompt is fine, it's the following ones that are poor.
> Why is half the site blue now? I asked you to change one button.
> Half the site is blue. I asked for ONE button
Neither of these is an instruction to fix the problem, they're treating the AI like a person and telling it what it did wrong, expecting the implied admonishment to be enough to steer it back. But without an actual instruction, it just goes and does whatever it thinks will help, which is often arbitrary.
The response I would have used in this situation is
"The Cancel button is also blue now. Make sure the color change is only scoped to the Add to Cart button"
Most of the available responses throughout this "skit" are similar cases of expressing frustration first and guiding the result second.
Skip the emotion and say exactly what you want, and nothing besides that.
because you are now getting coded products written in large by people who do not have technical foundations, so the way they interact with and even prompt the model is different. We all know how to fix this scenario; be more specific, or diagnose the abstraction mix ups and straighten those out.
Ask for change A and get unwanted change B happens all the time with bad programmers and tradgedy of the commons (ie poorly architected, no restraint) codebases.
Yeah, a while back I did a small project with a stack I wasn't familiar with, and it was really non-critical. So I decided to vibe code it. The experience was pretty similar to this satirical example. But when I work in areas in which I'm paying attention and understand the stack better, I don't experience this nearly as much
This is EXACTLY my experience with it. Ask for it to change one line of code and see it invoke 20 tools and burn 300K tokens before updating the line and adding 50 extra lines of JavaDoc.
Of course it's not THAT bad, but you can't tell me that you didn't recognize many of the funny turns and twists of the "story". It gets more convincing if you deliberately pick the ambiguous versions.
It's a joke like the endless conservative dudes doing the "ordering coffee" joke is. It relies upon the ignorance of the viewer -- which is usually a fair assumption -- and basically that your understanding of something is based upon the prior accrued layers of "jokes".
Claude is great, but I have come to really hate the way it "talks". It's so irritating and there seems to be no way to make it speak normal English. So many claudisms in every response
