These models have no self awareness. I just spent days having it write some test framework with very clear goals and it was highly confident understanding exactly what I wanted only for the whole thing to be complete bullshit, one remark from me about the work being useless and it goes “oh you’re absolutely right that’s on me”…
Same, it’s so smart but so dumb. I explain an architecture change to it and write examples of strongly typed Go code and how to store structure, it agrees and then proceeds to write some untyped string map ball of mud that has 900 specializations.
But maybe that’s on us, AI doesn’t care about all these special cases, it’s not debt to it as it will simply read them
all when making changes. We’re obsessed with quality and what code is supposed to look like but those are human standards, AIs evolve to look at this complexity as a single picture, they can simply see through it so what is spaghetti code to us is merely some code to them that works as it should and is efficient. It’s interesting we can see how the two things drift apart, you would think at some point AI generated code should explode but it hold together unreasonably well in most cases…
Use hooks to run a bunch of review steps after -all- code writing steps your agent does and give it the exact review criteria you just described (via git hooks, or your agent harness of choice's own hooks eg https://code.claude.com/docs/en/hooks or AGENTS.md). So after every step where the agent writes the shitty untyped string map ball of mud, your orchestrator/main thread agent that spawned the code-writing-subagent spawns a follow-up review agent automatically that is given that output, your prompt that explains what well written Go code looks like, and even the sample/golden-path code of your choice to use as a style guide.
Each time you encounter a shitty thing you hate, add a new 'review type' / 'thing to watch out for' and just ask your agent to add it to your hooks for you. This works well with Claude at least.
I have about a dozen or so hooks that run on every integration branch my agents write that review for all sorts of things from correctness to spec, performance improvement opportunities, modularity, analysis of any dependencies added, 'definition of done', UI/UX, etc.
I recently told Claude it should run the whole suite of reviews twice. I will probably go on and proceed to having it run like 5 times eventually idfk.
But the more you start asking your agents to modify their own behavior, using the native solutions offered by Cursor, or Claude, or Codex, the sooner you'll start to feel better about the results.
I'm having bad results with SotA models. Some questions, if you're up for it:
In your workflow, who implements the review feedback - the review subagent or the code-writing-subagent?
Do have a baseline styleguide (like Google's Go style guide) for the review subagents, or is it entirely the subjective things and specific corrections? I remember 6 months ago it seemed like piling general "good taste" code advice into AGENTS.md was considered bad.
Do you move between harnesses or have you gone all in on claude? I've bounced between claude/codex/omp, maybe to my detriment.
Not asking for answers to these questions, just frustration dumping:
The biggest things I've struggled with are models having taste. For spec writing I was having a lot of issues with them making statements that were interpretable in a superposition of ways, eg "we'll do XYZ with entities that support and need it" when there's 3 possible entities and the model hand-waved at exactly the wrong tokens.
I added AGENTS.md guidance + memories to be unambiguous (with short but good examples) and "no coined shorthand". Now I'm getting a marked increase in specificity, but it's places that don't matter (claude explaining existing code to itself). I'm having difficulty controlling the spew of new text, but feature writing/research still gets fuzzy and lazy around the difficult underspecified aspects of the problem/feature.
Do I just keep dumping examples into reviewer subagent context and have them rewrite and simplify the research subagent spew?
I repeatedly have "Risks and gotchas" sections have a whole paragraph dedicated to things that don't matter, and then a single sentence bullet point that's actually a huge problem when I dig into it.
Do I have a generic hook to make subagents to review bullet points and size them commensurate with impact? If I tell them to "have taste", do they have taste?
I'm just trying to get good code done. I hate these things.
I tried writing a simple 20 page paper with the latest Claude model over several weeks and that thing got ripped apart by legal and data science (for good reason), not going to try that again. And there I had all the numbers and analyses down and verified, just the writing itself by Claude was subpar.
Can’t imagine the utter garbage that Claude will produce when you give it a simple prompt and let it write thousand of pages.
Kind of makes me wonder about all that AGI talk if these models can’t even the work of a good PhD or postdoctoral student.
Yes, the reCAPTCHA widget is localized[1], but that doesn't automatically mean every term is localized properly.
But the underlying point of the article stands. This is similar to the mythical statement that some park ranger once stated "it's very hard to design a trash can that no bears can use, but that every human can."
reCAPTCHA, like CloudFlare, benefits the wider population of the internet to the extent that everybody can use them, but there are edge cases where some outlier users get locked out of necessary internet infrastructure because they can't grok the ask, can't correctly identify the options, or because their computer / IP is being abused by bots so they are in the splash damage range of the bot checkers.
Just use mTLS via QUIC, it’s standardized, secure, future proof and has implementations in tons of languages and supports proper certificate checks and that whole ecosystem around it which they apparently tried to reinvent. It’s such a great protocol for these use cases, I don’t get why it’s not used more.
I don't think the consumer ecosystem is mature yet.
Adding a cert to my Android phone was a huge pain in the ass. Most homelab apps like Jellyfin don't support it. The homelab space is so buck broken it assumes i don't have a public IP address to just expose things without going through the convoluted VPN setup.
I don't see how mTLS would work for Radicle or other decentralized systems because, as far as I understand, both parties need to have their certificates signed by a shared root.
Lots of families there every year and older folks too, it’s actually a great date for a lot of IT folks as there aren’t many weeks when companies are effectively shut down, maybe even only this one week.
The more interesting question is what happens once we replace manual labor with artificially intelligent robots and we arrive at a place where 99 % of human labor got eliminated. How do you maintain civil order? Every country is built on the premise that you have to “earn your keep” but what when that’s no longer true? Think about it, no labor, only capital. What will the people owning the capital demand of people that don’t own any? They won’t be useful to them in any way. I expect something like forced reduction of fertility for regulars to depopulate countries, while hyper rich individuals will extend their life spans and become immortal. And I don’t mean they will really survive but someone like Elon Musk will have their consciousness mapped to an AI and will instate that AI as their official heir. That sounds crazy but they will totally go for it just listen at what Peter Thiel is spouting in terms of immortality and becoming godlike. Hope people will have enough left to say to prevent that but right now it feels we’re going hardcore towards that scenario.
There have been transitions like that before, but they were slow. It took about two centuries for the US to go from 90% farmers to 2% farmers. Some countries deliberately put on the brakes. India has done this. India is still about 50% farmers. India uses subsidies and discourages farm mechanization to prevent massive unemployment.
There have been fast single-industry transitions. But not that fast. It took about 20 years for printing to go from a large craft with a long history to a minor employer.
Not everybody got out in time, but there was warning. Same for longshoremen when containers came in. Computing in general took about 50 years to totally replace the clerical plants of the 1950s. Railroads in Britain took a few decades to really get rolling, and in the US, they made it possible to use the vast interior of the US.
The AI transition looks to be both broad and fast. That's harder for society to digest.
Underlying assumptions of politics and economics break.
> Elon Musk will have their consciousness mapped to an AI and will instate that AI as their official heir
My guess is that sort thing will become common. We'll probably need some lefty policies to distribute the wealth. To each according to their needs and that.
What you mean by consciousness and immortality - what you want to preserve indefinitely - is the waking experience of being alive.
Strangely, we don’t mind that we weren’t here in 1737 BC, and we don’t mind and we even crave plunging into the nothingness of deep sleep each night, but we have a big problem with not being here tomorrow.
But experience might not be encodable. Can you encode pain? Could a computer ever be programmed to feel pain? Not detect damage or report on it, but feel the pain associated with it?
What is “the aroma of coffee” or the “sound of a violin in middle C”? We can define them objectively, in various ways (some having nothing to do with each other).
But if the experiences we have are not encodable then we can’t map consciousness onto a medium. It must be experienced (whatever it is/means - nobody knows) in the here and now.
It’s not magic, but it might not be something that is printable to disk anymore than the “brisk inhale of canyon air” that I can play that back and have that experience, is printable to disk.
It remains to be seen how the experience of “the aroma of coffee” works but I expect there will be progress there. That stuff probably came about through evolution as a kind of situation summary so we can make decisions critical to survival and replication like whether to grab coffee.
>we don’t mind that we weren’t here in 1737 BC
This is true. Probably just an evolved instinct not to die. Even so I think it would be cool to live on in some form.
They will settle for what’s available at the time, I dont think it makes sense either but they will come up with something that is good enough for their purposes and as long as the copy parrots the original well enough it will pass.
reply