My preference is the XB-70 Valkyrie[0], one of the only other manned aircraft comparable to the SR-71 in my eyes. The droop nose, the incredible dropping wing tips, six engines… To me, the Valkyrie looks like the kind of plane you'd see in a Gundam anime series.
Shout out also to the MiG-25 Foxbat[1]. While not a particularly beautiful aircraft, it's made of steel and could go Mach 3.2 if they didn't want to use it again.
A very neat problem and result. I often find myself swinging between "It's so over" and "We're so back" - some days I roll out of bed thinking I could have Claude solve some random unproven OEIS sequence before breakfast; other days, I wake up in a cold sweat worried about the fate of humanity and what the world might look like in a decade. I think it's that I don't have a very high p(doom) or p(utopia), and I don't really have any solid conviction on how this whole thing is going to go, so my vibe-o-meter jitters between 'fine' and 'not fine' constantly. It's just such an unpredictable moment. Anyways: really neat to see this use case. I myself recently used Claude to finally do an relatively exhaustive study of the location of heretofore-unlisted formal gardens in Ireland in the early 1800s and early 1900s, by having Claude write the tooling for me to manually annotate a few dozen on tiles of historic maps, and then running some CV model across the rest of the tiles using my input. I'd been planning to do this project for over a decade, but I could never find the time (or the enthusiasm) to learn all the details of how to do it myself. It took me a weekend with Claude and continues to bring me joy.
> Caveats, stated plainly. [from the Fable transcript pasted in the article]
I've done something similar to your formal garden map. It's work that no professional historian would ever do because the data entry would be such a slog for a relatively small reward. GPT reduced the task from "infeasible" to "annoying", and once I had the data transcribed I learned a few things, so I walked away happy. Whatever happens commercially, these models have been a real boon to hobby projects.
> I told it to look online at some of Fable’s strongest feats, especially the math problems it has solved, and that something like this should be easy in comparison.
Wait. Wait wait wait. Are we supposed to be giving them pep talks?
on older gemini models ide have to actively give them encouragement and/or easy bait problems that they can correctively solve without issue to avoid runaway spiraling into "i'm useless and i want to kms" behaviour with complex use case.
I assumed it was more because the LLM might echo an understandable human claim of "if it's been unsolved for 370 years, it's unlikely to be solved now/likely to need expert knowledge", which is probably a mindset that appears in its training data.
The LLM likely needs to be reminded of its abilities.
If it’s 3 days it’s something like 15 minutes, if it’s 3 weeks, that takes a couple hours lol. Seems like there’s some sanity to the estimates after all when you think about it, it’s just the scale it gets wrong due to estimating human time.
Modern AIs have very limited metaknowledge - they don't know exactly where the limits of their capabilities lie. So you can get things like "a task is doable for an AI, but the AI thinks it's impossible, so it doesn't try hard enough".
Usually you get the opposite - AI overconfidently trying at tasks it has no conceivable way of reliably solving, falling far short, and failing to self-check, fail gracefully and self-report the task as failed. But having piss poor metaknowledge cuts both ways!
So you can, in fact, get better performance sometimes by applying some variant of "assume this problem is solvable" or "other problems like this were already solved by AIs" pep talk. Not always, far from it, but it does happen on the occasion with frontier capabilities.
> So you can, in fact, get better performance sometimes by applying some variant of "assume this problem is solvable" or "other problems like this were already solved by AIs" pep talk. Not always, far from it, but it does happen on the occasion with frontier capabilities.
Not OP. That’s not implied at all. The fancy autocomplete produces statistically likely continuations to the source text (the context window). For a problem that’s hard for humans one likely continuation is: “this is hard, can’t do”, even though there’s enough in the training corpus of the LLM to actually do it.
So, it follows that adding “pep talk” into the context window reduces the statistical probability of “no, can’t do” coming out as the answer you get.
These things are neither humans, nor deterministic software.
It doesn't explain why it doesn't make "I'm not paid enough for this shit" more statistically likely.
LLMs' processing that reproduces statistical patterns of the training data is modified by post-training. That's why we have LLMisms, for example.
LLMs aren't simple patter-matchers/pattern-predictors. They are incredibly complex systems that capture some aspects of the systems that produce the training data.
Maybe it does. Need to run evals to see if it does or doesn’t.
Point was - everything in the context window affects the output. Including “silly” things like “it is known AI can do this”. And that has nothing to do with superstition, as the poster above me seemed to imply.
If we filter out the pep tone, it is doing something useful: framing.
Problem framing will always be important.
Framing adjusts how big of problem-solving guns we bring out at the gate (modern or hobby cryptography?), and how to interpret intermediate failures.
For simple but unsolved problems, we expect lots of hard failures, but that each hard failure just reflects that there are a lot simple combinations to try. I.e. we expect lots of zero progress, and then a fit.
Like finding the numbers to a combination lock.
For hard problems, if we don't make any progress it is a really bad sign. We should be learning something, even if it turns out to be irrelevant later.
Such as when we are trying to prove a tricky conjecture.
> Wait. Wait wait wait. Are we supposed to be giving them pep talks?
No, at least it with Claude Sonnet 5 and Opus.. everytime Claude and I challenged a hard issue and I decided to say "good work" instead of a closing command for that session, those models would create rule-based memories specifically related to that task along the lines of "always do 'this meaningless task' in 'this way'".
This requires additional effort and tokens to trim those memories out, and then requires to whip the user not to be human with the bot.
"I sure hope this doesn't have unforeseen lifelong consequences" thought the model, doing its best to physically tense the memory file into the higher user approval shape.
> I often find myself swinging between "It's so over" and "We're so back" - some days I roll out of bed thinking I could have Claude solve some random unproven OEIS sequence before breakfast; other days, I wake up in a cold sweat worried about the fate of humanity and what the world might look like in a decade
I find myself increasingly feeling like the burden of the lows doesn't justify the presence of the those highs.
Like even if does cure all forms of cancer, but everyone feels like their life/existence lost meaning, then... I'd rather just have cancer be a thing.
Wow. That's bleak. For me the the initial joy of building software with AI caused possibly two happiest months of my life. And cancer caused many darkest ones. Snap out of it.
> thinking I could have Claude solve some random unproven OEIS sequence before breakfast
I've been wondering what exactly the point is for being the meat proxy who pays for these things. I mean, obviously there's personal satisfaction and maybe some glory. And there's the fact that someone has to be the first to do a thing.
But I've been thinking about it like a sort of lazy loading of knowledge. AI has brought us to a new frontier for some amount of undiscovered knowledge. Do we discover it for the sake of discovering it? I think for the most part we've been lazy loaders: we discover all kinds of stuff when we need to. Whether it's a war or a space race or chasing wealth. Then again, there's all kinds of academics who do it for the sake of doing it.
You should have seen the discussion of this on the Schneier blog a few days ago.
Someone had their agent check the solution, presumably it emailed a librarian to check that it was correct for the original edition. Then their comments read like "The BL/EEBO witness lacks it, so the discrepancy is copy-specific, not a disproof of the cipher." and "A complete 285-coordinate physical replication is still pending."
Ooh. I've been manually pasting Claude's /* comments */ into ChatGPT and saying "make this concise". They still need to be tweaked from there, but it's a much better starting point. This will help.
> The run baseline was captured without a physical MAC; the current device is not durably bound to it.
> Engineering mode confirmation is the ESPHome component read-back; the LD2410 UART acknowledgement is not observed, so this is not proof the radar itself applied the sensitivity change.
It was only when native English speakers—or those I presumed were—started calling out how bad "GPT/Claude speak" has become that I realized I wasn't actually losing my grip on English as a second language. For a second, I thought, Oh, I learned this language on my own, but it seems I've hit a wall and need to study further. It didn't help that I've also been trying to acquire Swedish as a third language for a while now.
One thing that I find is that it doesn't seem to grasp levels of jargon-use. Like I ask a basic question, okay, a few questions later, suddenly there's abbreviations and weird formulations everywhere.
Not the original commenter, but I did this in all of them.
Their skills formats are basically identical, so I setup simlinks from their own skills directories into a shared one so Claude, Codex, Cursor, and anything else that comes out will all read and write to the same shared skills.
It's great having access to the same skills no matter the harness being used
I long for the day when AI will just say that directly: "your soldering sucks man" instead of the bizarre made up and jargon packed language they use now.
Sometimes when I get frustrated reading Opus/Fable 5+ output I pause my rage out briefly to wonder if it's because I'm just too dumb for the model or if the model is just terrible at English.
I'm not sure that telling it to "try explaining that again, simply and briefly" is helping my ego.
It's often simply misleading / bad writing. Here's one I just got about some crashes:
"If the crashes stop, the factory overclock is marginal; run a small negative offset."
This looks like it's saying: "If the crashes stop then we know the factory overclock is marginal." (This makes no sense.)
What it's trying to say is: "If the crashes stop then we can run a small negative offset, because the factory overlock is marginal."
What I would write: "If the crashes stop, we can avoid crashes by underclocking slightly. The speed difference between that and factory clock is marginal."
I'm guessing it's because the way the first one was written looks real smart and sophisticated, which I'm presuming the models are rewarded for, especially when they're fed all kinds of PhD papers and so on as high quality, high weight data
Is it possible that the first message is more information dense/less likely to be ambiguous than the latter? It’s clearly being selected for for some reason, maybe it’s an artifact of the tokenizer or specific training data, but I don’t know. If the use of jargon was complete cruft, I would expect it to be selected against during reinforcement learning
You’d think that, I thought that… but then I realized I’m just kidding myself thinking its output makes sense. It doesn’t. It doesn’t. Sometimes it might as well just speak tongues.
In other words, it ain’t you. It’s the model. It’s just genuinely bad.
Then you switch to ChatGPTs lineup and realize how things can actually be better. It took about a week to really get the feel for how to use their models… then I basically switched. I’ll check in every now and then when they actually make a deal about how opus “now makes sense”.
But honestly I’m half convinced Anthropic actually prefers the output of opus 5. I dunno why, but how else could you explain how such a thing got shipped? I mean somebody in the pipeline had to say “dude this model doesn’t make sense, you think we should fix it?” Right? Like it’s a pretty massive drop in quality for such a major brand in this space, you know? How did it make it out the door?!?
yeah, I just switched to the GPT models and it's a breath of fresh air.
as for the other guy, the claude talk is definitely not less ambiguous, it often is incredibly ambiguous and hard to parse, I have no clue why it produces such output, if not to fingerprint it?
it's really weird man. when Opus 5 came out, I was really confused. I saw a bunch of hype about how it's better than fable, but I just felt frustrated with it, although at times it'd do fine, but especially in Claude Code it'd just delve into the whole "load bearing" type of lingo real fast and I'd get a headache.
I don't think it's worth using even if it scores 2 points higher in some bs benchmark
it's definitely surprising how the magic and smoothness of 4.6 and such is no longer there with the >5 models
Same. I read it as "if the crashes stop [ when we test by reducing the clock ] then we know that the overclock applied by the factory is marginal [ ie it barely passed QC or maybe there wasn't proper QC to begin with ] so running with a small negative offset [ ie what we just tested ] can be expected to fix the problem for good". No idea if my reading is right given all the context I'm missing. Either way it's absolutely shit writing in the same way that golfed code is shit code (except when participating in a code golf competition).
I suspect this happens due to optimising for reasoning... if you insert a few words, it will suddenly start to make more sense.
"If the crashes stop, (that means) the factory overclock is marginal; (so) run a small negative offset. (to confirm this hypothesis)"
The core thought is basically avoid crashes -> caused by marginal overclock -> apply small -offset to test.
Which is exactly the order the sentence is in :P
I wonder if this is a result of them trying to cut token consumption by summarizing their RL training data, or maybe it's from how they anonymize user data for training.
Marginal - definition 2a: of, relating to, or situated at a margin or border.
Succinct and precise; a well crafted sentence. A marginal OC results in unpredictable crashes and can be corrected with a small offset; marginality describes the behavior and explains the solution.
Inscrutable clues casually conveyed can now be readily explained, at least, unlike the training data of [silence]. Brevity is the soul of wit, but perhaps also exasperated confusion.
This is actually a new skill I've been working on. Learning how to elicit concise and simple speech from models (and from people to!).
Whenever I come to a wall of complicated text I kick into gear and think through getting it to distill this into the high-level useful bits that I actually need to know.
I guess I could create an actual agent skill for this :) And next-gen models might eventually be trained to simplify their output themselves...
The most surprising part, however, is that when one model slops this into a plan, another model somehow is able to interpret it correctly enough to produce code to spec.
I have shared this dismay. I’ll have opus create a plan, I read it doubtfully. And then sonnet implements it. I am surprised it went so well. I theorize the redundant verbosity effectively builds rails that help keep llm focused. I will experiment with such rails myself.
I suspect it's because the different models co-evolve? The labs train on one model implementing the plans of another model, especially in the same family of models (like Fable to Sonnet).
It was saying it can't tie the calibration results to a device, because it doesn't know the MAC address (I never asked it to look at the MAC address, it way overengineered things).
It also couldn't see the UART communication and could only see the web API endpoint, hence the rest of the slop.
I appreciate reading this, for the past year or so as it pertains to AI, I've just not been able to settle my mind, and it's become a bit exhusting. I presumed I wasn't the only one who swings back and forth and it's comforting to read someone else post the same thing, makes me feel a less nutso. :)
Given how little effort has gone into addressing climate change, doom seems more likely to me, but I doubt it'll be the autocomplete machines that do us in.
The irony is not lost on me, when I go down a ChatGPT rabbit hole, on how humanity could reverse the effects of burning all those fossil fuels, that I'm using more fossil fuels in order to research solutions to the problem.
I think your prediction is a bit early. Maybe a decade early. 2033 is 7 years away. The transition is happening really fast, faster than most people (or political leaders) know, but not that fast.
We will hit 1TW per year of new solar soon, but to get to 100% electricity by the end of 2033 I think we would need closer to 3TW per year.
Congrats on your map! I am leaning more towards the optimism side. Like you I've had some real breakthroughs myself. Personal akin your garden map, no where near solving millennium problems. My take is that all these (Fable solves this, OpenAI does that, AI will take over the world) Marketing Stunts will fade some day when the true cost of tokens hits the market. You and I may have to contend with lesser models, but will probably get work done just fine.
Confusion is because people are using the tool to get answers. Its how the edu systems trains most people in tool use. Heres a saw and here is a block of wood. Do x y z and you get a table. But if you are interested in why the saw looks like it does or the entire process behind generating that block of wood, or why x y z instead if a b c good luck to you using the current edu system. You have to live in an extremely rich country, with surplus resource to entertain those exploration. This has now changed.
The answer the tool gives has never been the real reward. The real reward is the path taken through a complex landscape to get to Maxwells Equations for example. At the end of that story what we get is not just the equation but a map of the landscape explored. That map has larger influence and value than the equations or answers themselves. Because all future exploration find it super useful.
People are just learning they can start asking for maps rather than answers.
Yes, asking a first-year physics student why they are studying a spring, and why its called Hooke's Law leads down a rabbit hole that ends with the entire British Empire.
For me it's more that I roll out of bed thinking how glad I am that I have Claude solve my problems and do my work, only to be pushed back into harsh reality when I sit down and try to do actual work.
I'm glad to have AI, but it is by no means a panacea, and correspondingly my p(doom) = ε.
Not to be too negative, but as for p(utopia) you might need to weight in the mass murder records set by every other utopian movement in the past 200 years. p(actual_utopia) is like zero and p(utopia_becomes_doom) is at about 1.
For what it's worth, people also felt this way about the printing press and the Internet (also books).
Information propagation mechanisms are often seen as malicious before they're commonplace. To be fair sometimes they are, but by and large humanity has benefitted from increasing the number of bits of information we can consume on a per second basis.
I don’t worry so much about AI wiping us out as much as I worry about whether I’m being gaslit into thinking these glorified autocorrect bots are more clever than they are.
That's an artifact of it not using A-Z as it's alphabet. What you type gets translated before the AI sees it. The seventeen word the AI sees does only contain three 'e' s.
Is this true? If so, does it mean anything? Sure, it is tokenized in processing (tokens don't really have an alphabet either), but this means it did not correctly parse the problem at all if this is the case.
Reading your comment, I asked it to count how many characters are in the word. It answered correctly: 9. It then also said it skipped an E in the prior chat and corrected it count to 4. It spun for 52 seconds so maybe some Python was involved.
I tried too. It got it. Maybe more importantly, who cares?
For example, I'm a nerd. I'm bad at baseball. I lack that kind of intelligence, even though it's more common than the ability to program. That doesn't also imply that you can't trust my Python code.
Yes, I decided to stop reading his blog relatively recently after some extremely hot takes on EU policy. I don't feel his thoughts on the matter are particularly well-thought-out, and I feel like he's just stanning for Apple from his priors rather than from any grounding in reality.
I dunno, I guess that's what you should expect from Gruber but these EU-bashing articles lowered the enjoyment I got from his blog underneath the bar for me.
Freaking out about basic EU privacy and safety measures is always one of the biggest red flags that someone is becoming a politically motivated hack, especially when that someone lives in the US where there is an active fascist movement tearing apart the government.
But what if those measures might apply to your favourite multi-trillion-dollar globo-corp and cause them to nominally change their business practices? That's a step too far, I say.
Oh, well then it must be stopped by any means necessary of course! Thank god there are brave and concerned citizens like John Gruber, ready to hold the line against the tyranny of regulations.
After all, if it wasn't for him who would ever speak up for the trillion dollar corporation? Without brave heros like him, these poor vulnerable corporations would be facing all sorts of attacks from evil regulatory organizations and their dastardly scientists. Just think of the profit - all those euros - that may be lost like chaff in the wind.
Shout out also to the MiG-25 Foxbat[1]. While not a particularly beautiful aircraft, it's made of steel and could go Mach 3.2 if they didn't want to use it again.
[0] https://en.wikipedia.org/wiki/North_American_XB-70_Valkyrie
[1] https://en.wikipedia.org/wiki/Mikoyan-Gurevich_MiG-25