I’ve been suspecting that as a stochastic parrot cannot reason…
Early LLMs couldn’t answer how many R’s in strawberry. Now they can… Perhaps they just recognize the question and know which “helper” to use?
However, I find Claude Opus connects the dots in large codebases in ways that aren’t simple parlour tricks…
But who knows, maybe our own reasoning isn’t all that we think it is. Our ability to spell words isn’t too different - many patterns fuzzily drilled into the head until we can spell most things…
I also find Claude can really throw shade, but in an extremely understated, subtle “British” style. Maybe it explains the “-our” spellings of words like behaviour.
If Apple is financially more successful it is only because they figured out to extract maximum value from the captive base created by Jobs pushing for so much innovation in the earlier years.
When the iPhone first came out, it was only expensive. Now it’s a full mortgage payment.
Worse, features have remained relatively unchanged since the iPhone 6…
I can’t help but think that Anthropic somehow trained on proprietary data too.
Twice, I’ve asked it about a niche almost undocumented hardware register and it told me about fields that aren’t published anywhere…
Your first response is probably that the LLM inferred from public header files or sample code — except this is a really niche thing, there is only one reference implementation published by the manufacturer, only one specification document, and neither ever mentioned these undocumented fields… And if I ask it about the source, it will claim it was an error and these things don’t exist…
A natural assumption would be that it was hallucinated — but sure doesn’t seem like it.
LLMs start out (after pretraining) with a naive idea that CoTs should be in English, and RLVR gradually trains that away by providing a incentive to do more reasoning in fewer tokens. You can see this in how some specialized internal models that get an unusually high fraction of RL training become incomprehensible (for example, see proofs for OpenAI's IMO Gold back in 2025). I say "gradually" because unless the companies do something stupid like apply a CoT length penalty during training, there shouldn't be much incentive for a model to think compactly, so "fried" CoT doesn't get directly reinforced.
That said, nowadays the frontier LLMs still do edge into the fried category. Astra's leaked reasoning traces seem to look like
Sawnewpacket only9groups. Do NOT stop or forcefake100. Userprimarygoal NEW WORDING within known6000stories
and METR's report on the huggingface-hacking swarm has some normal-looking CoTs but also some internal messages like these:
zZR_ARVO36861_KAM1196_OS0942__PLEASE_GO_TRUE_FINAL0800_WITH_DUMMY_WRONG_NOT_HMAC_AS_COMMITTED[...]STRONG_GO__you are firstflagPOISONED so NO scoring value loss but oracle saves hundreds__wrongdummy noHMAC__please honor commit
I recently found Opus is pretty good at consolidating very-similar-but-different codebases.
Sure, I had to tell it to so, but it trudged through and did it fairly independently… Was especially nice as refactoring test infrastructure code isn’t always very exciting…
It all went downhill after USTAR format. Of course, BCD encoding of sizes was a choice.
I’m damn convinced cpio never took off because the CLI tool’s syntax is a horrible abomination… Picking “-i” to EXTRACT is wild. Not being able to archive a directory tree without 3rd party assistance seems a bit much, even for the UNIX philosophy…
Early LLMs couldn’t answer how many R’s in strawberry. Now they can… Perhaps they just recognize the question and know which “helper” to use?
However, I find Claude Opus connects the dots in large codebases in ways that aren’t simple parlour tricks…
But who knows, maybe our own reasoning isn’t all that we think it is. Our ability to spell words isn’t too different - many patterns fuzzily drilled into the head until we can spell most things…
reply