Works with GeForce RTX 20 series and newer, RTX PRO (Turing and newer), DGX Spark and Apple M4 or newer.
It does not pool memory or split one inference request across machines. Adding machines increases parallel throughput but won't let you run bigger models.
> While a human may say “aha” to indicate exactly a sudden internal state change, this interpretation is unwarranted for models which do not have any such internal state, and which on the next forward pass will only differ from the pre-aha pass by the inclusion of that single token in their context. Interpreting the “aha” moment as meaningful exemplifies the long-neglected assumption about long CoT models – the false idea that derivational traces are semantically meaningful, either in resemblance to algorithm traces or to human reasoning.
This paper addresses something that has always bothered me about LLMs. You read their reasoning, see something like “Wait, that’s wrong” and then watch them make the exact mistake they just identified.
By itself, "aha" carries no insight, but the insight is probably stated immediately after it. In that case the aha is semantically useful, by identifying the insight it is near.
> but the insight is probably stated immediately after it.
If the intermediate tokens represent reasoning or thought, you would expect "aha" to occur after the thoughts that led to the realisation, including the thoughts encoding the explanation: they don't have any other state. There is no reason to draw the conclusion you've drawn. Furthermore, what LLMs are doing isn't thought.
> If the intermediate tokens represent reasoning or thought, you would expect "aha" to occur after the thoughts that led to the realisation, including the thoughts encoding the explanation: they don't have any other state.
Yes they do, they have their KV caches-- it's a pure function of the input tokens, sure but that doesn't prevent it from containing latent 'insight'. LLMs can and do pre-form the tokens they're expecting to output multiple steps in the future.
I wouldn't argue that the 'aha' means anything, but the structural argument that it can't that I think you're making isn't sound.
Consider this: while the inner state of an LLM (all its activations, residuals stream that is cached in the KV cache) is fully deterministic given its input sequence, the information contained in it IS NOT identical to the information in the input sequence. The reason is obvious: the LLM itself contains an enormous amount of information in its parameters and it transfers it to its residuals stream at each forward pass.
In other words: the final state given the two input sequences (where NT stands for "null token"):
[NT]
and
[NT] [NT] [NT] [NT] [NT] [NT] [NT] [NT]
is not the same, and at each forward pass the LLM keeps working on the solution even if the input tokens provide absolutely no further information.
If this is correct, then there is no need for the model to have already verbalized the key elements that drive the "aha" moment, so no need for the "aha" to appear after a full explanation.
Let's say that the forward pass that selected "Aha" produces activations that indicate a wrong assumption, and a plausible explanation.
It puts learned projections of the activation into the KV Cache and outputs Aha.
Both the cached projections and the current Aha token can now influence further activations in an additional Forward pass that the Aha bought the model.
Beats me how it works, honestly can't wrap my head around it.
From what I understand, at position Aha in each layer it's constructing a query based on the current activation and looking at the key of each other token position for that layer, in order to decide how much attention to pay to the value.
In this way it attends to the previous values, such as perhaps the incorrect assumption and plausible explanation.
it's a rhetorical heuristic that a writer should know to use when directing a reader to a declarative that they want them to pay attention to, usually because it's a non-obvious or roundabout insight
when utilized by AI, it's a probabilistic output and it's variable whether or not that rhetorical trick is useful. it also pushes a non-skeptical reader to focus too much on the following text or even to believe that they, themselves, derived some insight. this is effectively a kind of persuasive sophistry which is not helpful - adding rules around it prevents people from deluding themselves with AI
Did not read the paper so apologies if this is covered but isn't it possible that there is some recognizable semantic pattern in the training data where an "aha" is often followed by a subtle semantic shift that proves closer to the original premise in some critical way, and by emitting the "aha" token the model causes itself to produce such a subtle semantic shift that pushes the subsequent reasoning closer to the desired response?
It amounts to noise overall, but it has further unwanted and potentially misleading 'properties'. I think it's rather sobering to see how much bandwidth is still being wasted.
I've done that before and the code was a mess. It works at the beginning but APIs do much more than piping data from the database. When you start dealing with ACL, external calls, code reuse, etc. It's just nice to have all the tools available to you from something like Python or Go.
Pixels are the only devices providing the updates and hardware-based security features required by GrapheneOS. Motorola and Qualcomm are improving their hardware so future devices will all of our requirements. Motorola will provide us what we need to provide proper updates and is helping us port to their devices. We'll have their support to keep it working well. GrapheneOS needs a lot of hardware related work for our security features.
Our hardware requirements are listed here, but our features also need to be ported to the hardware such as getting hardware memory tagging working for the whole kernel and userspace:
This is not a good example. Pinephone Pro had a very small developer community (of volunteers) and thus quite undeveloped software. It was discontinued for this reason. Librem 5 and Pinephone are usable as daily drivers, albeit they do require compromises.
This paper puts words to something I’ve noticed repeatedly with LLMs, particularly Qwen3.6. When I read its reasoning, it appears to recognise the mistake and then carry on as if it hadn’t noticed it at all.
> models often determine their answers based on implicit biases tied to question templates, then construct reasoning chains to justify their predetermined conclusions
> its reasoning was correct right until the final step (Yes/No answer)
First author here, surprised to see the paper in HN! :)
When doing the paper we noticed that models are very good at generating post-hoc plausible CoT, which to me knowledge can happen quite often with relatively easy tasks.
I agree, and I very strongly dislike it, to be polite about it. It contributes absolutely nothing and is an excellent way of hand-waving away literally anything an AI model does. Saying "well people do this too" is a great way to rationalize away anything you can imagine that an AI model would be capable of, because "humans do it too so what's the big deal, guys?"
I can just immagine the response to a headline "LLM chooses mass death: thousands killed in horrific AI accident" being something like "lots of humans have caused mass death too."
Maybe we can think up a (personal) rule we can follow?
A naked "natural intelligences do this too" might be a bit too short to be useful. But if we can add when/where, cite papers, or show ways in which the parallel operates, then it might be useful.
Compare, eg, talking about a robot arm, and someone goes "a natural arm does this too". You can tell about the fact that it has the same degrees of freedom in the same places, or how this pertains to inverse kinematics, or etc...
Same way here, "this happens to be how natural intelligences seem to solve this too! According to Foo, Bar, Baz et al (2026) the gadget is always twiddled beforehand in macaque apes. " or "Same for natural intelligence: I've noticed I use the same general algorithm myself. I've always considered this the correct way to do translation between languages".
--
A more concrete example of a useful answer here.
Natural intelligence does this too! When given the question "explain your reasoning" humans are indeed quite prone to post-hoc confabulation. [1]
It's the grounded portion of a feedback loop searching for the 'why is this happening' thinking. I'd imagine most of my own comments in this area boil down to "GIGO" most of the time.
It's a relevant comment in this instance because we're discussing concepts you need to be both trained and practiced in to reason about, and that our discipline has traditionally been blind to. Plenty of people working with LLM context issues who've never been exposed to the idea of 'subtext' or could tell you why it would matter to their direction of effort.
The current political climate is well reasoned and intentional. It might not be yours or mine, however the system is working exactly as the ones paying for it have intended.
My brother and I have been arguing about that all of our lives. He believes everything is intentional and it's just a matter of discovering who benefits. I see chaos that nobody intends or controls. His political landscape is a tapestry of conspiracy theories and mine is a fog of war. I think his is more comforting, since it admits a possibility of a rational, predictable world.
I'd synthesize those as: "There is a lot of chaos with no central plan, but every small piece happens because someone believes they will benefit."
In other words, a lot of this depends on what scale/scope is being inspected. On the high level, the world is chaos rather than a meticulous and inscrutable plan of the Illuinati Shadow Cabal. On the low level, people do things for reasons, even if they're dumb ones.
With respect to the "current political climate", I'd like to suggest that a lot of dumb or seemingly "against their own interests" stuff is due to people prioritizing costly in-group loyalty signals. Their interest in staying good with the tribe is just higher than their interest against a dumb national policy.
It’s intentional in the sense that actors are acting intentionally for their own benefit (or at least what they believe is beneficial) and following incentives. Not that there is a master planner who manipulates everything
The current admin seems to have quite a few long term plans they have been working towards.
Project 2025, Maralago accords.
So far the only major policy item the Trump admin seems to have not intended was the Iran War. Israel killing the intended replacement, Iran leveraging the straight of Hormuz, and dropping three Tomahawks on an elementary school really botched that one.
Yeah, if we are talking about the Trump admin it’s definitely a conspiracy. Pretty much Peter Thiel’s cabal. But they are pretty open about their plans
the thing is, that fog might've been true decades ago, but for 100 billionaires to sit in a virtual smokey room and do the shit they want to do, that's not really a conspiracy.
It's just peter theil's texting groups.
The ability to conspiracy both willing and unwilling is such a low threshold now, it's virtually indistinguishable.
You watch one billionaire do something and you're like, I'm a billionaire, I should do that too.
The fact that there's so few billionaires, the probability that they conspire together both direct and indirect approaches 1.
The inverse of course is rediciously hard to conceive: the working class bands together to get something like universal healthcare.
It means the company is measuring the wrong thing.
I would much rather have someone on my team who ships less but whose work I can trust than someone much faster whose changes leave me wondering what problems we’re going to discover later.
And when production breaks (and it will), I need the person who made the change to actually understand it well enough to help fix it, instead of showing up with no idea what is going on.
You can obviously be an asshole about how you do it but I don’t think pushing back makes someone toxic.
You need to be flexible and compromise when the business trade-off makes sense. But you also need a backbone. If you think something is going to cause real problems, bringing it up is part of the job.
The difficulty of producing code WAS one of the circuit breakers.
We had tests, CI, code review, QA, architectural reviews, etc. None of those disappeared. But they were designed for a world where producing such a large amount of change was impossible.
Tests don’t solve that. Tests can tell you that the behaviours you thought to test still work.
How many times have you had a completely green CI with 100% coverage and still shipped a bug?
It does not pool memory or split one inference request across machines. Adding machines increases parallel throughput but won't let you run bigger models.