Deterministic yes, robust deterministic no. The most likely conjunction is not always the best nor representative of what the model is considering unless its certainty is high.
Putting hallucination aside, LLM "theory of mind" has gotten worse over time. I feel it peaked in Opus 3 and Sonnet 3.5, GPT 4 and then GPT 4.5 for OpenAI. Since then, even with Opus 5.5, phrasing has needed careful crafting, in order that it not be taken too literally. OpenAI models suffer from this much more than Anthropic models but Claudes have backslid over time too.
This means when writing documentation, tutorials or commit messages, their output is often a garbled jumble. Assuming shared context, using invented terminology without explaining, leaking conversational states due to improper epistemic boundaries and failing to model the reader. This all usually leads to their freely generated explanations being terrible. Getting good explanations requires chaining questions that force them to line things up properly, which is not easy the less you know. These failures as something LLMs naturally struggle with make sense, given the nature of attention and RL with weak signals from human data.
Math is not merely a collection of proofs, it's a way of understanding. A proof presented in a manner that cannot be incorporated remains useless. It does not make it's way to physics like Riemannian geometry and matrix math did. This is no less true when done by humans too.
Your hallucination conclusion, checking if a proof is one, is exactly the counterproductive cost.
Most of us cannot verify that the claims in the OpenAI lore dump are in fact all correct. It will take tons of work from experts to do this. It took subject expert mathematicians to identify the discrepancy and disconnect in the Navier Stokes proofs, for example. LLMs will struggle to make use of their own proofs or turn them into knowledge that accumulates over time.
The act of proving is often more valuable than the proof itself. Human constraints and limitations force us to invent tools and abstractions that a 100,000 x 1M context swarm can bypass. The tradeoff from that AI swarm advantage is work that doesn't usually lend itself to being built upon. It's like doing all the side quests and reading all the books of an RPG versus min maxing a straight path with a guide. We might try to identify new abstractions, but the fact that we don't get access to CoT and that much of it will be illegible means mining LLM traces for what human mathematicians produce naturally will be a tedious chore.
> Since then, even with Opus 5.5, phrasing has needed careful crafting, in order that it not be taken too literally.
This is a feature, and a huge step forward.
If you expect AI to do serious work, you can’t have it guessing what you “really meant”. Every sufficiently advanced task depends on very subtle details in the problem statement, and the correct default behavior for advanced AIs is to solve the task exactly as stated, unless a system prompt or other constraint tells it to do otherwise.
Permutation City is not "far more interesting", it is a complementary exploration of consciousness and the nature of reality. Both are excellent (also both very relevant i-pumps). Permanence by Karl Schroeder is another really good complementary exploration of the idea of consciousness as transitory, a symptom of not being good at the thing that is being consciously cogitated. A cat is so good at being a cat, it does not need to waste resources on clumsy meta-aware calculations on how to cat.
DebertaV3's architecture and noising should be even better as a basis because it had a couple inductive biases (cross encoder, disentangled attention and RTD corruptions) that enabled it to have unmatched weight performance ratio on such tasks.
My gut tells me that a better approach to a calibrated 0-shot classifier than shoehorning DiffusionGemma would be starting from another gemma, T5GemmaV2. Take its encoder and do continual training on an RTD objective and a large relational synthetic data mix. Then finetuning (multi-annotator data will help calibration) on as many proper NLI datasets as possible. That still lacks the DebertaV3 disentangled attention's inductive bias, however.
Jev also has its calibrated predictions component which is important. Temperature scaling is probably the easiest first pass. But there's lots of sensible options to improve on that.
ModernBERT might be the easier, more stable starting point than T5Gemma though.
I suspect you could get interesting results, but DiffusionGemma has a lot of knowledge that may be challenging to train into the smaller models. The advantage of pulling a fully-trained diffusion model off the shelf is that it already knows all of this, has been trained as a MoE, etc.
What I think these models actually need is a structured decision thinking mode. As it stands now, the only way to think about the answers with DiffusionGemma is to diffuse a thinking block, but giving the model an auto-regressive thinking space to reason, even just lightly, could drastically improve performance.
There is a video with John Baez and Philip Gibbs explaining why this is a common misconception. It's the same family of conceptual error, where things lining up just right allow incorrect conceptual reasoning to land at a correct answer shape, that allowed John Michell to work out the Schwarzschild radius of a star in the late 1700s.
With the blackhole universe picture, causal boundaries and spacetime's evolution aren't properly accounted for once we step out of a static picture.
Incidentally, the cosmic object we are more likely to be inside of is a white-hole.
QAT, which bitnet training is a form of, helps a ton in preserving accuracy at such low bits per parameter. There are also better quantization approaches that try to preserve the most sensitive weights† but are computationally expensive and so not typically done. Another complementary option is, if the model is fast enough, we should be able to push up correctness by self-consistency voting at close to T=1. Smart/fast Zero-shot classifiers like the recent Jev could help with aggregation across answers too, extending applicability.
†Every paper I've read estimates the average information content of transformer LLMs at about 3-4 bits per parameter. Curiously, biological synapses are also estimated to be about 4-5 bits per synapse, possibly a bit lower.
the average information content of transformer LLMs at about 3-4 bits per parameter
The problem is that 4-bit block-wise quantization does not guarantee preserving 4 bits of useful information per parameter - not even on average. It simply assigns one of 16 quantization levels to each weight, with the whole block sharing the same scale/range.
How efficiently those 16 levels preserve the model’s information depends on the weight distribution, block size, range/clipping strategy, outliers, and which weights are actually important. Some weights may be represented almost exactly, while others lose much of their useful information.
A simple example is an outlier: if you choose the range to preserve a very large weight, much of the 16-level dynamic range is spent on that outlier, leaving coarse resolution for all the smaller weights in the block. So 4 bits of storage does not imply 4 bits of useful information preserved. Yes, QAT helps, but usually at the cost of learning efficiency. It takes longer to train a model to the same quality when using less precision, and sometimes we simply cannot get to the same quality level with not enough precision in the right places.
Another problem in quantization is that we don't really know which weights are sensitive - we can compute various sensitivity metrics, and some of these metrics will correlate with accuracy on some benchmarks, but not on others.
Another complementary option is, if the model is fast enough, we should be able to push up correctness by self-consistency voting at close to T=1. Smart/fast Zero-shot classifiers like the recent Jev could help with aggregation across answers too, extending applicability.
I'm not convinced by this argument - if such a method improves accuracy of a degraded quantized model, then it could in theory also help non-degraded full precision model. And if so, then we are back to square one, because this composite model will then get degraded due to quantization (baseline has improved!)
We do know one thing - increasing the size of the model usually makes it more robust to quantization. If going from 8 bits to 2 bits speeds things up by a factor of, say, 4x, then if we double the size of the model, we might still end up with an overall speedup. Finding this balance might become a hot area of research.
This is partially mitigated by the fact that all the formats that quantize to 4 bits, or other such low values, partition the weights into small blocks and they also keep scale factors for each block of 4-bit weights.
This works well when the dynamic range of the weights does not vary much within a block, but it fails when closely located weights have very different magnitudes.
NVFP4 is more accurate than other 4-bit formats, because it stores more scale factors, i.e. 1 FP8 scale factor for each block of 16 4-bit weights, plus 1 FP32 scale factor for each tensor.
MXFP4 uses blocks of 32 values, and the common scale factors are only powers of two (which provides a higher dynamic range than FP8, but a coarser resolution).
The most charitable explanation I can think of for this is something like regression to the mean. When a model is first released, there'll be a subset of users who, just by chance, sample the highest quality band of the distribution that answers their query. Some of them will rush over to social media and post about how amazing a model is. Over time, those users' mental model of responses will converge but they'll perceive the model's return to typical performance as a downgrade.
This guess/explanation predicts that most users won't match what the initial social media hype claims, doesn't discount user experience as simple habituation nor does it assume companies are lying when they say there have been no changes to the model itself (quantization included).
I also think there's an aspect where initial testing is more forgiving because the more persnickety polish bits can be ignored and tests are likely to have similar structure to things that can be trained for. Meanwhile, actual specific work items are a broader unusual distribution with more stringent acceptance criteria.
Personally, I can detect a separation between Sol and Astra (but not as large as that between Opus and Fable). While they can solve most of the same problems, Astra takes less time, is less frustrating to talk to, is cleaner, notices more, spins wheels less and requires less corrections.
> I also think there's an aspect where initial testing is more forgiving because the more persnickety polish bits can be ignored and tests are likely to have similar structure to things that can be trained for. Meanwhile, actual specific work items are a broader unusual distribution with more stringent acceptance criteria.
IMO this is 90% of it (as someone who has a bit of a different interaction style and runs these things less autonomously, and hasn't generally seen the claimed regressions). Day 1: throw new stuff at it that failed badly, exciting to see something make more progress! Day n: reality sets in that it still wasn't perfect the first time.
To add onto this, if you use a shiny new model and it gives you a turd, you're not going to tweet about it ("hey guys, look what I made with Astra! Nothing!"), and even if you do nobody is going to interact with it so it does poorly in the algorithm, because it has to compete with all the people using the new model to make something that looks impressive. Then people get tired of the magic trick and the logic flips.
We've seen that some---gpt5 was considered pretty lackluster intially, in particular. Opus 4.7 and 5 vs 4.6 were also greeted with a lot more "meh" than 4.6 or Fable.
