So the clear implication is that this proof leaked into Anthropic's training data set, who then used the models which were trained on it to independently create a proof. From memory, previous such mathematical discoveries also came with similar implications.
Has someone confirmed whether those previous discoveries were due to such leaks? If so, it does really seem to portray GenAI as a giant auto complete engine, rather than an intelligence that can deduce novel solutions to tricky problems.
Uh, why is that the implication? I see nothing suggesting that, and they weren't using AI in their own development.
I think the implication is that they want to stake a claim on potential credit for the result. They got scooped, but they didn't get scooped by a finished paper, they got scooped by a lean formalization, so credit is ambiguous.
This result is almost reverse of what they have - they have a human-understandable argument that they believe, but it could have holes, potentially fatal ones. The formalization is (probably) correct, but less illuminating (though it sounds like it's better than Navier-stokes - at least one human at least mostly understands it).
Mathematics as a field is still determining what counts as a result in this new age. Is a lean formalization that no one understands a result?
It is at least different from Navier Stokes in the sense that the author did not use any AI software, so Anthropic could not have used his prompts for it.
Now did his proof leak into Anrhropic's training dataset... We do not know yet
EDIT: Looks like the 2 proofs use radically different ideas. It's unlikely there's a link...
It is such a massive miss from Google that it took them this long to release this model. I just ran it on my benchmark[1] and it matches Fable 5.2. They could have been first to get there, why didn't they?
That being said, it is still slower and more costly than Fable 5.2. So Anthropic wins in my eyes.
I think the fact that Gemini is now in the same conversation as Fable 5.2 is a crazy feat for them. So I dont think this feels like a miss to Google. Especially knowing that they have their next version in the works.
I apologize if this comes off as dismissive, but why would hype matter? Google is already everywhere, I'd imagine they don't really have to generate hype like a startup would.
Don't see the incentive to generate hype. There's no IPO even if they convinced everyone they were leading AI it would be little benefit to them, in fact might hurt their cloud growth if people started jumping more into Gemini. They're better off keeping up for internal use and lying low publicly unless it gets very profitable
According to Artificial Analysis, Fable 5.2 is more expensive than Carbon per task in every effort level in which it scores higher. So, can you be more specific about what you’re referring to? Other than vaguely suggesting Carbon is more expensive
Out of interest, do you run the same tests more than once to generate that?
What I often observe is that models being asked the exact same prompt can alternate between getting something right and getting it wrong, thanks to the wonders of a non-deterministic output.
The original is the best in my book. Understated, placed huge emphasis on the cars and the experience of long GT drive from A to B. Shame they basically abandoned that formula immediately with NFS2.
Those were 2 of my faves, NFSHP2 started a lifelong love of cars and the motorsport! But let's not gloss over the original NFSU - that one changed the game! All the customization options blew my mind as a 10 year old.
Will this make it to OpenRouter? Running your own benchmarks against these models is surprisingly frustrating when you don’t want to create hundreds of accounts for each AI lab’s API.
A lot of these models are likely benchmaxxing the existing benchmarks that are out there. Some perhaps even by accident.
I think looking at some more niche benchmarks helps, I created one and it does show Kimi K3.1 Pro as the most intelligent of all open-weight models, though it is still significantly below Opus 5/5.5[1]
Has anything changed? I already ran it on my adversarial programming language benchmark[1] a few weeks ago and it performed pretty poorly (on par with Kimi K3).
For what it's worth, it does perform on par with DeepSeek V4 Pro on my adversarial programming language benchmark[1]. I wouldn't think they would benchmax on that.
Any chance you could try qwen3.8 flash next? That is currently the best local model for actual coding tasks in my + many coworkers experience. It will run reasonably fast if you have 64+ gb ram and use strata or a strata fork (vram not super important with this method).
Has someone confirmed whether those previous discoveries were due to such leaks? If so, it does really seem to portray GenAI as a giant auto complete engine, rather than an intelligence that can deduce novel solutions to tricky problems.
reply