They're solving them in a way that isn't particularly interesting because it's near impossible to read and understand, and follow, what the LLM did. It's this way not because it's so smart, but because it's so stupid and still reached the goal by pure force and effort.
I don't know how to communicate this, but what you are saying is almost entirely wrong.
People can follow what they did. The documents OpenAi produced in their big drop are hard to read because they are poorly written in weird ways, not because they are beyond human intelligence. They didn't brute force in the normal sense. AI doesn't get tired, so it can try 1000 promising ideas, but it's like 1000 Ph.D.-level mathematicians trying different ideas, not 1000 calculators trying all combinations.
The biggest results -- quasi-Riemann, no Siegel zeros, and Unique Games Conjecture -- have already been digested by experts. No Siegel zeros in particular already has like five different write-ups, and Tao has written a exposition on his blog.
So did we get any new mathematics out of this, or just solutions to problems without new techniques or insights? I haven’t been able to keep up with this or read the writeups you mention, genuinely wondering since this seems to be a central concern of the math community
It's only been a month and people aren't that fast, but I think yes. Analytic number theory and complexity theory were stuck on the lack of progress on certain questions, that is now unstuck.
I am sorry but your argument looks to contradict itself: They solve hard mathematical problems because they are stupid and use brute force? Then why a human skilled in mathematics guided them for many turns in their conversation?
If they were really stupid, a Prolog program, a genetic algorithm or a SAT solver would be enough to reach the goal. Clearly it was not the case.
Can't you kind of think of them as a genetic algorithm that has a very good hypothesis generator? Humans could, in principle, help guide that (already very impressive) hypothesis generator to make better guesses to find the proof more easily.
Conversations with an ChatGPT with Terence Tao at the helm are a thing of beauty. An advanced practitioner prompting things beyond my understanding. Jarred Sumner's paper on the Riemann Hypothesis, however, was not that. No slight against him, but Jarred was a non-mathematician, and his prompting to improve the minimum fraction of zeta zeros by Claude amounted to "keep going" and "believe in yourself". Shit, anyone coulda done that in his position.
My point is that it does matter who's driving the AI but it also matters the route. Just like F1 champion Max Verstappen is a ridiculously better driver than me at driving an F1 car around a track, if we're just driving a friend to the airport, it doesn't really matter if it's Max or me driving, so long as we don't crash.
I think it does gel. You're right that the comment you're replying to is weakened by that fact.
It gels because they solved about 700 problems out of 4000 attempts. And they threw a ton of resources at each. So "per problem" it's less than 1 in 5, "per token" it's probably not a great record. I mean it's astonishing that it's possible at all of course. But on average, even an expert with a top model will probably have an underwhelming response to the vast majority of prompts.
and I'm guessing it was with "tool calling" ie. they can write a bunch of Lean proofs and hit compile and basically brute-force a whole bunch of hypotheses quickly?
Ie. the LLM itself isn't as "smart" as the combination of LLM, Lean, looping, search and vast gpu/cpu-time.
In my experience they write an informal proof first, check it without Lean, and only then turn it into Lean code. That last step sometimes takes them much longer than checking their proof without Lean, which is a problem humans have as well. What they're truly bad at, however, is explaining their proofs. Until they improve at that, we'll have to keep resorting to Lean to check whether they solved something correctly.
I'm less sure about the following, however: I think that while current LLMs are bad at explaining their own proofs, some models are great for discussing older, human-written proofs. For this use case, from best to worst, Gemini > Claude > GPT-Astra >>>>>>>>>>>> GPT-Sol.
I definitely don't know but I got the impression it's the opposite. Based on the observations about Navier Stokes and the natural language proof not matching up with the lean proof and other things. They generate lean and convert it into natural language at the end. But really I could be wrong just giving anyone interested a lead to check out.
Look cmon, this is pure cope. The problems it has solved in TCS are far from trivial, with plenty of smart people making their careers in trying to solve them. The techniques these LLMs are using are non-trivial, and not just brute force.
I'm more interested in the techniques used to generate these proofs, and how they can be improved to be as reliable at solving a broader class of problem than already formalized mathematical conjectures.
I was surprised in reading the article that it had nothing to do with the contents of the model. It could have been any model capable of 1M context that triggered this bug/misconfig on Cloudflare/SGLangs side. The title is extremely misleading.
Are there inference products focused on say, optimizing frequently used internal contexts, so that they are compact in the kv cache? Perhaps there rep could be optimized over hundreds+ of distinct real use cases?
Maybe your law firm doesn't want your data going outside your business. But you obviously want high quality analysis.
What is inefficient is thinking in 5000 floats, and then quantizing all that computation into a 17 bit token, just to immediately turn the token back into a vector.
Yea, I think LLMs would be much less powerful if not for the inherent randomness in the GPU...
It is like a decompression program that results in a slightly different result each time it is run on the same compressed data. If you are lucky, the delta could end up with answers to all your questions..
Because - it's the part people are having hard time to accept - creativity was never the hard part. It's randomness. The hard part is navigating the narrow border between "not enough random, therefore boring" and "too much random, therefore unhinged/insane".
That border is determined not as much by ground truth, as by people's sentiment. As such, many famous artists and creatives never succeed navigating - their contemporaries would call them crazy, and then decades or centuries later, the border shifts, and suddenly they're remembered as greatest creators of their era.
That is not true at all. As I previously stated: reference, stealing well, blending styles in new ways that obey or break the rules of your art form… what appears random to outsiders is a story that follows as naturally as 2 after 1 if you understand the context.
Until an AI can have desires or experience life it cannot tell a story.
Even if it could it is questionable if such alien experience qualifies as story.
It can retell stories. It can swirl around extant media like a child playing with its dinner. It cannot blend in its own lived and new perspectives which is what gives new art value.
> what appears random to outsiders is a story that follows as naturally as 2 after 1 if you understand the context.
It's interesting to compare your opinion with Andrew Wiles quote:
> Perhaps I could best describe my experience of doing mathematics in terms of entering a dark mansion. You go into the first room and it's dark, completely dark. You stumble around, bumping into the furniture. Gradually, you learn where each piece of furniture is. And finally, after six months or so, you find the light switch and turn it on. Suddenly, it's all illuminated and you can see exactly where you were. Then you enter the next dark room...
This is analogous to “I saw Abe Lincoln at the hall of presidents at Disney.” and insisting it was really Abe Lincoln.
It is infantilizing to suggest this is originality or creativity just as much as it is to suggest David copperfield is a scientist.
It is an illusion. A subterfuge being performed by the puppeteer. It works due to the theft of human’s work.
Ask yourself,
"What are this creative engine's sensory inputs to experience life?"
"What can this creative engine tell me about the experience of life from its own direct inputs?"
"How does it blend/shape or re-interpret raw input direct sensory data with the, admittedly, astounding body of knowledge it has about creative work?"
"What form of change or growth has this engine experienced that teaches us about ourselves?"
If you take those questions seriously you will come to the same conclusion. It is a Roomba smearing the valuable creative work of millions like dog shit across the living room floor, not a consciousness experiencing the agony and ecstacy of life.
If you think I am wrong ask an AI. It won't come up with an original answer, but it can certainly tell you whether the collective body of slightly-outdated human knowledge thinks I am correct.
In practice, hallucinations have only mitigated within verifiable domains. The vast, vast majority of human activity is unverifiable, or at least significantly less verifiable than mathematics.
Brains aside, it S1/S2 seems like a good description?
System 1 is quick and intuitive. System 2 is slow and deliberate.
Compared to models that compose responses with many forward passes, thinking in token space, these models make a quick decision after processing their inputs.
It’s ok to use it as inspiration, but using the term comes with psychological baggage in an industry that already abuses anthropomorphism. Give it an informative name
System 1 is quite informative about what it's for. It means it's a system designed for bounded time quick classification, vs. agentic unbounded or slow thinking with scratchpads and tools.
Does it inject thoughts into system 2 without system 2 knowing about it? Does it have instinctive social skills? Can it do instinctive arithmetic? The list goes on, there’s a nice book about it…
Were people getting their pants tied up in knots like this over names like daemon programs (a term that carries "baggage" of its own) and installation wizards (which, to my knowledge, do not actually invoke in the arcane)?
Here is a law I wish we had for social media. Let's just align incentives.
If you are working on a social media platform, and your total comp is >= 500k, your children must use your platform for the average amount of time that children of that age range use your platform nationwide.
> "Instead of picking the highest-probability token, we can use different selection strategies to balance safety and creativity in the generated text".
Safety is definitely the wrong word here.
Temperature 0 generated text actually has a weird "lack of surprise" character that makes it seem artificial. [1]
> "high-probability texts can be dull or repetitive. Humans use language as a means of communicating information, aiming to do so in a simultaneously efficient and error-minimizing manner; in fact, psycholinguistics research suggests humans choose each word in a string with this subconscious goal in mind."
I'd completely drop the dropout explanation. It's just not part of the modern recipe anymore, AFAICT.
As for the ambitious goal of explaining transformers with a single interactive visualization, I just have a hard time imagining a person is going to newly understand both word embeddings (word2vec blew my mind in 2014) and also gain an understanding of attention.
I am making my own visualizations for a presentation on "Full Bandwidth Transformers"[2] that I am giving tomorrow at the Deep Learning Study Group (SF) (on zoom for the non-locals)[3]. It's not meant to be stand alone/context free, but I'd love some feedback.
Hey, take it easy on the guy. Words are just probabilities in some high-dimensional space; it's a probabalistic selection anyways; he probably just nudged the latent vector a little too hard. Spymarking and watermarking are for safety reasons, you know. There was zero loss of quality there.
> I'd completely drop the dropout explanation. It's just not part of the modern recipe anymore, AFAICT.
Is there any other mechanism replacing it? As far as I understood, while following the LLMs from Scratch book, the dropout is a mechanism to prevent overfitting, and it makes a lot of sense (basically cripple some neuron connections randomly during training such that their weights aren't updated).
Edit: well, I could have searched on the internet, but other people may stumble on your post and you seem quite knowledgeable of the inner workings of these things. :)
I suspect weight space programming is not going to be the way models are practically optimized ever.
https://x.com/NeelNanda5/status/2117375052618539485
reply