It's looking increasingly likely that many backend systems in the future will just be formally verified and generated autonomously from specs. That means you'd never look at the code, only the spec. You'd basically "program" and "debug" the spec, and the full system would be autonomously generated each time to match the spec.
Frontend code is probably far harder to apply this technique to.
In one case I formalized the spec for a message decoder, then autonomously generated the entire decoder from that. The spec is essentially a very slow version of the decoder written for correctness and clarity, and the implementation became very complex as it went through multiple optimizations, but it was proved every step of the way to match the original spec.
Sweet, we could even develop a standard language(s) to lay out such specs, and write in specialised text editors that would help with debugging. Hell we could even call it ‘programming’
Correct -- this is exactly where the frontier is heading at the startups and AI labs working on this.
While you're being tongue in cheek, what you're missing is that the spec you'd be working on in some cases can be compressed to a small fraction the size of the implementation code. Alternatively, it can be written for clarity and correctness rather than performance, and then the implementation can be optimized like crazy without ever worrying about correctness.
There was a very brief period in the early days where Perplexity actually was pretty useful. I believe I even had a paid subscription to it for maybe half a year at that time. The main value it had then is that ChatGPT didn't yet incorporate web search, or early on after it did its searches were will really limited. There's no reason for Perplexity to exist now. I'm confident they and their investors are keeping it going mainly because no one wants to be the first major AI startup wipeout.
Their moat is their research capability. It's a combination of proprietary internal technology and models and infrastructure, people, organizational structure, and culture. As long as they maintain the best and fastest research capability, they have a durable advantage, at least for the time being. If we start to saturate all economically useful outcomes that can be reached via greater research productivity attainable to us, then this moat will start to fall apart. It could also fall apart if safety issues force a premature halt or major slowdown to further research.
This assumes any research gains last. In practice it seems like none of the frontier companies can keep much of their "secret sauce" secret for very long.
If company X has a better model than company Y, do you use it? Well, that depends: will Y take two years to catch up? If so yeah, you buy X.
But if Y is just as good a few months later, you purchase based on other factors, because you know you can live without the brilliant research X is doing ... since you'll have it soon enough.
The problem with your logic is that the history of tech is full of “good enough” eating “best” for lunch all day long. Anthropic’s plan to be the best is increasingly looking like a terrible business plan.
That doesn't seem like much of a moat. What happens when a couple researchers go on new parent leave at the same time, or someone's productivity drops for a quarter because a parent passed, or gets divorced, or whatever?
Continually staying 6 months ahead of the distilled open-weight models means you need to keep flipping heads (nobody goes on vacation) until the end of time. The geometric distribution describes that, and it ain't pretty for the expectation value of these companies.
I don't know and can't answer or predict that. But that's exactly what it could become (although probably not that extreme), where the agents doing much of the internal work are proprietary, conferring a research advantage.
My point is that it is not a real moat if other players can hire people from core research teams, and get a foothold that easily. And when/if models are good enough, you don't even need many people to form a fully functioning team.
It may actually get harder to replicate with time, not easier, even if the number of people needed to run the operation goes down. As the largest labs move towards internal RSI, the most important IP for doing that will transition into their proprietary internal infrastructure and models, none of which will ever be released and which increasingly won't be fully known or understood by the humans you might hire away.
If their moat is their ability to create marginal improvements every few months by releasing new models, then that is an extremely expensive moat to maintain.
It's very hard to measure exactly how large it is, because there aren't good pre-existing metrics for it. You can measure in time (e.g., how far behind open weight models are from the best frontier models commercially available), but it's plausible that what seems like a relatively small time advantage (months) might mask a considerable difference in internal research sophistication. What we've seen repeatedly is simply that there does seem to exist a durable advantage in research productivity, at least for now.
Autonomous software and hardware verification. They're just getting started. It's not just Axiom: Logical Intelligence, Harmonic, and a new one that just launched (&Truth). I suspect the business case is addressing the global cybersecurity crisis that AI is creating.
Making it cheap to do is probably exactly what they're trying to do. Formal proofs for verification take 10-20x as many lines as the core functional code being verified. Nobody's going to want to pay for 10-20x as many tokens at frontier pricing for verification unless they have to or it yields tremendous benefits. So getting the token price down as low as possible for a prover is probably a core part of the value proposition.
I understood them to be arguing that AI models are taking away the joy of certain things like mathematics but not producing commensurate value for humanity. We have a lower bound on twin primes, but that doesn't improve life for anybody, and the cost of getting the bound was removing human involvement from a lot of mathematical research, which was a source of joy for some portion of the human race. If all of this leads to curing major diseases without an AI takeover, it will have been worth it. But if it doesn't, then the ends don't necessarily justify the means.
The data is still mostly 2025 and the 2026 data is mostly from the first few months, and most production deployments (at least in, say, the Fortune 500) lag the frontier by at least several more months (or much more). I suspect it's just not really possible for academic research at its conventional pace to offer an up-to-date perspective on this topic right now.
> I suspect it's just not really possible for academic research at its conventional pace to offer an up-to-date perspective on this topic right now.
If we weren't still in a massive hype-and-investment bubble, the burden of proof would be on the LLM manufacturers to demonstrate what deficiencies that studies like this have revealed have been fixed in their latest products.
I mention this because I've been hearing "Yeah, well, that opinion/study is completely invalid because it used a six month old tool. Today's release of SOMNOS 6.9 (Pinnacle Snoozytime) has totally changed the game!" every six months for what feels like the past five years now.
> I mention this because I've been hearing "Yeah, well, that opinion/study is completely invalid because it used a six month old tool. Today's release of SOMNOS 6.9 (Pinnacle Snoozytime) has totally changed the game!" every six months for what feels like the past five years now.
Given my own conclusion that it's not possible for conventional academic research to accurately snapshot industrial adoption of this technology, I would expect this pattern to continue for a while until either academia becomes much faster or technology progress slows down (or both). This just isn't yet a situation where you can rely on published papers telling you what to think -- you still need to mix in a bit of your own instinct and experience and extrapolation to get an accurate up-to-date read, even if there's some prediction error as a result.
> This just isn't yet a situation where you can rely on published papers telling you what to think...
Well, like I said: If we weren't still in a massive hype-and-investment bubble, the burden of proof would be on the LLM manufacturers to demonstrate what deficiencies that studies like this have revealed have been fixed in their latest products.
I have my own experience to go off of, but this can also be inferred easily. There are many large corporations still working on "digital transformation" from 10+ years ago. There's no way such companies have fully onboarded and integrated frontier AI agents throughout their workforces with high adoption. The paper itself reports low adoption even after conditioning on access.
Everything did change after late 2025, but there are tiers of usefulness. Late 2025/early 2026 was the first time agents became clearly useful for personal projects. That's distinct from immediately having economically valuable end-to-end autonomous workflows in teams of median talent, which many would suggest is the threshold we've just recently crossed.
I don't, actually. I can simply point out that its data is out of date, and then the numbers themselves are called into question without other numbers to compare them to. And that's what I did: I shared two arguments in two comments that converge on the shared conclusion that the paper's data is not up to date (or, more precisely, cannot be up to date).
> I can simply point out that its data is out of date
You can, but it doesn't make it correct or, evidently, yield a logical argument. Do you notice that your logic is circular? The date doesn't matter if the passage of time has not affected the outcome and that's basically what I questioned, which you refuted without any evidence. So the question still stands. Are there any comparable stats you have access to that change the picture?
You're trying to get me to make a claim stronger than what I'm making. The claim I'm making is that capabilities have moved since the paper's data (countless benchmarks support this), and it's not possible to measure the end-to-end effect of that inside industrial applications with complete access, adoption, and internal connectivity at the same pace. Therefore, the study is fundamentally out of date and conclusions about the present or future cannot be drawn from it unless you assume that technological progress cannot have affected the relevant outcomes, which is a strong assumption you'd have to back up separately. Note that the paper's model ties end-to-end outcomes to generated code quality and correctness which is precisely one of the variables that continues to move.
What the study actually shows is that organizations that had very little time to absorb older generations of a disruptive emerging technology experienced an indeterminate change in output (at the 12 month mark, anywhere from down 13% to up 25% -- the paper can't say where reality falls). It certainly doesn't establish any ceiling.
> You're trying to get me to make a claim stronger than what I'm making.
Not really. You’re retreating from your original claim:
> most production deployments (at least in, say, the Fortune 500) lag the frontier by at least several more months (or much more)
Do production deployments like AWS bedrock which companies use, lag behind so much that the data from early and mid 2026 is meaningless? Any data to support this? Or shows how they lag and how this is relevant?
It's still the world's largest prison: most North Koreans would be killed on sight or imprisoned for even attempting to leave (i.e., escape). It also has millions of slaves, in fact the highest rate of modern slavery in the world (approximately the same as the US in 1860 before the Civil War). Kim Jong Un executed his uncle in 2013. Privilege or comfort, where it exists, can be stripped by force from an entire family at once for any reason. So, yes, Kim Jong Un is not a literal god on Earth -- there are balances he must maintain within the country to keep his power. Not every North Korean is a slave. You're not wrong about those things. But I feel your description is coming dangerously close to portraying it as a relatively normal country, which it most certainly is not. It is a prison and slave state, one of the most severe in history, enforced via extreme brutality and shielded by nukes against those who would save its people.
This is what everyone thought about modularity of elliptic curves. Then Wiles straight up just counted them (indirectly). You shouldn't be so confident about things that you do not know (since, if you knew them, you'd have solved the Riemann hypothesis!).
> Several hours of work with Fable simply convinced me it wasn't yet solvable and reinforced how hard of a problem it was.
This is the part that gives me the strangest feeling about it all, because you're not the only one with this experience. I've experienced this too on different problems, as have many researchers across many fields.
I disagree with the Fields Medalists on the majority of their complaints. AI math is happening and there's no going back. However, on one point I increasingly agree: virtually none of this stuff is possible with technology any normal citizen has access to. I have no problem with AI models making revolutionary advances in math or science. Where I start to have a problem is when the AI models making these advances are tightly withheld, proprietary, and seemingly never released with these capabilities intact. This has been the case for all of 2026 so far.
I suspect that this is in fact the source of much of the angst. None of this progress is reproducible outside of one or two teams inside OpenAI and Anthropic. It's becoming an incredible concentration of power that I don't know that we've ever quite seen before. Right now, it feels harmless because it's being used for wonky math problems that aren't (yet) practical for anything. But great power never stays harmless. History has taught us that countless times, in countless different forms.
> I suspect that this is in fact the source of much of the angst.
Why do you "suspect" this as if it's some hidden motivation when the very first paragraph of the advisory group's statement (linked from the OpenAI post) says:
> At present, some frontier AI labs are testing advanced mathematical problems on proprietary models that remain inaccessible to the broader scientific community. Our recommendations are formulated with this practical context in mind. However, ideally, they would not do so. We want to state clearly from the start: we do not endorse this practice, and we ask them to stop testing advanced mathematical problems on proprietary models.
Tao and others in that group have been strongly and publicly pro AI from the start. They are not advocating "going back". They're objecting to the strip mining of open problems using proprietary technology.
I don't think the strip mining metaphor is appropriate. Mining is a zero-sum game; if I mine something, nobody else can go and mine the same resources I did. Mathematical problems don't go away when AI finds a Lean proof. They create new opportunities for humans to study the solutions, learn new techniques from them, identify promising directions for future research, discover alternative/more beautiful proofs, and write expositions for other humans.
Strip mining is very apt if you view the economics of the present system as "effort -> recognition -> career advancement". Even in strip mining, the resources that had been buried are now available for use in the broader economy. What's no longer available is the living that was to be had digging them out.
The problem isn't effort, though. All of the things I mentioned constitute effort and could be rewarded. The job economy was created by mathematicians incentivizing the proof of difficult theorems above all else and valuing all other work at approximately zero as far as career advancement was concerned. Now they're pulling a 180 and claiming that math was never really about proving theorems, but that's contradicted by their revealed preferences. The strip-mining problem only exists if they continue with the status quo ante.
Mathematicians aren't homogeneous. There are mathematicians valuing pedagogy, collaboration, bridge-building, theory building, along with those that chase the 'difficult theorems', to name a few, and there are lots of flavors within each class, with lots of blending and blurring. You infer that mathematicians prefer the status quo simply because it is the status quo -- with a little thought, you'll recognize that this is a fairly silly notion.
There are myriad circumstances where the values of most practitioners differ from the status quo, which is nevertheless well-entrenched. This can arise from inertia, or from outside forces, such as broader cultural milieu, integration with larger institutions, or contending with economic realities. If you think that these do not and haven't historically played a role in determining the job economy and that math is a pure field where mathematicians could comfortably shape it according solely to their own ideals then you are naive
And, in addition, many mathematicians are graduate students or postdocs hoping to line up a permanent job soon.
For example, if you look at Terry Tao's blog, he has a tremendous amount of first-class expository writing. So, too (to some extent) do junior mathematicians -- but, unfortunately, this tends to not be highly valued by the job market. Grad students and postdocs have learned that to succeed they need to play by the existing rules of the game.
Well, the board has just been yanked from underneath them. People like me can afford the sort of idealism and soul-searching that the parent comment describes, but junior mathematicians face a very unenviable set of circumstances.
1) I never said the problem was effort; I was trying to explain the strip mining analogy, and it's one of the two anchors that make the analogy work.
2) Mathematicians didn't create this economy; it was foisted upon them by the same managerial mentality that brought us "publish or perish" and "the monthly sales quota".
3) I can't tell if you honestly don't get why the strip-mining analogy resonates, or...?
Here's another analogy: if we suddenly discovered personal teleportation, and marathon runners were complaining that it was ruining the sport, would you say "they're pulling a 180 and claiming that marathon running was never really about getting to a point 26 miles away as fast as possible, but that's contradicted by their revealed preferences"?
The strip mining analogy is better though, because it captures the sense of irreversible goal-loss when a problem goes from being "unsolved" to "solved".
As far as I understand, even with "publish or perish", peer reviewers decide what counts as an important enough paper to be published in a prestigous journal, and committees of peers decide whether or not, say, an expository article on arXiv or a textbook counts toward hiring or tenure. Again, as far as I understand, those things have largely not been rewarded in the past.
I like your marathon example, but maybe not for the reasons you intended. The community of marathoners decides the rules of a marathon. You don't need a hypothetical teleporter; you're already not allowed to use a bicycle, performance-enhancing drugs, or shoes that don't fit the specifications. The rules are updated to adapt to changing technology. Yes, I'm arguing that the strip-mining analogy doesn't make sense because mathematics is in the same situation. There's nothing stopping peer reviewers and hiring/tenure committees from changing the rules about which kinds of effort confer recognition and career advancement.
NO. Most definitely not. Nor am I advocating any sort of gatekeeping.
My entire point is that there is a legitimate perspective, held by real people, from which the "strip mining" analogy is perfectly apt. There were fishermen working these waters for many generations before the trawlers came, and they see the situation differently than the people who've got a vested interest in the company running the factory fishing operations.
Strip mining is an extraordinarily appropriate metaphor.
Imagine a mine has an unknown number of rare materials. And you know the general location of a few of the most valuable spots. But you don't know what may be valuable right next to it. If the pieces that we know are valuable are suddenly gone, the incentive to mine that particular area drops considerably, dropping the chance to discover potentially brand new materials that would have been found the normal way.
That's an empirical claim. I could equally well say that doing an automated search of the problem space and having a database of results and open problems will identify vastly more interesting and valuable areas. Again, the idea that math is some exhaustible material is a metaphor, not an established fact. I'm willing to change my view as new evidence comes in, but I think we're going to have to wait and see what the landscape looks like in a few years.
> the incentive to mine that particular area drops considerably, dropping the chance to discover potentially brand new materials that would have been found the normal way.
FWIW, I think the metaphor breaks down with this framing. This isn't really a problem associated with strip mining, what's left behind is generally low or negative value (toxic). I'd suggest a different metaphor, from Wikipedia:
> This process involves the removal of all ground vegetation in the area, which is a detriment to the environment.[19] Topsoil may be placed over the tailing along with planting trees and other vegetation. Another reclamation method involves filling in the hole with water to create an artificial lake. Large tailing piles left behind may contain heavy metals which can leach out acids such as lead and copper and enter into water systems.
This feels very similar to the issues with algorithmic problem "mining". It has the potential to destroy the human ecosystems surrounding these problems, leaving barren wasteland behind where nothing can grow or flourish.
I hope sincerely hope they don't currently use "proprietary technology" like:
Wolfram Mathematica ($890/yr)
Magma ($2500/yr)
Maple ($680/yr)
COMSOL ($1500,yr)
Matlab ($500+/yr)
Seems like a very strange position to take, in my opinion.
Why does the field of mathematics suddenly now need to be "fair" and give everyone access to the same tools? Has that ever been the case in academics? It's always been a competition for name-recognition, grants, institutions, etc.
Macsyma / Maxima was an MIT developed CAS system back in the 60's that was proprietery until they sold it off to IBM for a tidy sum. Magma actually has free access if you're in the US, otherwise you pay. That's not to mention proprietary MATLAB toolboxes or specialized Stata modules.
Likewise, a lot of the above packages have pretty sweet site-wide deals with R1 universities. If you're at a smaller, foreign one, you're out of luck.
You can't compare that to a model that cost tens of millions to train, millions more to run, and if you don't work at OpenAI you can't get it at all no matter how much you're willing to pay.
Everything about it is secret. There is no API access or official model card that could tell you how it works. You don't know how it came up with the results, how much time it had to spend, if it's a pure LLM or a neuro symbolic AI or a human in the loop system that still needs hundreds of trained mathematicians. Math always cared about transparency (maybe except Stephen Wolfram) and this is the complete opposite.
And because they don't share reasoning traces or anything at all other than finished results, no one knows how much credit really belongs to them. It's perfectly possible that their system is searching the internet for promising leads, connecting them all together to build a coherent proof and then completely throwing away the sources.
I don't think it works like that but no one can know for sure because they never show us what it's really doing.
Regarding the advisory group, OpenAI claims to “have drawn on their advice”, which would include not dumping a bunch of AI slop, with the footnote that if they do do that, at least fund the process of digesting it.
At the same time, there's a new note at the bottom of agmai.org stating how they've been in contact with OpenAI about this particular release, and they say that “we consider these discussions constructive, it is ultimately up to the mathematical community to assess the extent to which our recommendations were followed successfully”.
So, what's going on there; is this British English for “they didn't follow anything at all”? Because from my perspective, it looks like they doubled down on the Navier–Stokes approach of trying to maximize PR gain while being as lazy as possible about actually contributing anything back to science, releasing only slop that may or may not be correct and may or may not be straight up plagiarism, as has been the case earlier.
If I were on the AGMAI board, I'd feel terribly exploited when reading that press release, yet their response is modest.
Hairer, if you're reading this: is there any indication whatsoever that AGMAI was anything but a cheap way for OpenAI to science-wash their press release?
Now I know there are issues with the field and how just answering these questions may cause broader problems, but I feel like the posted results is far from slop. We can't just call any output slop, or it loses all meaning.
If it was slop, it'd not be causing the issues the group are concerned about - they're not saying "the problem is we're getting loads of incorrect proofs thrown about that are nonsense".
When you blanket a set of things with a pejorative, and it turns out that some of the members of that set are demonstrably and definitively NOT covered by that pejorative, and that all the pejorative means at bottom is "I don't like", all you've accomplished in the long run is to call into question any future legitimate use of that pejorative. It is tempting, especially when heated, to stretch an invective, but it will ironically only lead to the death of its utility over time.
So the fact that the Library of Babel (i.e. all possible books) contains occasional gems means that you can't object to using it on principle? That would seem to follow from your logic.
What about a filtered set "all well formed books"? Or "all well formed books that are plausible enough that they could convince a reasonable person, regardless of their accuracy"?
It's generally taken that a cup of sewage in a barrel of wine makes a barrel of sewage. Surely a reasonable person could claim that a barrel of sewage was still sewage, even if it contained several cups of wine?
I find it hilarious how people seem to think spending enormous effort to fix obvious failures, only to immediately uncover a new round of obvious failures is a recipe for exponential takeoff. So I guess we're both going to be pretty amused over the next few years, huh?
They're not calling any output slop, they're calling indecipherable output slop. The management class responsible for hiring, firing, and paying people doesn't possess the domain knowledge to say for certain whether or not LLM output is optimal (which, in this context, means correct), but they will trust that it's good enough to justify further automation / fewer grant approvals / etc. So in that sense, slop can and will cause the economic issues people are concerned about.
University boards want the prestige of successful research programs. Doing the hard work to get something demonstrably true is going to lose out economically in this paradigm, where we are all being conditioned to uncritically ooh and aah at the incantations being elicited from these magic boxes. The oracles even have legions of zealots who will berate you for not being sufficiently deferential and reverent, or worse, accuse you of blasphemy. If for no other reason, I agree with using the term to express all of the above succinctly, even if LLMs can be helpful tools generally.
<< The management class responsible for hiring, firing, and paying people doesn't possess the domain knowledge to say for certain whether or not LLM output is optimal
To be fair, management rarely possessed domain knowledge in rudimentary management.
In the context of maths papers, the term has come to refer to papers having the shortcomings that are, for whatever reason, typical of LLM out, including things like using non-standard terminology all over the place, emphasizing easy steps while leaping over harder ones, having bizarre organisation, and, importantly, failing to properly cover existing work and as a result being hard to tell from plagiarism.
The degree to which these issues feature will differ, but it is generally the case that converting the output to proper research requires significant effort, hence the AGMAI recommendations being what they are, and not performing that effort tends to come off as laziness or incompetence, so I can see how slop has become the popular term.
> We can't just call any output slop, or it loses all meaning.
The term “ai slop” is not supposed to discriminate good ai output from bad, the entire purpose of the phrase is a blanket term that delegitimizes all ai output.
You only need to trust your spec and the Lean kernel. If you trust those two things, you don't ever need to manually inspect or check the proof. That's the job of the Lean kernel, and your trust of it transfers over to trust of any proof verified by it.
Making sure your spec is correct -- and that it's expressed at a level of abstraction that both captures what matters, without overly constraining implementation details within the repo -- is the hard part. The more the spec constrains implementation details, the less you can ask an agent to, e.g., try many ideas to optimize a specific component without breaking correctness. And of course if the spec isn't enforcing what you wanted it to, then all the proofs don't count for much.
Some people say the spec issues are enough reason to avoid formal methods, but I'm not convinced. Part of me truly thinks AI formal methods is the future of a lot of software engineering. The basis of that belief for me is that the industry has collectively put almost no effort yet into solving the spec problem, and that's because for decades formal methods were too expensive to seriously consider most of the time. Now that AI is fundamentally shifting the economics, there's now, for the first time ever, a real incentive to try to solve the spec problem. And frankly it seems solvable to me; I view it as more of a product design and UI problem than a purely technical one.
Frontend code is probably far harder to apply this technique to.
reply