We have thousands of years of precedent that suggests that breakthroughs in mathematics tend to accumulate into broader technology breakthroughs in other domains.
That's a misconception. Only a tiny percentage of mathematics has seen any applications whatsoever. There are vast libraries full of mathematics no one (in this discussion, anyway) has ever heard of that no one reads anymore and has never been applied to anything.
This idea of trying to "prove all the math" with AI makes as much sense to me as using chess engines to try to "solve chess."
> There are vast libraries full of mathematics no one (in this discussion, anyway) has ever heard of that no one reads anymore and has never been applied to anything.
And that's an issue why? It would seem to me that producing that also produced the mathematics that revolutionized the world repeatedly for centuries. I would go further and claim that, if you want the mathematics that revolutionizes the world, there's no way to get it without advancing mathematics as a field broadly. Those are not two separate activities, and thinking that they are is indeed a misconception.
> This idea of trying to "prove all the math" with AI makes as much sense to me as using chess engines to try to "solve chess."
You're right: "prove all the math" does not make sense on any level, and nobody serious would phrase any of this in that way. I certainly didn't.
The issue is SNR: signal to noise ratio. Generating exponentially more mathematics, particularly if the process is indiscriminate or optimized for something other than usefulness or mathematical relevance (such as optimizing for machine-provability), does not imply that we get exponentially more applications. We may end up halting the progress of applications altogether as the entire capacity of the world's mathematical apparatus is consumed by the interpretation and investigation of machine-generated proofs.
You can already visit arXiv and find vast numbers of not-yet-published mathematical papers. Most should never be published. None of this junk is benefitting humanity in the slightest.
It could just as easily be the opposite: it could end up being far easier to reasonably direct and evaluate the research direction and output of AI systems than human mathematicians, who are forced to specialize over decades and essentially cannot pivot and often can't even meaningfully evaluate each other's work.
Moreover, the disdain you have for low-value output in mathematics is not unique to you. Talented mathematicians don't like it either. Your mistake is assuming that AI will cause math to be dominated by low-value outputs. In fact, the opposite is likely the case: the marginal value of proofs will fall so low that the bar for meaningful research will become dramatically higher, not lower. I expect the goals of research mathematics to become extremely ambitious relative to the past, organized around substantial and enormous goals, not mass-generated slop as you're imagining.
Of course, yes, there will still be lots of slop, just like GitHub is full of AI coding slop, LinkedIn is full of slop, etc. But that's a generalized issue of the AI era, not unique to math.
Moreover, the disdain you have for low-value output in mathematics is not unique to you
I didn't say anything about low-value output. No one actually knows the value of any particular piece of mathematics within that deluge. Mathematicians don't have a magical ability to differentiate high-value mathematics from low-value merely by reading paper titles and abstracts.
The dirty secret in the mathematical world -- that has been going on for a long time already -- is that papers get attention based on the reputation of the authors, not on the rigour or validity of the proof. The big headline-grabbing papers are getting read by mathematicians because AI researchers have leveraged media exposure to bypass the reputation network, but media exposure doesn't scale.
When everyone is using LLMs to generate proofs, only reputable mathematicians will be able to get their work read. And herein lies the crux of the problem: an exponential takeoff in the volume of output from respected mathematicians will leave a critical shortage of readers.
it could end up being far easier to reasonably direct and evaluate the research direction and output of AI systems than human mathematicians
That's baseless speculation. All indications so far are that LLMs produce proofs far longer and far more complicated than humans are capable of, such that only machines can check the proofs for validity. Digesting them into a human-readable interpretation of the results is an open problem.
False. You very plainly did. You simply used the term “junk” instead.
> That's baseless speculation.
It might be speculation (as is much of what you’re writing), but it’s not baseless. Obviously, it’s quite easy to direct AI agents, a single one of which can pivot across all of mathematics, unlike all human mathematicians.
> All indications so far are that LLMs produce proofs far longer and far more complicated than humans are capable of, such that only machines can check the proofs for validity.
I’m unaware of any clear evidence of this. Hence, it appears to be baseless speculation.
> Digesting them into a human-readable interpretation of the results is an open problem.
I’m unaware of any clear evidence of this. Hence, it appears to be baseless speculation. Moreover, and more importantly, to my knowledge there hasn’t been any meaningful result in AI mathematics so far that has posed any kind of blocking issue on understanding it yet.
No, they're saying that what is true in mathematics is contingent, not absolute. It all depends on which set of axioms use, what assumptions you make.
The area of a triangle doesn't have 1 unique formula, it has many, depending on the system you use. A triangle in plane geometry has a different area than a triangle in spherical geometry, and different again in hyperbolic geometry.
When you get to studying the geometry of manifolds, you realize the area of a triangle can be any damn thing you want, depending on how you construct the manifold you embed it in.
I don’t recall anything specific off the top of my head but I am confident that such a proof would have immediate actionable implications.
Furthermore, careful analysis of the latter would as likely as not yield further understanding and, actually /would/ help finding such algorithms.
Finally, it has been observed time and time again that often (again, nothing comes up and i don’t want to ask AI) the certainty that something is possible and has been done is motivation and inspiration enough for people to independently solve a problem. Sometimes it is even enough for someone new to simply not know that something is “hard” to solve.
Of course this is all pure speculation concerning a hypothetical proof that most likely doesn’t exist, or indeed might be so complicated as to not be approachable even after hundreds of lifetimes of study.
Nevertheless your conclusion does not follow from the premise
A proof that they are the same is of no use either, since it too wouldn't help you find algorithms that are faster.
You would need an algorithm that finds solutions, not just a proof they exist. So the value here would almost entirely come from how you proved p = np, since that proof will probably be the first step towards finding the polynomial solutions. But if humans don't understand it good luck finding any.
I assume an expert mode (where you can sideload anything you want) is what you are asking for. To that I raise the standard objection: how do you protect grandma from having a criminal social-engineer her into enabling expert mode and then having her sideload malware onto her device?
I mean if you're gonna stop people from owning their device and doing what they want with it just because of the possibility of crime occurring then their are quite a few more things that probably should also no longer exist because of their potential use in crime.
That's a pretty ridiculous argument if you ask me.
You can say that about a lot of things in life. Legalized gambling, for example. Tons of people ruin their lives with it and this creates big externalities for society. But we tolerate it so that everyone has the freedom to gamble.
Apple on the other hand offers a locked down product that many people want because they don’t trust themselves. I think a lot of people have short memories for all the hacking and spyware that plagued Windows users a couple decades ago. I think we also need to remember that Apple is not the government so people have alternatives to buying their devices.
Personally, I think people should have the right to buy a locked down device. It’s not on Apple to provide open competitors to their product. And why should developers have a right to develop for the platform? Heck, I wanted the iPhone back when it didn’t even have 3rd party apps at all.
> Apple on the other hand offers a locked down product that many people want because they don’t trust themselves
The idea that, for many people purchasing an iPhone, the inability to install apps from alternative stores or the web is a selling point seems rather questionable.
On top of that this could easily be provided by an optional switch that enables you to enforce this on your own device if you so desire.
It is a poor reason to request that all app distribution for everyone else also has to go through Apple and its terms, basically for your convenience.
I’ve already given one reason why the switch is a bad idea. Another reason is that powerful third parties such as Meta and Google could force users to disable their own protections and give those apps full access to all data as a condition of installation.
Ultimately, an option becomes a requirement and then the protection for users is totally gone.
>>hacking and spyware that plagued Windows users a couple decades ago
And that still plagues every computing platform now. Including Apple platforms. So the lockdown clearly didn’t completely solve that problem like a lot of people seem to claim
That is exactly what Apple has decided to do. You have named the tradeoff. Stop the possibility by disabling everyone. It has always been this way. People have had nearly 20 years to leave Apple
Are they finally going to fix the issue of having introduced a second right click menu that forces me to click “show more options” literally every single time I want to do anything?
Sadly I’m in a locked down corporate environment, so I can’t edit the registry. IT even closed my ticket when I asked for them to make the edit on my behalf.
I don’t use Windows on any personal machines, so there’s no way for this to benefit me. I hope it helps others who are frustrated by this though!
What, you don't find it charming that they shoehorned in the old right click menu without even changing its appearance to match the rest of Windows 11? It really shows how much thought Microsoft is putting into Windows now.
I'm not so annoyed by it except that I liked the old design (smaller text) better. What puzzles me more is why it takes >1000ms to open a simple right click menu? And even after it opening, it often has to "Load" some options? In what world does it take over a second to open a simple popup on modern hardware?
You don’t have to get paid to make contributions to math. There are still loads of hobbyists making contributions around the world. The guy that found the aperiodic monotile was a hobbyist who collaborated with a researcher. There was a significant proof in combinatorics (lower bound on superpermutations) contributed by an anonymous 4chan poster!
Hobbyist mathematics may long outlive a lot of research mathematics, and I think that’s okay. There’s nothing in the human spirit of inquiry that necessitates academia as an all-powerful institution. Heck, there have been a lot of problems with the institution in recent years, and those have everything to do with perverse incentives and nothing to do with AI.
Major reason I don't use password managers that much.
> Do you use the same password on every site?
I use a password I can reconstruct in memory for sites I care about logging in by hand. If I don't, or don't mind resetting my password each time, I just use random garbage + whatever platform password manager is the one active today, with vague hopes that it'll still be there the next time I need to log in.
> How do you deal with data breaches?
Who ever cares about those? I'm yet to hear about anything impactful being released on those. It only matters if you actually do reuse the same e-mail/login and password combinations on both important sites and garbage sites. Which is something you should not. But 2FA and magic links tend to solve that vendor-side, these days.
I use KeepassXC and Syncthing. It's not a single password manager doing both the managing and syncing, but it works very well, and I sync between phone/laptop/desktop without any third parties.
Keepass database is an openly documented format, and there are several apps available on iOS that use it and can store the database in your iCloud (ie syncthing is not required, unless you need to sync to non-iOS devices)
Just search for keepass in the app store (Keepassium, KeePass Touch, Strongbox, ...)
Right, that’s what they do in sci-fi books and movies. But then those collars magically never need batteries replaced, service, or replacement. They also don’t have any issues with false positive detonations.
If the collar can be deactivated to be removed for servicing/off duty/retirement/replacement/etc then it can be hacked and replaced with a placebo.
No, not morons, but people who have built a career on string theory. At this point, even if they regret their decisions, it’s too late to turn back now.
I don't think it's a good argument. String theory is just "one application" of advanced mathematical physics. It's like saying that software developers specialised in ReactJS would lose their career if ReactJS is suddenly abandoned.
Not a fair comparison because one is exposed to real life market forces vs. academic inertia.
ReactJS devs will retrain when the market dies. Professors still publishing theories/experiments costing large amounts of public monies better spent elsewhere.
There is no more inertia in the academic sector than in the software developer sector.
You have tons of software developers that cling to their preferences for ages. They have their beloved language, or beloved paradigm, and they will invent any excuse to explain how relevant it is. It is also visible with current LLM, where there are plenty of software developer islands where they blanket ban LLM usage as an emotional response. There is a big contrast with the academic sector where the most sceptical people did not blanket banned it, but find way to put guardrails (and I'm sure you can find anecdotal places that are emotionally against LLM in academia, but it is an order of magnitude lower than in the software development sector).
Same about "real life market forces". Especially when at the same time people criticize, correctly, the "publish or perish" hyper-competitiveness problem in academia. The reality is that a physicist need to retrain plenty of time, probably as much as a ReactJS dev. But that reality flies high above the head of people who jump into the first caricatural idea they have about the sector.
As for "costing large amounts of public monies", you don't seem to even know what you are talking about. String Theory probably costs society way less than plenty of other useless things, and still managed to provide very useful tools that are used in mathematical physics outside of string theory (not that I'm saying it was worth it, but it is just not fair to pretend that they did not produce anything). In contrast, the software development sector has a huge contribution on enshittification, stealing user data, making porn, creating social media addiction and creating financial bubbles that splash in the whole economy. Between the two, there is clearly one that is way more toxic on the society.
Don't get me wrong, I'm not interested in "taking side" and particularly defending physicists. I'm sure they are making mistakes and wasting money, like any other sectors, and that we still should find these places and fix them. But your depiction is so caricatural that I had to rebalance the situation by reassessing the facts.
There is no more inertia in the academic sector than in the software developer sector.
That is not true. In academia you have extremely niche research specializations and tenure as two major contributors to inertia. In general software development you have an enormous amount of career mobility.
Software developers learn new languages, new frameworks, new tools, new development methodologies all the time. A string theorist who decides to pivot away from string theory essentially has to start their career over from scratch. It’s not even close!
It's not even clear where String Theory is categorized in this article. Maybe High Energy Physics? Maybe Condensed Matter? It already shows how broad it is, as they already consider that people with such a different specific subject (for example theoretical dark matter in HEP or superconductivity in Condensed Matter) are not even "changing field" when they move to String Theory.
In this article, you can see on Figure 1 that about 65% of physicists (similar in HEP and CondMatter) are also working on subjects that are in one other category (so, it's not even that the person is doing String Theory and another subject inside their own group, like dark matter or superconductivity, it is more like String Theory and Astrophysics).
In Figure 2, you can see that 38% (resp. 21%) that have started their career as specialist in HEP (resp. CondMatter) ended up changing specialisation within about 5 years. (and this is "voluntary", it means that "changing specialisation" is in reality pretty easy, as of course the majority of people could change easily but just decide not to because they don't see the point of changing)
Even without this study, you can easily check the reality yourself. If I do a search on "computational tools for string theory", the first result I get is https://compstring.org/.
First, if you look at the list of participants, you will notice their CVs and how diverse they are, with background or even current activities in: SUSY, QFT, CFT, condensed matter, cosmology, quantum gravity, black hole thermodynamics, ...
Second, if you look at the software, you will see the majority are very recent (2025, 2023 and 2025). Each of them require learning the specific framework, with new rules, new ways of describing the problem and new ways of making the software do what you want. This is similar to passing from ReactJS to Angular (both are about 10 years old, so in fact, the String Theory community seems to change framework more often than devs).
You can also look up in wikipedia, the famous names in String Theory, they all have plenty of activities that go well beyond string theory (which means the reality is even stronger, as they were not even forced to do something else, they've done it because it was trivial for them to also work on these subjects): Maldacena and Susskind on black holes and wormholes, Shenker on quantum chaos, Seibert on SUSY, Strominger on gravitational waves, Fischler on blackholes production in particle colliders, Kounnas on GUTs, ...
> A string theorist who decides to pivot away from string theory essentially has to start their career over from scratch. It’s not even close!
That's just factually not true. And I've provided data that demonstrate that. It looks like you have no idea what a string theorist is doing and how they can transition to another field.
You were the one introducing the argument "they are getting advantages out of it so we should be careful to not take what they say as an impartial view". Same can be told of Hossenfelder (personally, I would say that Hossenfelder has shown more clues that indeed she is not very reliable and would prefer caricatural sensationalistic descriptions, which is not what the typical string theory physicist does).
No, my point was only to refute the idea that people need to be egotistical or moronic (the only two options) to cling to an idea past its expiry date. People could also be “pot-committed” career-wise. Or they could have myriad other reasons (identity, sunk-cost fallacy, etc).
Nowhere did I suggest that “the reason for discrediting string theorists is that they have financial stakes in the idea.” Please try to be more charitable than that.
> No, not morons, but people who have built a career on string theory. At this point, even if they regret their decisions, it’s too late to turn back now.
This is not presented as "yet another option that may or may not be the reality", this is presented as your conclusion of why we observe what we observe.
Maybe it is not what you are thinking, but you cannot blame people for interpreting your message the way you have written it.
I agree with you that there are several options, one of the most probable, that you did not mention, is that the situation of the string theory is just "normal" and some idiots are not able to understand that.
We don’t trillions of dollars in LLM investment to build things mathematicians don’t understand. We already have plenty of those, even from ancient times.
As to your second point, Terry Tao already has an answer [1]: the proof isn’t the contribution, shared understanding is. This issue was already raised back when the four-colour theorem was proved. Machine proving and machine proof checking are useful tools but they don’t mean anything without the interpretative work and the communication necessary to build shared understanding.
If humans have nothing to contribute then shared understanding is a pointless endeavor. It makes sense now in the "centaur" period where human + AI > AI alone, but when AI mathematicians are both more rigorous and more elegant, then taking the time dumbing down their proofs to a human level of understanding is like requiring that we ensure all our current proofs be understandable by a monkey.
> If humans have nothing to contribute then shared understanding is a pointless endeavor.
I agree, but as a software engineer this gives me pause because I keep trying to insist on coding standards but I’m unable to come up with a compelling reason why it matters. Ostensibly the reason we cared about things like DRY and code quality was so that it would be easy to understand and easy to maintain and easy to make changes to later. But it now seems like a shared understanding of the codebase is less important than ever, and it’s more about shoveling requirements in without breaking any existing functionality.
Is a well tested slopfest better? That seems to be the conclusion for mathematics, so why not software too?
IMO it's still a problem with LLMs; we still have to build in a way that makes it easier for an LLM to make changes later and arguably it's the same things that made software development easier for humans. IME LLMs tend to not know how to do that for themselves and instead just amplify/copy patterns that already exist.
If an LLM can't pave the way for itself then ultimately shared understanding is required to take advantage of LLMs in the first place.
It used to be the same with assembly. Programmers complained the one generated by compilers was not pretty, but now in 99.999% of the cases, it does not matter because nobody look at it.
I was more after the fact i can trust the compiler to give me the same result - even though via an optimized path.
Certainly scopes vary, but in my line of work i define memory layout and how this data will be processed myself - thus it's great a compiler might do that, but the result of the computation will not change.
Now in comparison giving an LLM specs ... i a) cannot be sure what the computation will be b) it might be something else on another run.
Where is the value in an unintelligible gibberish proof?
We already have countless examples of such filling up the arXiv, written by hacks long before LLMs started writing proofs. No one cares about them. You might as well build a box blasting radio static into the void. You could save a lot of electricity that way.
An unintelligible but correct proof is better than no proof. These first AI proofs may be overly complex and un-elegant, but they are the worst that frontier math proofs will ever be. AI math in 2030 will be leaps and bounds ahead of humans both in rigor and elegance.
Those are some strong claims. I’m deeply skeptical of all of them.
I believe Terry Tao when he says the bottleneck will no longer be the writing of proofs, it’ll be everything else: reading them, reviewing, publishing, and teaching from them. A bunch of proofs that nobody reads are of no use to anyone.
Why not? We build cranes to hoist weights construction workers can't lift. We build electron microscopes to measure things physicists can't see.
Why is it so hard to imagine we can build tools to think thoughts we can't comprehend?
If there's commercial value, I think it's inevitable. We don't fund mathematicians because it's cute when they understand a problem, but because their work tends to have applications with commercial value. The value can be captured without understanding the details.
Can you give an example of an incomprehensible piece of writing (any writing, never mind a proof) that has commercial value commensurate with the costs involved here?
> Can you give an example of an incomprehensible piece of writing (any writing, never mind a proof) that has commercial value commensurate with the costs involved here?
It depends on what exactly you mean by "commercial value commensurate with the costs involved" but I'd volunteer the 3G/4G/5G specifications and the other documentation required to implement the mobile network protocols. 5G is currently sitting at over 50,000 pages and it's one of the reasons Qualcomm/Broadcom/Apple are the only ones who can realistically make a mobile radio.
I don't think there is a single human to whom more than a few thousand pages would be comprehensible at a time except for the occasional genius.
Read the rest of the discussion. The claim is about text which is in principle incomprehensible to humans.
If there exists a text which only one person can understand, that person can communicate their understanding to others, even if that doesn't help them with the original text. That dissemination of knowledge is what provides the value, not the mere existence of the text. If that person forgets or dies before they can share their knowledge then it will be lost.
We have many examples of this from history: ancient texts written in a lost language. These texts provide us with no value until the day they can be deciphered, unless you count linguistic puzzle-solving as a virtue.
> If there exists a text which only one person can understand, that person can communicate their understanding to others, even if that doesn't help them with the original text.
If there exists a text which only an AI can understand, that AI can communicate their key conclusions to others, even if that doesn't help them with the original text.
The only difference here is the amount of meat involved. Perhaps tossing a few steaks on the server racks could help with that.
If an AI can communicate its findings to us in a way that we can understand it, then its findings are by definition intelligible to humans. Your claim was about texts for which this is not possible.
So, you're willing to claim that a conclusion that you don't understand the reasoning for is equivalent to comprehension?
Because, again, I can point to hundreds of examples of texts where nobody but the author understands it, and they're only giving summarized "commandments" that you should follow if you want good results.
If I told you "don't use spin locks, call futex instead", do you think have gained an understanding of the Linux scheduler?
If I told you "don't use spin locks, call futex instead", do you think have gained an understanding of the Linux scheduler?
Yes, because I already knew what schedulers are, what spinlocks are, and if I want to know what futex is I can go look it up. Comprehension is within my grasp.
Your original claim, which you’ve repeatedly distanced yourself from (by trying to use comprehensible examples) but won’t admit to, was about incomprehensible stuff. That is, text that no human could possibly understand, ever.
You’re repeatedly engaging in intellectual dishonesty rather than simply admit that “human comprehension probably will continue for the foreseeable future”, which is really not a controversial idea at all.
Bullshit. Human comprehension will not continue, it will reduced to "when the AI tells us to add Bismuth, strontium, and copper together in this way, we get a superconductor", but that doesn't mean we understand high temperature superconductivity.
For what it's worth, we've known about BSCCO for nearly 40 years, and we still don't have a great physical understanding of how it works, thought we made decent progress in 2022.
You're using a very bad definition of incomprehensible. We don't understand the chemistry behind how these things work, it's that simple. If you're going to argue it's comprehensible, you'd better show how it works and go collect your Nobel prize.
All we know is that if we melt the right rocks together, we get a superconductor that works in some mysterious way that we can't explain. We know it's not Cooper pairs.
Since it's comprehensible, please point me to someone that can comprehend high temperature superconductivity and how it works, and then explain why they didn't explain it.
With the definition you seem to want to use, it's impossible for anything to be incomprehensible, and therefore it's tautological that there's no incomprehensible LLM output: nothing at all is incomprehensible.
Do you have an example of anything incomprehensible? Anything at all? Even the things that Gödel would say are inaccessible could, in theory, become accessible: we just don't know with absolute certainly that the mathematics it was based on got all the axioms right, though we see no errors now.
Yeah, how many people do you think understand the Linux kernel in full? What percentage of the people using it to great commercial effect can understand it?
How's your understanding of Schroedingers "An Undulatory Theory of the Mechanics of Atoms and Molecules"? You seem to be using the results of it as applied to semiconductor engineering just fine. And, I promise you, most semiconductor engineers haven't read it in full, they just accepted the results as passed on by several layers of teacher.
I have a paper on routing algorithms, which I have attempted to read to my cat. I don't think my cat retained much, but they seem to be enjoying the cat food that got delivered using the results.
I'd suggest that we're going to be a lot closer to the cat than the author of the paper when AI takes off.
> how many people do you think understand the Linux kernel in full?
"Not many people understand some things fully" is so massively different from "the human mind is incapable of understanding some things that AI will understand for us"
Yeah. We're only starting the journey of building tools better at thinking than the human mind, so expecting me to have examples of things it produces is a little hard, don't you think?
The best I can do is things that are incomprehensible to nearly everyone, but still provide value. There's a small leap of imagination to consider an author that understands it and can show others how to leverage results without understanding be mechanical rather than biological.
You asked for examples, for a technology that we're still building. Maybe you can see the issue with that?
Anyways, people benefitted greatly from Newton's laws of gravity, even though we still don't have a quantum-compatible set of laws for it. The laws of gravity are still incomprehensible for people, but the approximation that we've observed is still immensely valuable.
No, I gave you a lot more leeway than that. Take any utterly incomprehensible piece of writing from the entire history of civilization and demonstrate its value.
You keep falling back on "incomprehensible for some people" but that wasn't the claim. It was about a text which is incomprehensible in principle; that is, utterly impossible for any human to ever understand.
His examples work fine but you aren't accepting them because they don't confine to your paradox.
Every writing must be comprehensible to at least the author, regardless of whether it has commercial value or not. If I hit the keyboard a few times, I've created writing, but it doesn't mean anything. It is just gibberish and without meaning, so there is nothing to try to comprehend. So if there is something to be comprehended, then at minimum the author should know it.
Therefore what you keep claiming is the only refute of your argument of "an [...] incomprehensible [...] writing" is actually a paradox, and cannot be disproved itself. However "humans comprehending things" is not a paradox, which means that your specific request to beat your paradox is not actually related at all.
His examples disproving the non-paradox version of your challenge (writing incomprehensible to folks other than the original authors) are sufficient to disprove your statement, as he gave examples of both people not comprehending human made and 'God' made writing (the universe/gravity)
That's because his examples are obvious and not at all interesting, since we've already seen them.
His original claim amounts to creating an AI that takes its place above humans as some kind of electronic God, delivering edicts to humanity that we cannot comprehend, but which somehow have value to us. It's unskeptical, pseudo-religious nonsense.
I don't see that at all. Nowhere did this person evoke placing AI above people or even remotely evoke religious imagery. You imagined that yourself, and that creeps me out.
Can you give me an example single piece of writing that the author didn't claim they understood? I don't think that any utterly incomprehensible writing exists.
If you want examples where nobody but the author understands it, examples are a dime a dozen.
So you concede the point then. The age of humans comprehending things is not coming to an end. And therein lies the rub. When it comes to intellectual labour:
Understanding == Value
If a mathematician produces something incomprehensible then it has no value. It's meaningless. Indistinguishable from random noise.
An AI which produces incomprehensible text is producing no value. We didn't need to spend trillions of dollars on LLMs to figure that out. Markov chains can do that job perfectly well.
Again, do you believe that there are documents, of any value, that humans don't understand?
There is no value in an undeciphered document until understanding is achieved, just as a lode of gold ore in some asteroid orbiting a distant star has no value until we can fly there and extract it.
If an LLM can help us understanding something then it was not incomprehensible, by definition.
Nothing is in principle impossible to understand. It just takes too long, is inconvenient and/or economically unviable.
I don’t find it hard at all to imagine that an AI comes up with a fundamental proof applicable to physics which results in some widget we can now produce that would otherwise not have been produced yet nobody takes the time to fully comprehend why it works. Somebody could, in principle, devote their lives to it and possibly get it, but for what purpose?
There are plenty of artifacts that, if not impossible for humans to understand, then at least no human has ever completely understood. To start with, the universe as a whole. Despite that, we are able to choose legible pieces of it to model and perform useful actions from.
This will shift your argument--that doesn't count! etc., to the point where it's by construction unsatisfiable and vacuous. And it doesn't matter: an LLM might e.g. break some cryptographic algorithm in a way utterly unintelligible to humans, but the fact that it works would be sufficient on its own to make all of us choose to abandon that algorithm and choose different ones.
One person need not completely understand something for it to have value; it's sufficient for there to exist a shared understanding.
an LLM might e.g. break some cryptographic algorithm in a way utterly unintelligible to humans, but the fact that it works would be sufficient on its own to make all of us choose to abandon that algorithm and choose different ones.
No, that is the entire point. If it is an algorithm which accomplishes something useful, then it is intelligible as such. That which is incomprehensible cannot be understood even in part, so it provides no value as a bit of knowledge (unless you're looking for a strong random number source, I suppose).
I know right? And there’s only a market for maybe 5 computers in the whole world.
Paul Krugman (1998): predicted the internet’s economic impact would be no greater than the fax machine’s.
The 1876 Western Union memo dismissing the telephone as having too many shortcomings, and the banker telling Horace Rackham not to invest in Ford because the automobile was a novelty.
The idea about the goal of mathematics being shared understanding seems to come at a convenient time.
Mathematicians have never been known to communicate their ideas very clearly.
Regardless, even that target llms will likely win - an llm will likely be more efficient at teaching me string theory than a professor in a room with 463 other students.
I don't think "shared knowledge" means "shared knowledge between mathematicians and lay persons" (there is no much point in that, the same way a smartphone technician knows how a smartphone works deep down to the details but there is no big interest for society to have every lay persons being informed about it). I think it means "shared knowledge between mathematicians".
And at this level, while there are anecdotical exceptions, mathematicians have always been pretty decent (with their conferences, workshops, paper publications, international collaborations, ...).
So, it does not mean "teaching the subject", it means "creating a human network of people that share the understanding". LLM can be useful at telling a human, but you still need a human. The point of Tao is not that LLM is not good at providing explanations, it is that "providing explanations" is not the contribution to science, "the human network" is. It's like saying "LLM are great cook, they generate tons of food in space", but the point of having cooks is so that people can eat food and not die. Having LLM generating mathematical proofs is as useless as having LLM generating food that no one can access: the point was never to "generate proofs" or "generate food", the point was "creating a shared human understanding" or "eating the food so human can survive".
I get that there is a cultural benefit to keeping it alive. Just like we ideally want the languages represented at the universities.
But keeping humans in the loop does not appear to be necessary in order to call it science, and certainly not in order to have progress or dessiminate that progress.
I don't have a problem with people doing math. As long that we don't idiomatically hold on to that way of doing things.
I do, however, find it hard to belive that individual humans will play a big role from here and forward, in any scientific desciplines.
The goal is indeed to have progress or disseminate that progress.
The point of Tao is that people see LLM providing "proofs" and are concluding that this is all that is needed to "have progress or disseminate that progress". That is the same mistake of thinking that "generating food" is all that is needed to "have people not dying of hunger".
The hard part of "have progress or disseminate that progress" is the human network. A fundamental point of this human network is that it generate trust, accountability and reliability. Generating "useful new theorem" is useless unless the society also built the trust around the theorem to distinguish it from a fake theorem.
Maybe in the future, we will have AI doing some part of it, but this is a totally different AI animal than the one we are able to have now, and people who think the current AI that we see now is able to do that have no understanding how it works. This is demonstrated by the facts in math: current AI is able to provide math proofs, and yet, a lot of human work is still needed to get progress out of current AI.
I would not bet that individual humans will still play a role as big as today in the future. Maybe AI will be different in the future, but the reality is that we don't have any indication if this is even possible.
Why is there a cultural benefit to keeping it alive, or to having universities? It seems like education is toil that could be automated for those that don't have fun with it.
That's a misconception. Only a tiny percentage of mathematics has seen any applications whatsoever. There are vast libraries full of mathematics no one (in this discussion, anyway) has ever heard of that no one reads anymore and has never been applied to anything.
This idea of trying to "prove all the math" with AI makes as much sense to me as using chess engines to try to "solve chess."