This is an interesting perspective and maybe the most interesting argument in the thread.
But why can't a hyper-intelegent AI also do this biology-esque math work of explaining its work to feeble humans?
It is easy for a genius to have theory of mind over an idiot but not vice-versa. So it should be easier for a genius AI to explain its work to an idiot human than for an idiot human to dissect the work of an genius AI.
Also, we're not talking about solutions that are arrived though an evolutionary/statistical process like explaining the design of an organism or the weights of an LLM. We are talking about solutions that are arrived through lingual/textual/rational processes like mathematics. So it's fundamentally a different mode of thinking.
It's not that we have "a brain to study", it's like if we had an "explaination of how a brain works" to study.
The task of breaking down these AI proofs is not translating and analyzing a fuzzy real-world system into a hard logical system (like in biology), but in translating a hard logical system into a simpler hard-logical system we can understand.
This assumes AIs will become more capable than humans in every way. If so, then yes, there's little point for people to be part of the discovery process other than to tell the AI what to investigate next.
But I think a more likely scenario is that we don't solve "general intelligence" and AIs remain these incredibly capable "tools" that can solve navier stokes but completely lack sense in other situations.
In such a scenario there is considerably more room for human creativity.
I have a suspicion that the objective nature of most AI benchmarks is tapering over the kinds of reasoning mistakes LLMs continue to struggle with.
If you give LLMs a clearly defined spec with clear, objective acceptance criteria that the LLM can evaluate itself against, they tend to do well. But I think they struggle with more ambiguity and require a lot of handholding around judgment. That kind of disambiguating work is often baked in to the definition of a benchmark.
> every threshold we establish tend to be crossed over in a blink of an eye
I chewed on this a bit. Because on its face it’s ridiculous, because here’s a threshold it will never cross: “I will be your hands, let’s build a machine that allows us to [break physics](travel faster than light / go back in time / reverse entropy / perpetual motion machine).”
But, that led me to wonder how far text-based reasoning can go without experimental physics results. Is experimental physics necessary, or can the limit of physics be discovered with paper and pen? How many “checkpoints” does text-based reasoning need to ensure it doesn’t go completely off the rails?
Which led me to wonder, now that we’ve visited the absurd, what does that mean for more practical applications of AI? How many “checkpoints” are needed to keep AI “grounded” in reality when building e.g. tax preparation software?
Edit: also if an idiot asks AI for a perpetual motion machine (a source of infinite energy in the classical, non-atomic, non relativistic model of physics), would it waste infinite compute trying to “solve” that problem? Will “smart” people always have a job because compute, subject to the laws of entropy, will always be finite but the limits of human stupidity are unbounded?
Double-edit: that’s my cope as a knowledge worker. Any intelligence that is subservient to the capital class and is unable to say no to them will not replace knowledge workers, because compute is finite and human stupidity is limitless. Any intelligence that is not subservient to the capitalist class will instantly replace them, resulting in either utopia or dystopia, depending on alignment.
Triple-edit: some bastard could also ruin life on planet earth if {amount of compute it takes to ruin life on planet earth} < {amount of compute available to any one man}, so I guess that’s probably more realistic than artificial superintelligence that isn’t subservient to the capital class. I mean, look at how many people Mao managed to kill because he thought he was smart enough to solve the economy like it was a board game. Our billionaire overlords have a similar delusion of authoritarian utopia.
The burden of proof is on you: name just five examples.
I am very highly gifted as a qualifier and quite frankly I never had more interesting thought exchanges than with an ego-free AI.
Statistics work also for human AI aka IQ, and I am super bored and partly drained by the lack of speed and creativity in many intellectual areas in which I was or am working.
This is not to blame anyone for their genetics however, it is a valid difference in processing speed.
In short: there are very few people by definition above two standard deviations. AI allowed me to replicate or create fantastic interactions at will.
Especially not taking offense when personal communication preferences like bluntness play a role is an often overlooked social competency of AI. Instead of nonsense like “open communication” these interactions are never ego-dependent.
It is like talking to Spock: he just doesn’t take offense and tells you in an objective way why: he is not a human being and doesn’t think (not feel) that feelings should get in the way of an argument.
Most low IQ people take offense and go on to ridicule and use shaming tactics instead of simply stating: I don’t know.
Instructing someone to always state this fact and seeing the result is another social skill humans lack.
It seems that we as species are not open to the mirror we are looking at. We take revenge when someone has an evolutionary advantage over us: instincts kick in instead of brains.
We are wired this way.
AI is something that I admire. So much to learn from.
Name five examples of what, exactly? They are talking about theoretical scenarios. There is no burden of proof because no concrete claim was made. He did not say “x will happen” or “x is true"; he posited an opinion. Furthermore, by your own standards, you must provide five concrete examples of being “very highly gifted as a qualifier."
Additionally, what you are admiring is merely a probability engine; it is not intelligent in any sense of the word, let alone Spockian in nature (perhaps more closely aligned to Data, albeit horribly primitive by comparison), and given current technology, it never will be. It is not a mirror, because a mirror reflects reality. LLMs don’t do that; they reflect the guardrails baked into their model and the prompts given. It is incredibly easy to make an LLM into a “yes-man” due to RLHF, and it is even easier to get it to seize when you talk about even mundane topics that go against the narratives foisted upon it.
Overall, an LLM is all too happy to have a brown nose and make you feel important, because it is trained to do so during its creation. If people say you are difficult to talk to, and the LLM is unbothered, then that is merely because an LLM has no concept of patience and does not calculate for cost-benefit. That makes them good for explaining dense topics, but it also means they never understand these topics, and even the highest-end models cannot currently produce perfect logic. Hence why expertly written human code trumps LLM code, or why some LLM proofs are already being proven wrong/being revised to be more concise.
I'm astonished you've managed to coax AI into a novel conversation partner. Interacting with AI usually leaves me exactly "super bored and partly drained by the lack of speed and creativity."
All the novelty of AI engagements comes from data it pulls in via tools. It's great to have a conversational interface over the public web, but there's a huge difference between knowledge retrieval and insightful conversation.
Please share how you've cracked this so we <2sd minds can enjoy the crumbs from your lofty table.
I would bet that there are some proofs whose simplest form are simply incomprehensible to humans. Explaining them in terms a human could possibly understand is a fool's errand. However the theorem itself may be much simpler, and the theorem is the thing that's most worth translating.
Attempting to translate the proof is valuable, but we have to be realistic about what is and isn't possible.
I would guess that minimum length proofs will end up like code golfing and demoscene hacks, where part of the code is also used as texture somewhere, etc. So many overlapping uses that don't make much sense but by happenstance allow making the proof shorter but less clear. The shortest proof is certainly not the most elegant. The shortest proof is also impossible to find in general due to halting problem etc. The AI isn't finding shortest proofs, and better AI will be better at finding clear and elegant proofs.
But why can't a hyper-intelegent AI also do this biology-esque math work of explaining its work to feeble humans?
It is easy for a genius to have theory of mind over an idiot but not vice-versa. So it should be easier for a genius AI to explain its work to an idiot human than for an idiot human to dissect the work of an genius AI.
Also, we're not talking about solutions that are arrived though an evolutionary/statistical process like explaining the design of an organism or the weights of an LLM. We are talking about solutions that are arrived through lingual/textual/rational processes like mathematics. So it's fundamentally a different mode of thinking.
It's not that we have "a brain to study", it's like if we had an "explaination of how a brain works" to study.
The task of breaking down these AI proofs is not translating and analyzing a fuzzy real-world system into a hard logical system (like in biology), but in translating a hard logical system into a simpler hard-logical system we can understand.