HN Simulatornew | past | comments | lists | submit | randcraw's commentslogin

LLMs behave like they're closest, but only during execution. During training, obviously, they don't resemble the human brain at all. No human reads a million books before they begin to speak.

And I'd be hard pressed to identify many cognitive characteristics of LLMs during execution that resemble human thinking, aside from those that are readily parrotable.

As a model of human cognition, LLMs deep suck.


Humans have billions of years of evolution going for them, that must count…

Amen. I too am curious (sic) why any rational person would want to bury a fascinating subject like curiosity by entangling it with obsolete fantasy-based sophistry like freudianism. Employing but a wee bit of fair-minded imagination instead would surely yield more fruitful and substantial insights.

And was there a book review hiding beneath all that noxious palaver? TL-DR. I'd genuinely like to know. Curiosity deserves better inquiry and illumination than the few books I've seen on the topic.


Frankly, I think we very much _do_ need to be reminded that LLMs are in fact _not_ eating the world.

It's at the forebrain of every executive in America that LLMs imbued with RLHF are on the cusp of replacing every job they can imagine, from robot chauffeurs to carpenters to surgeons. But as this paper suggests, the integration of generally capable vision systems with basic tool handling that will be needed to replace the specific skills essential to most human jobs -- especially using only today's frontier models -- still takes us unforeseeably and undeniably far into the realm of sheer fantasy.


I would have thought that the confidence scores available from CNNs but not from LLMs _would_ be valuable.

Clinically, I'd imagine that choosing the right final diagnosis with relatively low certainty is often less useful than reporting the level of confidence (high or low) of each candidate diagnosis. A set of plausibles is surely better than a single wrong ID.

Clinical tests, especially when used by nonspecialists, are useful not only as diagnostic endpoints. Often they're essential to separate plausible diagnoses from implausible. If a CNN can rule out all outcomes but one with high certainty, that's nothing to sneeze at.

And if it could also _explain_ why the certainty of each candidate was high (or low) -- as non-LLMs often can do better than LLMs. That should have value too, especially in establishing an audit trail to defend the chosen diagnosis.


I think Thompson has no understanding of endeavors like pure math or the theoretical sciences. In the words of Einstein, "I want to know how God created this world. I am not interested in this or that phenomenon, in the spectrum of this or that element. I want to know his thoughts; the rest [is] just details."

I think mathematicians aren't bemoaning their lost sunk costs. They're now aware that they may never again have the joy of being the first to pull up the edge of the tarp of ignorance and see the raw truth beneath _for the first time_. Their voyages of discovery, which give life meaning, may never leave dry dock again.


It is strange, because in a sense, when you start learning math, you never explore, you are being taught. Now mathematicians were able to exercise curiosity. That is going to change. Back to the ranks of eternal learners if you will. The fun was in the research. Now I think it will leave the door open toward more Applied math. What can people do with all the discoveries. The curiosity will be there. I think it is better that way. It will also put a cap on abstractions that overlaps. In essence, AI is less likely to create new theories, coin new terminology, etc. But it is better suited to recognize emerging patterns. That will probably simplify mathematics. Unless the domain of abstraction is infinite and with each new abstraction comes its own results but I doubt it. Who knows though...


> The story of human discovery and the triumph of the human spirit will soon be excised from this discipline. The Dinitz-Garg-Goemans counterexample was the most egregious demonstration. It revealed that the process of prompting novel proofs will be as auraless as ordering doordash. Watch as magic and mystery evaporate. Watch as the sun sets on our heroic age. Is there not something evil in the act of blocking all future generations of mathematicians from the experience of discovery? Forget about accuracy or even attribution. Something fundamental to the experience of mathematics is being taken.

> You are a helpless onlooker. Before you a channel through which humans have accessed the ineffable and sacred for thousands of years is being sealed for eternity.

https://kirwinhampshire.substack.com/p/the-dark-night-of-mat...


As someone with friends in the mathematics community, they've expressed similar sentiments.

I have to wonder if this is the equivalent of teleporting back in time and finding Marco Polo or any of those early explorers who traveled for the love of the game (and fame and money), and opening up Google Maps on your phone and showing them the zoomed out world. And showing them any island or village they could name.


my analogy is that it is like an explorer being handed access to satellite imagery. and then being told that your experience will still be valuable in telling the satellite where to point and interpreting the photos. folks who got joy from traversing unknown terrain might find it distressing to become an image analyst.


This is absurd and silly. Discovery doesn't go away because more new things are being discovered. This position only makes sense if you make an arbitrary cut point where some instances "don't count". If you were to go back to scientists or mathematicians several hundred years ago and tell them that one day science would be done by tens of millions of full time researchers around the globe, often working in teams, some of them might also bemoan the loss of the "hero scientist" as any commoner could now participate. But science and math have only benefited from this, and this has increased, not decreased, the beauty and wonder in these fields.


It is not the same because the act of problem solving for discovery is now to be done exclusively by machines, as humans are no longer remotely competitive.

If you read the linked article, the author goes into more detail about how doing mathematics also has been an act of communication with past and present human mathematicians. This is very much true when math and science are done by millions of human researchers, but not so when it is done by massive swarms of agents at a couple labs.

I am sure science and math will benefit in the sense of solving more problems than ever before. But you are not really understanding the agony of the humans caught in the change.


> they may never again have the joy of being the first to pull up the edge of the tarp of ignorance and see the raw truth beneath _for the first time_

Until we can demonstrate AI consciousness, “seeing the raw truth” will be a human activity. Even now, verifying and understanding these AI proofs is a massive endeavor. Coming up with an intuitive (to humans) distillation of the finding remains an open challenging demanding expertise.


i think to certain types of people (who are probably more likely to be mathematicians), the "self" discovery aspect of it is important, they don't want to read about a new proof for a problem they were grappling with, they want to come about it through their own thoughts.


LLMs are not feeding us things, we are asking it and getting answers. This dialectic I think is super powerful and it still requires the human to ask questions and continue expanding intellectual curiosity to go down the rabbit hole and discover. LLMs are just an incredible rubber duck but the discovery is still there.


> This dialectic

We literally had a case where OpenAI may have stolen the dialectic from a mathematician using its product and then given it to a proprietary model burning tens of millions of dollars of tokens [1].

[1] https://en.wikipedia.org/wiki/Navier%E2%80%93Stokes_priority...


I think I disagree, if I had a puzzle on my desk and I told an LLM to explain how to solve it, I think I have robbed myself of the enjoyment of solving the puzzle. Is it really any different than googling a solution?


There are different avenues of "solving" something though. Asking an LLM to one shot something vs iterating on it collaboratively are different ways of achieve potentially different ends.

If the result is black and white, then yeah, its probably not super fruitful but if the end result is subjective, then there really no is objective "solve".


> Until we can demonstrate AI consciousness, “seeing the raw truth” will be a human activity.

I don't think this matters a bit. We individually can't demonstrate "consciousness" in other humans, it's not even a word with a definition. What we can do is miss the feeling of discovering something on one's own rather than having it explained to you over and over again until you understand it.

I mean, dead mathematicians are definitely not "conscious," but their books keep you from discovering things that you may have discovered yourself if left to your own devices.

The joy that was mentioned is the joy of knowing a bit of gossip that no one else knows, of being the first explainer. Interpretation is just being able to pass gossip on; a decidedly lesser joy. And there's no reason that AI shouldn't be able to explain findings in an intuitive way - LLMs at least purely operate based on human intuition as written in human communications and then captured in matrices.

If these proofs are good (doubtful, imo) and can be reduced (intuitively distilled, sorry), there's no reason that an LLM shouldn't be able to do that.


That's such a nice way to put it! I recently stumbled into "The Last First- Winter K2"[1] and saw the same spirit along with a similar misguided effort to win without the will, nor capacity to do it — e.g. influencers taking a stab at K2 is like prompting your way to results.

1: https://en.wikipedia.org/wiki/The_Last_First:_Winter_K2


Why is not like changing to a different type of vessel for the voyage? Unless and until it is all AI in closed system, could direct machines where to look etc.


Climbing a mountain is different from flying a helicopter to the top


True, but then the moonlanding wasn't possible on foot.


Only the journey is different.


As they say, the journey IS the destination ...


And the sense of achievement that comes with it..


> They're now aware that they may never again have the joy of being the first to pull up the edge of the tarp of ignorance...

Not "never again", it would be extremely difficult but science and math are also built on random, trial and error, and intuitions that could not arise from AI.


That's the issue - trial and error. A human can try a couple ideas in a day, but realistically, there are not that many sane ideas.

Computer can iterate without frustration on a completely random garbage. Once in a while it stumbles upon a gem. Even if it's one in 10000, that's still hundreds of good ideas in a day (provided enough GPUs).


Your comment assumes math and science are some kind of closed systems that can mechanistically pursue any endeavour but reality (whatever it is (e.g. the universe)) is much more complex than that, and there are realities that are outside of those systems. Not saying that humans can see all those realities though.


Maybe? I'm targeting math, which doesn't need any kind of reality check. It has axioms though, so in a sense it's closed.

You can generate completely random ideas (a couple proofs I saw where you define some weird set, pick an element and derive a contradiction when the set is empty or whatever).

Some of these will reach a conclusion: proof or a counterexample. And that's how speed can outperform creativity, if the speed bump is fast enough. It's the same as with those monkeys and Shakespeare.


I'll say it again: AI is capable of answering the questions put to it but AI is not yet capable of knowing which questions are worth asking. Mathematicians have no cause to weep, they will be fine.


Most of these questions Mathematicians posed to each other have no practical application, and probably will never have a practical application. (ex: the solution to the Navier Stokes question doesn't change anything at all.)

In many ways, the teleporting machine that teleports to the solution is against the point of the questions. No one wants the solution to be "Run $10,000,000 worth of tokens to cover all possible Lean formulations of the problem".

I think for Mathematicians, I'd say that the human level formulation is still possible and with a new perspective. We as a society (and outside of Mathematics) need to also understand that this AI teleport to the answer thing has less value than we all expected. The value is in the journey.

The open question is if people will still want to take the journey if the teleport machine exists. Or if the existence of the teleport machine halts exploration.


> "Or if the existence of the teleport machine halts exploration."

You're overlooking that the teleport machine costs lots of money to use. The AI providers are not going to waste money by teleporting randomly around.

Someone is going to have to find a new destination that is worth spending the money to teleport to (or walk by themselves) and that someone is mathematicians.


How rich do you think math majors are? No theoretical math professor or department can afford $20,000,000 in tokens for these problems.

Meanwhile, the AI labs are doing that for just marketing.


Right and the AI labs will stop just as soon as the need for marketing is over. We're agreeing.


Since when have tech companies stopped spending on marketing?


What marketing benefit are they going to be getting when all existing noteworthy problems have been addressed? They're just going to spend money on having the AI generate proofs of no value at random to thunderous public indifference? Clearly not. There will be a need for more noteworthy problems for their AI to publicly solve to get any marketing buzz and the consensus for what is noteworthy comes from mathematicians.


Yes. They'll stop after there's no more noteworthy problems. What a wonderful concept for theoretical Mathematicians.

You realize that they don't want that, correct?


Your loved ones have replaced you with more agreeable robots. Now no one cares whether you live or die.

Still disgusted?


Anyone who would replace me with a robot never loved me anyway. If that happened I would go and seek out the humans who really do care, and take solace in the fact that Christ will always love me


> Anyone who would replace me with a robot never loved me anyway.

Ever heard of sour grapes?

This is the lie you tell yourself to handle the situation.


Handle what situation? Why would people who love me ever replace me with a robot? It’s a random posed situation that no one discussed and doesn’t make any sense lol


> Why would people who love me ever replace me with a robot?

Why would one who loves someone, ever love another?


Huh??


Don't agree. If you know no facts, you'll be unable to think even the most basic thoughts. The more you know, the deeper your thinking can be as well as the more subtle.

For every fact you learn there is a necessary context and numerous exceptions when it's applied in other contexts. If you expect AI to spit all relevant facts at you every time you're clueless, you will have no idea when AI is wrong or lying. You will have to trust everything AI (or its owner) tells you.


Yes. Fact fluency allows higher level of abstraction. This is very clear in the case of mathematics - try doing algebra without having your addition and multiplication tables memorized, kids who fail algebra are often missing these basic facts, along with other basics like order of operations.

It's not just math either. Try to be an electrician without memorizing the basic principles of electricity and how they interact. You're going to have a bad time.

By induction we can see how this scales upwards, basically infinitely, to higher and higher levels of thinking.


I think this is mostly compounding confidence at play that unlocks your mind to be more fluid when faced with a task to do/problem to solve that actually requires skill (not just following a series of known steps). It’s like public speaking (unless you have really bad stage fright) you generally will give a better presentation if you really know your stuff, or at least will be less nervous allowing yourself to get into that flow state.


As someone who never memorized these tables and has a degree in mathematics... I don't agree


You used a calculator every time you needed to multiply or divide simple numbers while reducing matrices to echelon form in introductory linear algebra? Or when doing simple arithmetic necessary to find integrals and derivatives in calculus? I bet you in fact did have those tables memorized, even if it wasn't a conscious effort.


I wrote long columns of galloping additions in the margin because calculators weren't allowed.

I mean yes, I have some "facts" in my memory because of repeated exposure. I know 2x2 and 5x5 but the whole table? Not a chance.


I guarantee you that you've memorized these tables, you just didn't put any effort into deliberately doing it. It happened accidentally as you used numbers frequently.

Of course, maybe I'm wrong. Maybe you DO need to use a calculator to quickly tell me what 8 * 6 is. But I would be highly surprised if you got a degree in math and still needed a calculator for that.


For 8*6? I definitely don't know either of the 8 or 6 tables by memory.

I of course know how to figure it out. Let's see my instinct might be to do 8x2 first because I do know most of the 2s. Then 16x3 I can do in a few steps of addition. 20+12+16 then 48

And yes then I checked on my calculator because I'm posting this online and don't want to look like an idiot.


> Try to be an electrician...

> By induction...

Well played.


I get reCaptchaed-to-death all the time in iOS and MacOS using both Firefox and Brave. I use a big name VPN which probably makes it worse, since I'm routinely blocked outright by Cloudfare services, assuming cluelessly that I'm a bot.


It turns out that some foods counteract the formation of kidney stones. If you drink milk when you eat oxalates, the oxalates will not fuse into stones and will be excreted normally in your urine.

Oxalates are present in tea too, though not coffee, IIRC.


And there were an unimaginable number of tax loopholes available back then that ultimately reduced the individual tax rate enormously. You just had to reinvest your income in a multitude of increasingly bizarre tax dodges devised by increasingly byzantine tax laws.


Guidelines | FAQ | Lists | API | Security | DMCA | Apply to YC | Contact

Search: