You’re only considering the front half of the network though, classification is a helpful example for explaining the concept - but if it follows through to the output then you could quite easily imagine just shunting some activation from one branch of the representation tree to another to steer behavior. Which we already do with steering vectors - but if you could read off the actual topology you could do that with much higher fidelity.
>I see this quote being thrown around a lot as if it supports the article's point, but this quote is just some of the most famously smart, skilled, and effortful people ever admitting even they cannot condense their knowledge that much.
But that’s explicitly not what the quote says - it’s not that they can’t, it’s that it takes work.
>The central problem is that the majority of people go into math because they love discovering mathematics and want to participate in the process. That itself is a huge part of the reward. That creates the community. Simply have huge numbers of people outputting math as if it were an industrial factory kills the joy in the process and so in itself destroys the community.
Seems like math is split in much the same way software is.
I’m a software engineer and not a mathematician, and, yeah I read this article and immediately recognized the same pattern from my field. Even in this comment thread is somebody making the “AI is a tool, get over it,” argument. I feel sadness for the mathematicians because I already know the misery that’s coming for their field.
Software engineering has always felt somewhat broken and dysfunctional to me even 20 years ago when I started. And LLM economics have turned into something downright hostile to its practitioners. I don’t have a similar intuition about the field of mathematics. Possibly they are in a better state to have a healthy disagreement about the use of this technology.
The question for them is similarly existential - who is going to pay for research grants when the machines do all the work? Why pay for a PhD student to work on some research topic if an llm already proved the theorem?
short term profit over long term thinking - because no one in the system is incentivized towards long term thinking. The counter balance should in theory sit within the remit of politics, but even senators can be bought and few are far sighted enough to realize the consequences of incremental deindustrialization.
Long term thinking is pretty bad too tbh when you dig in at what the different schools of thought imagine about the drivers of politics.
The electoral constituencies for the antinomianism, authoritarianism and victim-complex revanchism in the West that has prevented friendship between the world powers are all comprised of the same alliance between two groups repeated hundreds of times: moderately impoverished populist masses facing more of the same and industry oligarchs looking to return the west to feudal plantation economies as reprisal for any cultural slights they feel the public has given them.
The latter group always exists to some extent but the former group only exists because disposable energy (joules, watts, BTU, etc) per capita has been steadily falling since 1973. The golden age of shale exploitation bucked this trend but that has ended too. You can fiddle about with using different financial or interest rates proxies for this if you want but going into joules per person after extraction is the heart of the matter.
If this decline did not exist, we would not be in the world we're in by a long shot. But if you look at the predictions they're all off. The media, academia, popular culture and business culture expected continuous growth to put fusion power plants on Mars. The UN, IEA, DoE, Long Now people expected moderate growth to sustainable replacement of gas turbines with optimistic greentech buildout figures that use more lithium and copper than can ever be mined. The economic counterculture like NakedCapitalism and The Oil Drum expected Mad Max.
Nobody predicted the existence of the voter base that brought us the last run of presidents, they predicted different voter bases. Some of the predictors even imagined that the importance of Pennsylvania Avenue would slowly vanish without anybody minding.
It’s really bizarre to me that anyone would think otherwise. Though - I was aware of the need to spend on sufficient local llm hardware to buy what I could afford for that purpose in late 2024 - so maybe I’m an outlier.
It’s also very useful signal that the reasoning trace is leading to solving open problems - you can be certain that you’re not landing somewhere inside the training data.
I think the entire thing here is that these solutions _are_ inherently interpolations of existing work in the field. That's the "super power" that LLMs have. To interpolate mass amounts of multi-dimensional data.
This depends on some handwavy use of the term "interpolation", not the mathematical definition. Mathematically interpolation usually means that the query point is in the convex hull of the data points, and that almost never happens in high-dimensional spaces.
See:
Learning in High Dimension Always Amounts to Extrapolation
Randall Balestriero, Jerome Pesenti, Yann LeCun
https://arxiv.org/abs/2110.09485
I guess you mean by "interpolation" that it's some kind of nonlinear combination of the training data, but that's an almost vacuous statement. Any input-output relationship has to be so by definition.
Or perhaps you mean that interpolation is when the test input comes from the same distribution as the training input (though this is not technically the meaning of "interpolation"). But this is also quite difficult to pin down.
