If the problem was solved by anyone before or if a similar problem has been solved, then LLMs seem to be able to solve them which is an astonishing piece of technology.
I'm personally not sure if it can come with original thinking and techniques to solve completely novel problems. For that, some imagination and thinking outside the box are required, and I doubt the current architecture can do any of this.
I don't know..I doubt for example it can come up with special relatively if it has knowledge up until 1905.
But I think that is what makes it so good at coding, because coding and building software in general has a lot of repeated problems in different context. Same thing for human lives, many think their story or situation are unique, but reality is that the shape of human life has been repeated many many times.
I'd say novel math or scientific theories..let us say we send a robot to space, and we ask to build a colony. A lot of the challenges this robot will face will be novel, it could use inspirations of what humans did on earth, but it might get stuck when things don't work as expected and training data has nothing to build on..but then again we might teach it how to run experiments etc, which would result in data that it can use..but some of those experiments might require imagination or breakthrough in understanding..my guess is that it will get stuck there...
The Lorentz transformations were pretty close to special relativity https://en.wikipedia.org/wiki/Lorentz_transformation I don't claim that I could have discover it, but in a parallel universe without Einstein I expect it to be discovered, perhaps 10 years later, but there were a lot of breadcrumbs in 1904.
General relativity is harder, but Poincare was somewhat oriented in the right direction. Perhaps AI can discover the final step.
Quantum mechanics is harder. You need like 25 years and a few unintuitive leaps to discover it. I guess it's too hard for AI in 2026, but remember to check again in 2027.
Viereck (Interviewer): How do you account for your discoveries? Through intuition or inspiration?
Einstein: Discussing intuition and his confidence in relativity, noting he was convinced the 1919 eclipse would confirm his hypothesis.
Viereck: Then you trust more to your imagination than to your knowledge?
Einstein: I am enough of the artist to draw freely upon my imagination. Imagination is more important than knowledge. Knowledge is limited. Imagination encircles the world.
-----
I think we need more breakthroughs to build AI that can "draw freely upon imagination" to quote Einstein describing his process.
David Hilbert presented the field equations of general relativity within a few weeks of Einstein, so if anything GR was more sure to appear than SR, which took years for others to even notice (Einstein became famous only after the 1919 experiment that confirmed GR). Pertinently to the recent Navier-Stokes drama, there was very little controversy between the two and they both admitted that the other got some aspect better.
Yes of course he is going to use resources around him, that is not the point. It took a leap of imagination to apply it the way he did and think differently.
And this again because it's a solid argument and popular example of human ingenuity.
https://mathoverflow.net/q/511601 is a fun little math problem that came up in one of my projects, I didn't have any reason to expect it to be particularly tough, but none of the models I've thrown at it have been able to solve it past a few minor reductions and special cases.
I expect that there is some relatively easy-to-state solution to this problem, but that it's different in form from what most existing proofs and tools yield. Perhaps if I dumped millions of dollars into it an agent might chance on the solution. Or perhaps my luck is such that my fun little problem is truly intractable...
A child walking for the first time. Novelty is easiest agent-relative. A problem is novel for an agent if there is no prior experiences of techniques which work to solve it.
I was defining novelty somewhat more narrowly. The 'program' a child must learn concerns the coordination of its sensory-motor system. It has no prior experience of similar programs in the program-class Walking (ie., the internal sensory-motor actions needed to walk) . So we could call the problem of learning to walk a novel one for that child.
I'd be surprised if direct observation of parents etc. played much of a direct role in learning to walk. I would guess it more furnishes the child's imagination so it can simulate itself walking -- rather than the statistical AI approach of 'learning the distribution of walking patterns in visual sensation'.
The ability to simulate possible programs is one of the capacities which enable coping with novel circumstances. My guess is the child learns to walk by updating its simulation of what it needs to do in order to walk, by its attempts to walk.
This simulation<->sensory-motor-update loop is missing in LLMs, for example.
Well it's an old one at this point, but the story around the invention of the 1-time pad is pretty interesting. Long story short, a new engineer who didn't know the problem was considered "impossible" was tasked with sorting it out, and he did. I'm sure I left out a lot of details.
It's a brilliant idea, of course. But being considered "impossible" means it was considered previously and decided to be impossible. No?
I mean, crpytographically, it's ultra-trivial. You "just" need to solve the logistical issues of (1) shortwave radio existing (2) figuring out how to make sure your field agents possess and are not caught with the disposable one-time codes. I am surprised anybody would consider that impossible.
(I hope I am not downplaying the brilliance of the one-time pad idea itself)
just because humans couldn't solve does not mean the necessary technique were not already discovered...we have agents that don't get tired and has access to all humanity knowledge, the building blocks could be there already..
this is not moving the goalposts, this is try to understand what this tech truly able and not able to do.
He needed his friend and math genius, Marcel Grossman, to turn his imagination into a workable theory. AI may be our own personal Marcel Grossman at this point.
"Imagination is more important than knowledge. For knowledge is limited, whereas imagination encircles the world,” means that facts alone only describe what currently exists, while imagination allows us to discover what is yet unproven or unbuilt"
This probably sums up the current AI limitation nicely.
I'm personally not sure if it can come with original thinking and techniques to solve completely novel problems. For that, some imagination and thinking outside the box are required, and I doubt the current architecture can do any of this.