Is that meaningfully different from the study methods of the past? That doesn't require an LLM, and using one risks correctness in exchange for speed. You might not even get that speed if you're stuck in the weeds often enough.
A decade ago, a google search for study guides written by another professor would have been slightly slower. A decade before that, you'd be even slower fumbling through several books. Every single word could at least be trusted. You don't get that from an LLM.
> Is that meaningfully different from the study methods of the past?
The fundamental service a teacher provides is personalized feedback, quickly identifying where you are stuck and focusing the explanations and exercises on that area, drastically increasing the speed and quality of learning versus the self-supervised route.
The lack of this closed loop effectively killed the high hopes that were placed in e-learning and MOOCs 15-20 years ago, TV learning in the 1960s and many other failed revolutions, seems every generation has its own version.
It appears to me LLMs have a real potential to close this loop and become the failed educational revolution of our own generation.
> personalized feedback, quickly identifying where you are stuck and focusing the explanations and exercises on that area,
This has been a huge blocker when I tried to study advanced math myself. Many of the exercise books don't have worked out answers, so often you're either stuck or you have to hunt a variety of sources online for solutions and advice. It kills flow.
If you have an actual mathematics professor to ask questions to then sure do that, but most people don't have that luxury. Also the error rate of frontier LLMs on textbook highschool/college level mathematics is going to be extremely low.
are you referring to this section?
> Verification is different from generation: Models scoring 98 can solve problems but can't always explain why their approach works at the level a human mathematician would.
It doesn't say it can't explain why with any accuracy, it just says it can't *always* explain at the level of a mathematician, but most of us don't have such a mathematician at our beck and call to answer our questions anyways (thinking of the perspective of a self-learner outside of formal education)
It’s not just math. Anecdotally, LLMs struggle the same way with software engineering where the code they write is correct (compiles and passes tests), but reasoning is wrong often enough to eliminate most trust in these models’ ability to explain codebases or even features they themselves produce. It’s not about the _level_ of the supposed intelligence where a model struggles to summarize things succinctly or simply enough (responding to the “pocket mathematician” comment) or can’t grasp certain concepts at all (if so, how tf is it able to apply them?). It’s that by their design LLMs have no concept of truth and no concept of causality. They guess with every single inference and it’s very hard as a user to understand which guesses are more or less certain, since, you know, confidence ratings aren’t part of these models’ design either.
I wonder how long before LLMs need to cash in and even simple prompts are beyond the non-existent budget of a student. Which means it will need to be subsidized by education... and we're back in the teacher loop once again.
Or perhaps libraries. That would be neat.
>It is also significantly more engaging and fun.
That won't last long either. I remember when phone apps were the "engaging and fun ways to learn". Half life of 2 years, and we're already seeing people lose the fun factor.
When studying a mathematics text, it is good practice to question what you are reading, trying to prove things to yourself etc., which IMO makes the problem of trusted sources much less than it is in things which you are not able to verify e.g. historical accounts.
I certainly would only trust my textbook as the authoritative source, but i can see that in the absence of an expert teacher it's nice to have something that can critique a proof. Imo the fact that it's hard to verify that your own proof is correct is one of the main barriers in self-studying math, especially if one is at a level where one is not completely fluent in applying the various techniques. This also applies to judging answers to open-ended questions in any other field.
I'm a private math tutor specializing in exactly this sort of material, and I agree with this very strongly. Knowing what "counts" as a proof is one of the most common gaps I see in students who come to me after self-studying, and most students do need some back-and-forth with an expert to really get that skill down. I imagine that LLM's could be very helpful for this if they were used judiciously!
A decade ago, a google search for study guides written by another professor would have been slightly slower. A decade before that, you'd be even slower fumbling through several books. Every single word could at least be trusted. You don't get that from an LLM.