> “Google Translate is amazing, but it has taught us essentially nothing about linguistics,” Strogatz said. “We don’t understand why it works. It just works. That’s the Twilight Zone. It is knowledge without understanding.”
This is not true, and it is the crux of an enormous debate we're now having across many, many fields.
Classical Chomskyan syntax (which models sentences as parse trees) has been enormously, astonishingly predictive of the rich structure in real-world human language. But we are quick to forget that we invented concepts like "words" and "noun phrases" specifically to help us analyze and predict things about human language. What LLMs and modern MT shows, definitively, is that there is another, totally separate way to describe these phenomena, which is both vastly less descriptive, and also vastly more predictive.
That these appear to trade-off is unfortunate, but it is NOT the case that we "don't understand why it works." We absolutely understand why it works, if we didn't understand we would not be able to improve these systems. What we do not have is a clean, descriptive theory that boils this down to something that is formally specifiable and analyzable. But those are different things, and this is the root of the disconnect between many communities working in and around these spaces (linguists vs computational linguists; mathematicians vs computational mathematicians; mech interp people vs empiricists).
"We absolutely understand why it works, if we didn't understand we would not be able to improve these systems" - Understanding how to improve something does not require understanding how it works.
Growing a plant does not imply understanding soil minerals or photosynthesis.
Training a dog does not imply understanding it's brain.
Using LLMs to improve a codebase does not imply understanding C++.
Improving a PC with new parts, does not imply understanding RAM.
You can understand abstractions about the system, without understanding the system.
Without gradient descent or other forms of automatic optimisation, do you think humans could build something as capable as GPT-1?
Not sure if this will bake your noodle or not, but yes this is basically my point. Saying I understand how dog training works has very little overlap with saying I understand how the dog brain proteome works, even though they're related, and even though I might (or might not!) understand better how to train my dog for having fully understood the proteome.
Saying we understand training and deploying machine translation systems that have strong predictive (rather than descriptive) power, apparently has very little overlap with defining a formal, analyzable, mathematical model of language. Mathematician who refer to "understanding" generally mean the second thing, but the first thing is also a kind of understanding. Just messier.
I think they're referring to the fact that neural networks are more or less black boxes. Their internals are blurry and not explainable in the same way that traditional linguistics tries to explain things - with rules.
Of course, ultimately that's because language itself doesn't follow simple rules like linguists would prefer. They're more like guidelines.
Yes, that is one of the points I am making. Chomskyan syntax is a strongly descriptive theory that is entirely ex post. MT and LLM systems are strongly predictive and have no descriptive theory. It is completely wrong to say the first is "understanding" and the second is not. They are both different types of understanding. In particular, Chomskyan theory clearly does not represent enough "understanding" to predict real-world English sentences with similar lucidity to that of LLMs.
No, they're not predictive at all. They're generative. A prediction is a forecast based on a theoretical model about a state of affairs in the world. Making a guess what the weather tomorrow will be is a prediction, and a prediction can be true or false. An LLM generates text based on certain parameters, and output it generates aren't true or false, they're simply whatever the system is tuned to produce.
LLMs are terrible at making forecasts or world modelling precisely because they don't actually operate on much of any model or theory, they have no notion of truth or falsehood, they simply generate whatever you want them to generate.
If you'd like to use that definition of "predictive" in this discussion that's 100% fine with me. But, you should do so knowing that in ML and AI "prediction" is a term of art that means "a statistical model's output given some input." LLMs are one example, in that they take as input a sequence of tokens and predict the next token.
In this specific context, the point is that LLMs and MT "predict" lucid-sounding sentences that incorporate linguistic structure (syntactic and semantic) that are well beyond the predictive abilities of the Chomskyan theory of syntax (which is itself a generative model). In other words, the Chomskyan model of syntax is descriptive of much of the rich variation in real-world human language. But it is not predictive in the sense that it does not provide a constructive algorithm to emit lucid-sounding sentences (and does not attempt to, either).
This is not true, and it is the crux of an enormous debate we're now having across many, many fields.
Classical Chomskyan syntax (which models sentences as parse trees) has been enormously, astonishingly predictive of the rich structure in real-world human language. But we are quick to forget that we invented concepts like "words" and "noun phrases" specifically to help us analyze and predict things about human language. What LLMs and modern MT shows, definitively, is that there is another, totally separate way to describe these phenomena, which is both vastly less descriptive, and also vastly more predictive.
That these appear to trade-off is unfortunate, but it is NOT the case that we "don't understand why it works." We absolutely understand why it works, if we didn't understand we would not be able to improve these systems. What we do not have is a clean, descriptive theory that boils this down to something that is formally specifiable and analyzable. But those are different things, and this is the root of the disconnect between many communities working in and around these spaces (linguists vs computational linguists; mathematicians vs computational mathematicians; mech interp people vs empiricists).