> a basic understanding of how LLMs operate at a technical level
An LLM with CoT is Turing-complete. Training is, basically, compression (the training data gets lossily compressed into the model's weights). The information-theoretic limit of compression is an algorithm that reproduces functionality of a system that produced the training data.
No "magic" is required to get to a system that reproduces at least some facets of the human brain functionality.
Three years ago I was skeptical that stochastic gradient descent (and other known techniques) are the way. But evidence kept piling up.
An LLM with CoT is Turing-complete. Training is, basically, compression (the training data gets lossily compressed into the model's weights). The information-theoretic limit of compression is an algorithm that reproduces functionality of a system that produced the training data.
No "magic" is required to get to a system that reproduces at least some facets of the human brain functionality.
Three years ago I was skeptical that stochastic gradient descent (and other known techniques) are the way. But evidence kept piling up.