Well if the question is why are there "separate key query value matrixes" then I would argue the person must have a fairly decent foundation.
Even if you have less background the jargon doesn't unpack as insanely as you might imagine. It's not quite the monad is a monoid level stuff.
A projection is just the linear algebra word for a function that takes vectors and produces vectors. And function in this case is just like a code function.
After that you need to believe that vectors (an array of numbers) can encode meaning abstractly (this is what "latent space" means). I mean this is sort of a given since what is a picture other than a list of numbers and obviously pictures can have meaning.
Finally you arrive at the statement that Wq is a projection (a function) from a token to the questions such a token could ask.
For example a token of "dog" could ask "what is the dogs name" as a query vector. Abstractly this is what Wq might do.
Maybe an earlier token "bella" turns into "this is a name" as a key. And "the name is bella" as a value.
The key and query in this case might line up, i.e directionally point the same way.
The real leap of faith is recognizing that this sort of abstract model of language can exist and be represented as just a vector of numbers. Which admittedly is a bit bizzare.
Even if you have less background the jargon doesn't unpack as insanely as you might imagine. It's not quite the monad is a monoid level stuff.
A projection is just the linear algebra word for a function that takes vectors and produces vectors. And function in this case is just like a code function.
After that you need to believe that vectors (an array of numbers) can encode meaning abstractly (this is what "latent space" means). I mean this is sort of a given since what is a picture other than a list of numbers and obviously pictures can have meaning.
Finally you arrive at the statement that Wq is a projection (a function) from a token to the questions such a token could ask.
For example a token of "dog" could ask "what is the dogs name" as a query vector. Abstractly this is what Wq might do.
Maybe an earlier token "bella" turns into "this is a name" as a key. And "the name is bella" as a value.
The key and query in this case might line up, i.e directionally point the same way.
The real leap of faith is recognizing that this sort of abstract model of language can exist and be represented as just a vector of numbers. Which admittedly is a bit bizzare.