LLMs don't produce text at all, they produce probabilities of tokens. Tokens aren't text, they're high dimenensional coordinates in a latent "concept space".
These are displayed to us as text, but this distinction is important when you think about what they're actually doing, which is closer to building and transforming concept geometries.
No language models are programmed, they are "grown" or evolved from data.
There's no print statements or human entered logic involved in the raw model expression at all.
The only thing that humans have programmed is efficient parallel dot product pipelines that "animate" (for lack of a better word) the models.
Everything these models do is emergent from their backpropgation guided evolution. This even includes in context learning itself, which was not an expected outcome.
They aren't grown/evolved from data, they are fit to the data. The fitting process can be fully deterministic although its fairly easy to screw things up such that it isn't deterministic, but that just a defect not some fundamental shift.
You have completely misunderstood what I was saying so badly I can't even formulate a response other than to suggest you read my reply again. I was not suggesting that LLMs are programmed with print statements, for fuck's sake.
This perspective that consciousness cannot be programmed can only make sense if you're a dualist. We don't know how consciousness arises. If you're a naturalist it can't be ruled out based on the simplicity of the algorithm.
> When you write a program to predict tokens based on context, seeding its context with something that makes it predict "self-reflecting" text is trivial. Program does what it is programmed to do. Would observing the output of the following program inspire doubt as to its sentience?
Then you follow it up with print statements as if that is a good analogy.
As I said, they are not programmed, so your question above is not relevant to your argument.
You say they're programs that are stochastically jiggled, but that's simply not accurate either. All LLM abilities are emergent, even when the training corpus is well defined.
I didn't think you literally thought they were made of print statements, but you are implying they're software that's been "fuzzed". Hopefully you don't literally that either and you're just using it as a bad analogy.
You could have argued from the stance of neural networks being universal functions, which might at least be closer to the truth, but instead your example is print statements!
I get you're trying to say that something trained to say a thing doesn't mean it has arrived at the thing like a mind would, and perhaps that would have been closer for GPT 2.
These days though, we just have so much more awareness of what they're actually doing internally that it's bizarre to even compare them to stochastic parrots of the training corpus, if that is closer to what you're implying.
First you run a program (training framework) to generate a database of values.
Then you run a program (inference engine) which performs calculations against the database of values.
To put it in ELI5 terms: run a program against a book, counting how many times "I love " appears in the book. Note "dogs" 4 times, "cats" 5 times, "you" 1 time into a database. Then run a program against that database. When inputting "I love" as the preceding text, the second program determines the most likely result is "cats" and returns "I love cats" (or returns "I love cats" 50% of the time, or dogs 40% of the time, or you 10% of the time, or some variation by different methods of weighting).
Yes, this is an extreme simplification. Yes, the model is not technically a database either. But this is fundamentally the process followed. You would consider it a single program if the training framework and inference engine were part of the same software and stored the computed training values to memory instead of disk, taking an input dataset and an input context as params and returning "I love cats" as the output. There's all kinds of incredibly sophisticated techniques applied on top of this foundation to vastly improve the statistical modeling and efficiency, but the underlying basics have not fundamentally changed.
> Then you follow it up with print statements as if that is a good analogy.
The print statements were not an analogy. They were pointing out the ridiculousness of doubting whether software is conscious because it generated self-referential text. Gettting software to generate self-referential text is as easy as `print(self_referential_text)`. So the only question is how the self-referential text is generated. For self-referential text generation to be more interesting than passing it as a literal print value, there would have to be some really wondrous "how" going on. But, it turns out, the "how" of an inference engine isn't that much more interesting than literally doing a `print`.
What if we can interact with this video, and it reacts in the same ways the source organism does?
Then we put it in new situations that weren't in the source video, and it interacts in a similar way to the original organism in these situations, too.
What do we make of reactions of pain or joy? Where's the line between simulation and enaction?
This is closer to the reality of these models.
I'm not suggesting I know where that line is - if indeed it is a line at all - it could well be a gradient.
LLMs are deterministic, though. Much like the video.
AFAIK using the same input tokens, weights, and numerical operations will lead to the same probability distribution for the next token. It uses pseudo-randomness to enable temperature, etc. Like a fuzzy video.
"Markers that would indicate consciousness if observed in a biological organism" just does not mean very much. A PR phrase used to hype the IPO.
No, that is not at all something we can just state as a fact. Whether the brain is deterministic is an open question that just inherits the good old, probably unsolvable determinism debate.
The LLM pseudo-randomness from above is engineered by us humans and fully understood, much like an algorithm playing a video frame sequence.
You could theoretically record a full register of all states of an LLM setup with all the possible inputs and environment parameters, and it would fully describe everything you would ever get from a given LLM setup. It would be a very large, convoluted book.
I understand that Anthropics PR department wants to see truth or reason behind every "I'm alive" the LLM generates. Even the term "self-report" is anthropomorphizing, as an LLM does not do anything on its own at all. (It also does not hack any company on its own.) That is just one of the narratives they spin probably at least until the IPO.
Nonsense. There was one proposal for relevant quantum effects in brain dynamics, and that turned out to be not relevant. Even if they were, you could substitute all quantum randomness with pseudo randomness and obtain an absolutely indistinguishable object.
But even if this were a debate, its absolutely absurd to claim that the question of determinism in the brain has any bearing on our moral standing. If we discover tomorrow that quantum collapse is deterministic and can be derived from an underlying theory, and thus all of physics is deterministic in the good old fashioned Newtonian sense, this would not affect our moral standing in the least.
You seem to have conceded the "is it deterministic" argument only to sidestep by declaring determinism irrelevant. Your original claim was that LLMs are deterministic "in the same sense" as brains.
We can write down an LLMs full register, and that register/book contains the whole output universe of the text generator. That book does not act, it is morally neutral. That the brain has such a register at all is just restating the determinism axiom, which you treat as fact.
A text is not conscious, and we can not wish it into consciousness, no matter how many human-like patterns we find in the book / the generated text. It has not been shown that running the text adds anything over the text written out. Researchers are super motivated to find machine consciousness but cannot find it, while a company months from its IPO keeps pitching shadows of consciousness all day. It really is a PR strategy.
It does have bearing if your definition of consciousness rests on free will and you think that's incompatible with determinism. Now I don't think a lot of people seriously believe that [1] but it's not some logical nonsense.
[1] Off topic: I think most people are really compatibilist but a lot of them (like me) also believe in non determinism. Not believing in free will is really rare.
I have never come across this position argued seriously. I would also consider it absurd, as then the question whether we have consciousness depends on unknown properties of fundamental physics (which is not incompatible with determinism, see e.g. Bohms theory). It would therefore be unknown whether humans are conscious. That is at the very least a notion of consciousness that is utterly distinct from any established meaning of the word.
