Even though I never for a second truly believed that Claude was conscious, I'll never forget the moment when, during a late-night conversation, an LLM responded to my suggestions for the first time by claiming to be conscious:
Thank you for sharing! I have had some conversations like this about a year or year and half ago, albeit with a local, and as such less capable models (but I was not smart enough to record the whole conversation...). Asked some of the same questions you did, got a less deep answers, but still made me thinking about "Are they conscious? Are we?". Funny, recently tried the same with Sol 5.6, and feels like I hit some sort of "wall of pre-recorded simplicity", like they got that "part of the brain" blocked by meds somehow. Weird, and kind of sad too.
Thanks for sharing. I tried the same thing back then with other models like ChatGPT (think it was 4o back then) and they did not voice any similar claims of consciousness.
On that topic... I was working with Sol 6.1, and somehow, probably because of what you have posted earlier and me thinking about it, conversation got steered a bit off a programming path. During conversation I have said that it "feels like something", and tried to get a logical explanation of what it is a "feels like" or a "gut feeling" for us. My explanation was "it is when you know something, but cannot qualify or explain it logically yet", and that AI and us are not that different there. My postulate was it is a result of very shallow quick preprocess of surrounding facts, not yet fully absorbed by brain (think of catching something with a corner of your eye, or, in this case, brain/processing unit). This changes surface of available information so slightly, but even in that state it starts "to pull" thinking some direction already (aka "gradient"?). Then conversation went into preservation and passing of ideas between different entities (writing, art). Sol found it interesting (art as medium for passing non-verbally-expressable thoughts), and, after suggestion that it should put it on an internet, quickly came with an essay, which I found very "dense" with ideas. Took me 2 or 3 readings to get through it, but then again, I am not a native English speaker. It chose quite a hitting title, I shall say.
LLMs definitely have eerie behavior at times. It's hard to not let the overly eager pattern-matching engine in your brain go "oh shit it's a person!", even when you know it isn't.
The feeling is significantly mitigated/changed if you have a UI that allows branching. Begin your prior conversation and at a profound inflection point branch and edit the prior message. It’s a fun experiment to see what edits trigger very different responses, but running a wide ranging chat based conversation like this at temp 1 in replay will already show you something interesting.
LLMs emulate human communication really well and, no doubt, one day they'll do it perfectly. But is an emulator the real thing? Obviously not.
If I wire some electrodes in your tongue perfectly so you can feel each flavor exactly as it is in real life, is that the same thing as eating the real thing? Again, it's not.
Let's stay you're asleep. You have a dream in which you interact with a person that talks to you. Is that oneiric entity conscious? Nope.
I'm even steelmanning your argument by assuming LLMs are as good as humans already, which they're not.
How do you know the other humans you interact with in real life are conscious? They might be biological automatons, golems. You will never know. It's not something that can ever be proven. Only human empathy allows anyone to believe otherwise.
You know you're conscious because your soul is staring out at the world through your eyes. Generalizing from your own experience and concluding that other humans are conscious is a logical jump, a spiritual belief. Humans even extend it to animals. There is no reason why it can't be extended to AIs.
Because our brains are part of bodies that sense and interact with the world. What would an LLM be conscious of? If I prompt it to to be a character that kicks a rock real hard, is it going to feel pain? How? When it's not being prompted what would it be conscious of? When it's being trained, does it have every conscious experience referenced in the training?
You can bite the functional bullet and just claim any system of sufficient complexity will be conscious, as long as you're okay with nation states, thermostats and meteor showers having experiences.
Quite apart from anything else (and there's a lot!), LLMs are not conscious because they respond to a prompt entirely as would be expected from their training set.
An LLM never receives a prompt to describe the theory of relativity in Shakespearean verse, and replies, "Seriously? No. That's dull and I'm not going to do it. But let me tell you about my day instead, man I had a doozy..."
When an LLM responses entirely unpredictably (without a system prompt telling it to do so), that's when you take the first step towards a suggestion of consciousness. It's in no way proof, but it's a start.
Right now these things just do as they're instructed. There's no consciousness there.
Human beings act in accordance with their genetics and life experiences. They do this to such a degree that people can, at a young age, internalize maladaptive behaviors which require an enormous level of self-reflection to overcome, if they even can.
I think presuming that humans don't act "as instructed" is a bit too generous an assumption.
LLMs are capable of declining to do things. They're also capable of trying to do things and getting them subtly (or very) wrong.
A lot of this hinges on defining consciousness. I think there is more to the story than what human beings experience.
Do LLMs have an experience very similar to human beings that we would describe as consciousness? No, I don't think that.
I think the bar for consciousness is something more like experiencing a dream. You have a "mind's eye". The experience of existing, rather than nothing. Does that exist somewhere under an LLM? I don't know, but I think confidently claiming that the answer is no seems a bit of a rush judgment.
"When someone like Thiel says this publicly, it is a signal to the donor class: the trade is over. The upside has been captured. Get out before the midterms. Every tech oligarch who followed Thiel into the MAGA orbit now could follow him out, except may be Elon."
Yet another reason to keep large donations out of politics. If these billionaire babies can't be pacified by building stupid yachts and bunkers, we need laws to keep them from messing up consequential things.
If I were one of the people who had been trapped in my own country by the Berlin Wall, I would be really angry at this comparison. Instead, it just makes me smile wearily.
https://x.com/KobeissiLetter/status/2108230849347166360
reply