Yeah sorry, I read too far into your position. There's a certain faction within these AI discussions that wants to over-anthropomorphize the LLMs in kind of a borderline spiritual way.
Oh that's a shame - I hadn't seen that, but I can believe it.
The philosophers who study these things have been clear for a long time - we can never know what it feels like to be in a digital brain. Or any brain for that matter. When push comes to shove we all might be phantoms in some guy's dream.
Don't know + can't know. I think that was the real point of the Turing test. Not: this means it's conscious. Just: this is the best we can ever hope to do.
Sheesh. Yes, I agree with you entirely. I'm merely pointing out that ignoring this behavior is dumb, too.
And probably not rationally based. Leads people to make crazy jumps. :)