I get the sense that there is a school of thought I’d call “biological exceptionalism”, a belief that somehow constructs made up of many functionally simple units, is fundamentally different and special compared to a similar construct composed of silicon units. I can’t see how someone could hold this belief and be a materialist (I’m not implying you are btw). Biological exceptionalism surely implies the existence of a “soul” or “spirit” (which raises a whole bunch more questions).
I don't think it'd have to imply a soul - one of these "functionally simple units" is much less well-understood than the other, and you can insert whatever belief you want to fill the gaps.
I think it goes the opposite way, starting with a soul and arriving at biological exceptionalism: You start by saying "I have a soul / spirit". Then you extrapolate "things like me probably have a soul / spirit". But since there is no theory of souls, the extrapolation is not based on principles but rather on vibes and guesswork, ending up somewhere like "there must be something weird going on with neurons that somehow create a soul, connect with an existing one or something like that.. maybe.."
Which is an interesting argument because: do neurons somehow create a soul (as you mentioned), or are do neurons somehow attract souls, or do souls choose what to inhabit. In the first case it posits that a “soul” is purely a function of biology. In the others that there’s some sort of inextricable link between the two (which in the absence of any theory of soul or how these are linked to biology, seems rather far fetched, unless of course souls could choose to inhabit anything with sufficient degrees of freedom: a brain, a mega computer, a planet, a galaxy)
Maybe a materialist biological exceptionalism position could be that biological substrates are capable of sending kinds of signals that silicon substrates are not, in the sense that brain matter especially might give rise to far greater complex processing in ways that we don't know about or appreciate yet, and which we may discover can't be efficiently recreated in silicon?
I don't know if I buy that argument or even if it makes sense. But yeah this all smacks inextricably of soul talk.
I agree that as per current understanding, the functional biological base units have more degrees of freedom and a greater functional diversity as a result, but that’s merely a quantitative difference not a fundamental distinction. One could imagine equally more complex silicon base units beyond what we have now. Whether they need to follow the same pattern/modes as the biological units remains to be seen (but it’s pretty hard to argue against evolution as an efficiency maximiser).
I think the questions would be very much different. LLMs are not made of human brain cells.