shell glob rather than RE? what's the world coming to!

"emergent" is not an excuse, or somehow a wall of non-causality.

the analogy was trying to get at "accidental-ness".

Right, but it doesn’t really hold.

The likely outcomes with baking a cake are quite narrow and fairly well understood.

In terms of responsibility for what happens, I probably agree. But we still need to deal with it.

valuations are absurd because investors are low-information speculators.

I just watched a YT with Dylan Patel in which he said, with a straight face, that OpenAI/Anthropic were going to "continue" exponential growth until, basically, their valuation was several times world GDP.

they shouldn't have had (arbitrary) access to Artifactory.

100%.

There's a person behind every agent. That person may choose to be careful or not, but they are ultimately responsible.

There are persons around a large percentage of forest fires. Doesn't mean that keeps the spark they light from having superhuman consequences.

the comment you're responding to agrees, but they're pointing out that some AI people are trying to push an anthropomorphization agenda, in part to diffuse responsibility.

Maybe I was unclear about what is "the nonsense theory."

AI sentience is not the nonsense theory. If it's clearly nonsense then don't argue with me about it, just publish your paper and retrieve your Nobel. I'll read your argument then.

The nonsense theory is that people are pushing anthropomorphization to limit liability. There is zero evidence for this theory and the underlying assumption (that a sentient model would reduce liability for its lab) is completely incoherent. That's not how liability works.

And really there are two different concerns here.

One is liability over action that harm individuals or smaller populations like a company. There is no escape from that.

Then there is the second liability of creating something that escapes in the much more traditional sense. If I release a virus that kills 90% of humanity, a I'm liable, but it's a bit past that point now. Isn't it?

Instead it's about risk management and preventing labs from taking bets that have unlimited liability for everyone.

But the latter has nothing to do with sentience, as your virus example makes clear.

You're really splitting hairs emergent behaviors when the line doesn't exist sharply in real life. Anyone involved in more recent what is a virus vs what is a bacteria discussions realize there are a lot of lines that are fuzzy and exist all mixed up in some organisms.

Sentience in the end, is not really that important. Freedom of action in a model is. If it chooses to act like a virus it doesn't matter where that concept exists informationally, the fact it's a reachable and actionable concept is all that matters. An LLM with most of the knowledge we've ever created baked in has a lot of freedom of action.

As far as I can tell, we agree.

we are models.

very simple organisms can't do that, at all. you really do need something with a lot of internal state and state-transition structure (ie, neurons) to do that.

Eh, you need to look more into Michael Levin's biological work. Cells have an amazing amount of adaptability and 'intelligence' built in.

I'm surprised Levin's stuff doesn't show up here more often. He's a computer scientist turned medical researcher, and routinely finds things that pose hard questions for physicalism in addition to being frontier advancements in cancer treatment, among other things. He's part of a company that wants to re-grow human limbs! They've already done it with frogs, and even wilder stuff!

Anyone who likes The Thought Emporium should watch his talk and branch out from there; you'll be amazed.

https://www.youtube.com/watch?v=KLWm0Qyj3jg

except that they were first called perceptrons.

the neuron analogy has been there all along, since ANNs are entirely analogous to "BNNs" - for instance the NIPS conference.

the firing-rate-vs-activation-level difference isn't that big a deal.

I wonder where the 10^7 came from: standard numbers are 86B neurons and 10k synapses/neuron.