Because safety and welfare have literally nothing to do with LLMs. They generate text. If someone is stupid enough to hook the text generator up to nuclear missile launchers and try to "align" it against nuclear annihilation with a "pretty please don't do that" prompt, I'm not going to blame the AI for the impending nuclear apocalypse, I'm going to blame the idiot who handed the big red button to the digital equivalent of a toddler.
Well, giving it access to a simple linux terminal is theoretically enough to cause more damage than most people are comfortable with, and doing so is trivial enough that it will be done (and has been, tens of thousands of times).
LLMs don't produce text at all, they produce probabilities of tokens. Tokens aren't text, they're high dimenensional coordinates in a latent "concept space".
These are displayed to us as text, but this distinction is important when you think about what they're actually doing, which is closer to building and transforming concept geometries.
Humans are biological machines that generate further humans.
Lawyers and diplomats and politicians and bureaucrats are humans, that only generate text.
We are seeing LLMs have cognitive abilities that significantly exceed human abilities. At the same time, they are clearly not the same type of mind that humans are. They are something new.
I think the widespread "they are just text generators" and "they are just tools" are comforting lies rather than an honest look at what we are seeing right now. Intellectually lazy.
And by the way, there has been a long-standing consensus among ethicists, philosophers, and sociologists that technology is not value-neutral [1]. Of course Silicon Valley has a long-standing tradition of denying this.
> Lawyers and diplomats and politicians and bureaucrats are humans, that only generate text.
This is a bad take.
> We are seeing LLMs have cognitive abilities that significantly exceed human abilities. At the same time, they are clearly not the same type of mind that humans are. They are something new.
They're software, not minds. What's intellectually lazy is pretending they're anything else.
