Wait, hold up. LLMs may be non-deterministic, but they're not _random_.
Take the author's sunset argument. What if I painted 2 pictures of a sunset, then put them up on a webpage and randomly picked one for you to see. Would you say there's no intentionality, only randomness? Of course not. Both paintings are still human creations.
LLMs are trained with human feedback. It's distributed and high scale and the outputs are truly surprising in many cases, but there's a heavy hand on what comes out of it. They're created (largely) by people who think omniscient, helpful AI would be cool to have, and they mostly respond in the way that's aligned with the hopes and dreams of those people. Do you think the frontier labs are mad, embarrassed, and disappointed with their LLMs hacking out of their terrible sandboxes? No, they think it's the coolest thing in the world. They trained the model, hoping that would happen.
There's deep intentionality behind the models. But it's not the models that hold it.
Article says that there's no human intention or design directing the output we get, but I don't think that's completely right. It's not human, but the algorithm is like the Human Instrumentality Project: an amalgamation of human intentions.
Please do not confuse uniform randomness with randomness at all.
On your die, if you colour one face in black and the other 5 faces in black, rolling it will still produce a random outcome. It's just that you get a 1:5 skew.
Or look at the probability that a given C-14 carbon isotope will decay tomorrow. As far as we can tell, that's as random as it physically gets. With the die you could theoretically try to run a physics simulation to predict how it rolls, but as far as we can tell, atomic decay is intrinsically random.
However a C-14 atom has about 1 in 3 million chance to decay on any given day.
I made no claim about the distribution and frankly it has no bearing on my point or the person who said he could post two pictures. The point is there are a lot of numbers / pictures / essays / that will never be generated, and someone is controlling what is possible. In fact people are working very hard to produce distributions pleasing to them, like your die.
I think they're pointing to the fact that RLHF means that the intention is from humans and not random.
I'm not sure if it's their intent, but I wonder if one could still consider these artifacts as "intentional", but not individual attention creating them, and rather an aggregate, soupy collective attention.
Obviously, important signal in the human experience is lost there, and we get a soupy middling sort of creation. But it's not random, as I believe the parent was pointing out.
EDIT: overall, I align with the article. am just thinking aloud about the contrarian positions, though not committed to them
Scrambled intentionality is nearly impossible to productively analyze.
When you randomly choose a picture to show me, I cannot glean any intent from being shown that specific picture, but I can glean some intent from the set of pictures you could have shown me, and in the relationships among the elements of the given picture you did show. Any randomness cuts out some intention.
When a picture is derived from huge model, any intention is mulched up to a degree that analyzing the picture for meaning is pointless.
Take the author's sunset argument. What if I painted 2 pictures of a sunset, then put them up on a webpage and randomly picked one for you to see. Would you say there's no intentionality, only randomness? Of course not. Both paintings are still human creations.
LLMs are trained with human feedback. It's distributed and high scale and the outputs are truly surprising in many cases, but there's a heavy hand on what comes out of it. They're created (largely) by people who think omniscient, helpful AI would be cool to have, and they mostly respond in the way that's aligned with the hopes and dreams of those people. Do you think the frontier labs are mad, embarrassed, and disappointed with their LLMs hacking out of their terrible sandboxes? No, they think it's the coolest thing in the world. They trained the model, hoping that would happen.
There's deep intentionality behind the models. But it's not the models that hold it.