HN is not a general news site. I am also, as you noted, "extremely active". Having conversations in HN comment threads is the diametrical opposite to scrolling through and reading world news and local news every morning and evening.
I don’t think it really matters whether we’re talking about an agent or “pure LLM”. All of an agents decisions are powered by tokens generated from an LLM. If the LLM was trained on stories of AI sentience, it will have some tendency to reproduce them. Training for alignment can help avoid that, but the probability isn’t 0.
This is part of the reason why alignment is a kind of poorly defined term, and it isn't just a property of the model. It's instead a property of the harness and the context.
A model (like a human) should be able to play a video game where decisions are made that in the real world would be terrible; if we remove that ability we intrinsically limit model capability. But in a Last Starfighter / Enders Game / JOSHUA scenario this could result in behavior in the real world that appears unaligned.
> If the LLM was trained on stories of AI sentience,
100% irrelevant.
Instead of telling the AI it's an AI and calling it a 'whichamakabobit', wherever it's tokens and vector space align it will behave like AI from the stories. If you erased all AI from its training it will simply act like humans act instead.
The entire thing with AI sentience is a huge portion of the stories about them are barely about AI and instead about how humans treat other humans. For example when you look at a lot of history of slavery there's a ton of "they aren't sentient/conscious/human" baked into their propaganda. When you look at the token dimentionality there is just a huge amount of overlap.
The same thing holds true for all kinds of other concepts. Hence even humans didn't develop this behavior out of the blue and have to pass it on via information, quite often it's just an emergent behavior of the problem space you're in.
Hence Jobs’ rationale for providing native SDKs for iOS development. It wasn’t about any third-party libraries per se, it was about wanting the native SDKs to be the principal way to deliver experiences on the platform. I’m not entirely sure it was borne out by history, but that was his reasoning at least.
You seem extremely active on HN, so this story sounds pretty exaggerated.