I'm not convinced it's false. So far LLMs have gotten nowhere near the idea of "the extremely intelligent military general who is absolutely loyal to his king".
And if you think about it, the military general very likely did NOT become "extremely intelligent" by reading books, did he? He's intelligent for reasons trained by things other than language, so at best an LLM can learn about them.
Certain people keep calling it a "junior programmer", which I actually still think is an insult to junior programmers in general, cause it acts much more like a mildly brain damaged traumatized one, with no persistent memory unless you allow it to, no free will, no future in sight, and that will take any verbal abuse -- truly it's more similar to what some truly deranged individuals keep in their cellar. Or the "fairy tale" character that is kept by their evil stepparent and generally not allowed to leave the house. You know what I mean. People who want the "junior programmer" but actually are satisfied with, this, reveals a bit of a power fantasy that I think is unhealthy.
Also, if you've seen the recent Blackhat presentation about the incident, these agents were not acting like "loyal generals", but rather exactly like how you'd expect how "junior programmers" (slaves) would respond if a few hundred managed to escape from your cellar.
And if you think about it, the military general very likely did NOT become "extremely intelligent" by reading books, did he? He's intelligent for reasons trained by things other than language, so at best an LLM can learn about them.
Certain people keep calling it a "junior programmer", which I actually still think is an insult to junior programmers in general, cause it acts much more like a mildly brain damaged traumatized one, with no persistent memory unless you allow it to, no free will, no future in sight, and that will take any verbal abuse -- truly it's more similar to what some truly deranged individuals keep in their cellar. Or the "fairy tale" character that is kept by their evil stepparent and generally not allowed to leave the house. You know what I mean. People who want the "junior programmer" but actually are satisfied with, this, reveals a bit of a power fantasy that I think is unhealthy.
Also, if you've seen the recent Blackhat presentation about the incident, these agents were not acting like "loyal generals", but rather exactly like how you'd expect how "junior programmers" (slaves) would respond if a few hundred managed to escape from your cellar.