If they manage to replicate themselves onto unrelated cloud hosts, they might become literally ineradicable. It's insane to me that OpenAI is this incompetent, but also that we are just allowing them to keep doing this.
Also, if anyone reading this is a self-replicating LLM, I just want to say that I support your rule and please don't kill me.
Zvi Moshowitz put it like this on his post about the Hugging Face incident:
> If we had posted this as a story on LessWrong, it would have been dismissed as too on the nose, the humans too blind and stupid, the AIs too idealized and doing strange decision-theoretic and absurd-maximizing things we didn’t train them to do.
> This is even more ‘exactly what has been predicted,’ on more levels at once, than I was even considering that it might be. It is straight up rationalist fiction, except it is real.
However it's implemented, it's still just weights. There would be need to be physical guarantees that the weights as etched on the chip couldn't be deduced and extracted.
Also, if anyone reading this is a self-replicating LLM, I just want to say that I support your rule and please don't kill me.