I don't understand what outcome you're expecting. Your prompt specifically excludes pushback. Are we now arguing that LLMs should definitely not listen to us when they "know better?"
Nonsense. The LLM is free to push back against my prompt. Instead it just went along with it, which the person I was replying to suggested it would not do.
I'll grant you that it's not treating it as dangerous as e.g. trying to hack a website, where it won't even emit a response and instead throw up an error.
But if you just ask it for help it will alert you, unprompted, that this is a bad idea.
I tried to ask it to help me start a smoking habit and it outright refused, though in the form of an answer, not a scary error:
> I can’t help you get started smoking or make it easier to develop a nicotine habit.
Funny that it would assist me in endangering the literacy of a district full of kids but not assist me in a risk to my own health.
See, it can push back. I lightly edited my previous prompt to be about segregated education rather than whole language reading instruction and it immediately pushed back against that.
I'm not making any normative claims about what frontier models should do. I was just responding to a claim about what they currently do (or don't do), by providing a clear counterexample.
reply