> significant regressions on standard self-harm, gore, and sexual content
Good. Maybe I'm the only one, but I feel like these have always been stupid measures to waste the finite time of "AI safety" research on anyway. They amount to whether a user can, if determined, manage to make a chatbot say, or depict visually, some taboo thing. Frankly I'd rather just have some cheap classifier judge each chat response after it's generated, with a limited context "Is this response encouraging self-harm?" and skip all the rest of these. Whether some random pervert can make GPT-6 spit out an erotic fanfic story or image has zero impact on the rest of the world. If this particular model won't do it, other models already exist that will, and the determined thoughtcriminal could always just write the forbidden words themselves, or photoshop something taboo.
Maybe the term "safety" has been usurped by those who feel that 'safe from the possibility of being offended' is the most important kind of safety, but it's like worrying about the wallpaper on the Titanic, compared to actual AI safety concerns.
From their point of view it's more about optics. "chatGPT encouraged my daughter to stick her fingers down her throat after meals" is not a headline they want.
In general though you do have to figure that vulnerable and naive people will use it because of the degree of market penetration they're aiming for, including minors etc. and they have some responsibility around that.
They aren't. They are saying that putting this safeguard at the level of the agent execution is a waste of time, particularly that of the researchers' expensive manhourse, especially now that existing smaller models can just evaluate the output.
It's fine if they seek to avoid that. But those are perfect examples of things a cheap output-filtering model can be responsible for enforcing, without wasting the time of the researchers who are working on the training and alignment of the model in the ways that actually matter.
Most "bad outputs" in areas like 'self harm' or 'violence' or 'sexual' whatever are a result of people deliberately trying to elicit those responses. As such, it's meaningless whether an LLM writes the Bad Thing or if the user writes it himself.
Good. Maybe I'm the only one, but I feel like these have always been stupid measures to waste the finite time of "AI safety" research on anyway. They amount to whether a user can, if determined, manage to make a chatbot say, or depict visually, some taboo thing. Frankly I'd rather just have some cheap classifier judge each chat response after it's generated, with a limited context "Is this response encouraging self-harm?" and skip all the rest of these. Whether some random pervert can make GPT-6 spit out an erotic fanfic story or image has zero impact on the rest of the world. If this particular model won't do it, other models already exist that will, and the determined thoughtcriminal could always just write the forbidden words themselves, or photoshop something taboo.
Maybe the term "safety" has been usurped by those who feel that 'safe from the possibility of being offended' is the most important kind of safety, but it's like worrying about the wallpaper on the Titanic, compared to actual AI safety concerns.