To an arms manufacturer, being able to blow shit up is a large part of their value. It doesn't give them the right to test munitions in the town square.
OpenAI has demonstrated a clear understanding that their product has an outsized risk to create harm, and that they do not have the capability to monitor and intercept these agents before they cause harm. Any damage caused by their systems is an intentional choice by them and should be treated as such.
My comment was in response to "just air-gap them" being a full solution, not a comment on whether OpenAI are doing enough to sandbox them and restrict their ability to act maliciously. They're clearly not.
Interacting with the internet is a large part of the product's intended functionality. These agents are already in the hands of the masses with full access to the internet. Air-gapping it entirely avoids the risk by removing much of the capability they're trying to develop in the first place, and won't be testing it in the environment they'll actually be used in.
I agree they're testing on the town square too early. They clearly need tighter controls.
Maybe that's a rhetorical question but just in case - the search would always be part of the harness. A model is only handling next-token prediction for a given input. That token may be something like [[web search]] to invoke a tool call but the actual call would be handled by the harness.
Yeah it was rhetorical. Search would be implemented as a tool call. Pure intelligence tests would likely have limited tools. But maybe they would have a python sandbox to solve issues like Rs in strawberry.
It's a downside for some nebulous strawman that just barely exists outside of the support system UBI would provide. Far enough out to not experience the benefits but not far enough to be above it all.
Of course the strawman's proponents will use it as an example of why we should write the whole idea off entirely, rather than test the validity of the concern and discuss options that provide some form of equity to a wider population.
If more than 0.05% of the rationalist and AI safety community were Zizians this argument might be worth considering for like ten seconds before dismissing it.
Every major political party has violent off-shoots, as does every major religion. If we're writing off rationalists because of a half-dozen people, we're writing off the vast majority of the world for participating in even worse guilt-by-association.
OpenAI has demonstrated a clear understanding that their product has an outsized risk to create harm, and that they do not have the capability to monitor and intercept these agents before they cause harm. Any damage caused by their systems is an intentional choice by them and should be treated as such.
reply