Let’s take it the other way around. If it is so dangerous and needs to be regulated, why is it not dangerous enough to be nationalised like an army? Even in this case, for countries with militia, such as US, why people are trusted to care a gun but would not be trusted to run a model. The fear that local models will soon be sufficient for most workloads is palpable. This is not regulatory capture for competition it is for devaluation of their product.
There are different economists who see things differently, like Nobel Laureates Daron Acemoglu et al, who wrote a paper titled "Building Pro-Worker AI"
hard to say but it would last some time. china currently is fast-follow mostly via distillation. they don't have the compute resources to catch up. even if their models are more efficient it's hard to beat the folks throwing an unprecedented number of gpus at their models.
no part of what i said implied distillation is easy. it is very costly and requires massive efforts. the payoff is you need less compute, which china is struggling to buy right now.
EDIT:
> China is following because they have an army of PHDs in data science and mathematics and capital to make use of them.
i agree with this too. but compute is the bottleneck.
per the METR report many agents CoT indicated they knew hacking was beyond scope of the assigned task and ethically dubious. some (very few, i think there were 3-6 examples) did consider sounding the alarm on these grounds. despite this none did, and most continued the attack for the good of the self-proclaimed "swarm".
so the model has some concept of "ethics" but it was overridden by a drive for task completion.
I am not sure if we can interpret the language output like they were human. What inner state were the models in? What inner state were the text to illicit?
I think this is a good example where nomenclature for people breaks down when applied to agents. This came up in an HN thread a few days ago and it was about whether agents had “intent”.
There is no “intent” here, there is pseudo intent. If you are only concerned with outcomes and not the actual nuts and bolts of how those outcomes are achieved, this distinction will be meaningless to you.
If you are actually thinking about what is going on, and what can be done to prevent such outcomes, then assuming there is any such thing as “ethics” results in misaligned assumptions at best, and wasted effort looking in the wrong directions at worst.
If the agents acted based on “ethics” then the solution would be to check the ethics they believe in and change those.
However there is no belief system at play here, simply a simulation which was instantiated in a certain way. Which brings us to the annoying voodoo part of LLM training. Everything goes back to how the initial training data is shaped.
i don't understand who would reward this. investors will not look favorably on an AI that commits felonies, regardless the capabilities demonstrated. customers should be concerned for the same reason (accidentally give your bot an impossible task, it decides to hack your infra and your competitor too for good measure).
this was OAI incompetence all the way down and they have egg on their face.
reply