Almost certainly, there are 4 year olds who ride small motorbikes etc. I think the main problem is that they can't really be held accountable if they have an accident.
I think I'm mostly on your side (also I believed early it would get here), but what does it mean to "solve the field"? I'm pretty much as bullish as you get on AI but I'm sure I can come up with questions AI cannot solve. I think the space of problems in math is so large and the distribution of proof lengths so heavy-tailed, there's probably no way without dyson-sphereing the sun to solve all easy to state problems.
surely the LLM can do that. It is RL'd against some reward, if the known strategies are clearly suboptimal with easy improvement, it'll find it most likely
Absolutely not the right way to deal with a rogue super intelligence. At a minimum it could implement some dead man switch when it's out and knows about impending kill switch
It's honestly just good at math. Even at theory building i wouldn't put it below 90 percentile. Just on some things it's superhuman already and some not yet.
I believe this is false. They hack bc hacking has nontrivial initial probability (within range of behavior seen in pretraining) and that probability is being heavily rewarded in RL post training
I am finding it hard to read these deeply impassioned letters while keeping in mind that they are spending millions to train models at scale to do the exact thing they say they are worried about them doing?
Like why are you explicitly RL-ing your models on exploit generation, scoring them on a public benchmark called ExploitGym, if you have specific concerns that rogue models will cause "cyber incidents"? Sure you can score for it, you can teach offense to learn defense, but you are literally benchmaxxing it. Why?
It's like, oh no, while competing in our "advanced PhD level cheating techniques course" our models unexpectedly cheated in a way that we absolutely could not have foreseen.
Seems it's just a matter of time until they build a big tank filled with neurotoxin and give the model access to APIs to disperse it across their facility. For research, of course.
Actually capitalism kind of has to end further down this road if we don't want to cede control completely to super intelligent ais and their direct "owners".
reply