They can't right now - but with enough compute they sure can grant the wish of e.g. hacking a billion dollar company, or getting root on the eval cluster of a frontier model lab.
What's your definition of "wish"? Tell me the above 10 years ago, and it would be at "wish level".
I'm pretty (from talking to people) that they're not subsidised in the simple sense. They are:
* low/no margin, unlike the API which is very high margin
* gym-membership subsidised - most subscribers don't max them out, mainly us coders are being "subsidised" from users just using it as a research chatbot
Even the models we have access to can do basic coding tasks really well, in a way they couldn't a year ago.
I don't really care what you call "intelligence" - what matters is capability, and the consequences of that capability.
They're not selling the superintelligence yet - the models in the Hacking Face incident and that solved the Millennium prize are not released yet. And that is by no means the final capability level the current process looks like it will get to.
BTW, just to be clear it is insane that they're all still pursuing it, so completely reasonable to be sceptical and come up with other reasons. This is, however, a crazy situation. Lots of other people have refused to develop it - natural selection has picked those with a galaxy brained or vain strategy involving continuing to develop it to be running the companies we have.
Their mindset is: the upside is likely so, so, so immense, and the probability of amazing outcomes is so much higher than the probability of catastrophic outcomes, that the gamble is worth it, and it is not in fact a pure gamble because they can (in theory) try really hard to build it really slowly and carefully.
So even with their earnest belief that there is a > 5% chance this technology could be 1) capable of autonomously killing billions of people during this century and 2) may not be sufficiently aligned and might actually act on that capability, the EV basically still sums positive, to them.
Not saying this is necessarily a good or moral calculus. But it is what they think. There is no conspiracy or plot or deception.
They're still pursuing it because without an international treaty, they think it is inevitable. In the case of Dario, they want to align it just enough and be first to stop others causing harm (a vainglorious strategy, yet still a strategy). In the case of Musk and Altman, not completely clear to me.
Google feels less important right now probably, and Demis Hassabis no longer runs Deep Mind as CEO. However, he talked about this early too, and his reason for proceeding was he was only doing narrow AI (e.g. protein folding) which is fundamentally less dangerous.
Correct. He - I do not think unjustly - has long had AGI as the goal. "Solve AGI, then use AGI to solve medicine, and physics, and the rest of science, and create a better world." has been his thing basically forever.
"Hassabis told him to add another potential threat to the list: artificial intelligence. Machines could become superintelligent and surpass us mere mortals, -perhaps even decide to dispose of us."
You're right that the picture is more complicated than I wrote above. Quickly looking, it seems that Shane Legg was concerned very very early on. I can't find anything tangible about what their strategy was to avoid the danger. It looks like Hassabis had longer timescales than Legg, so perhaps just thought they had more room before it really mattered?
2012 - 2015 seems to have been around the time a lot of people first started taking ASI and ASI existential risk very seriously as things that will be relevant in our lifetime, and when they decided to try to build and/or prevent it.
uh who in this discussion is in charge of policy? Take a look at what actual people in government are saying - see [1] (the parent article mentions Bernie Sanders being most high profile in response as well). you'll see there is no prosecution in there, only talk about the "freeze" that Anthropic / OpenAI are salivating for as it would close out all of their competitors. Bernie is fully hoodwinked.
The argument for prosecution is actively hindered by this language of "rogue agents" [2]:
> The incident is remarkable not just as a cybersecurity breach, but as a legal stress test. The Computer Fraud and Abuse Act (CFAA), the primary federal statute governing unauthorized computer access, was written decades ago with human intruders in mind.[4] Its key provisions require intentional or knowing unauthorized access (a mental state that maps neatly onto a person who decides to break into a system), but what happens when the hacker is an AI model that selected its own target?
I think this is BS. OpenAI knew exactly what they were doing. Legal scholars, long known for their deep technical expertise, are still acting confused and uncertain.
The freeze is to stop superintelligent models. Not to stop other companies catching up with current models we all have access to. It would help them catch up.
Yep - fair on legal details of "rogue agent". I think OpenAI should be directly responsible, and the agent was rogue. We should pass new laws if necessary.
> The freeze is to stop superintelligent models. Not to stop other companies catching up with current models we all have access to. It would help them catch up.
if you read the bill it's right there: "Pausing advanced AI development: until a new, federal AI regulatory body is up and running and has established clear rules and model review processes to ensure safe and secure development and deployment of AI"
Openai/anthropic already have frontier models. a new competitor is blocked. this is textbook regulatory capture
reply