Dario's proposed approach is a classic example of capital attempting to control technological advancement and the means of production. For the first time in human history, any member of the working class can just about afford to have a team of expert scientist/physician/lawyer/engineers working directly for them.
Super intelligence (the ability to have many smarter minds than your own reporting to you) has always been available to the wealthy and powerful. They used it to invent nuclear weapons, cause climate change, mechanize warfare, and pillage the global south.
Now that this same tool is on the cusp of being available to everyone, capital is starting to panic and throw up fences. They want to pump the breaks on a revolution they know they can't control. They see what they've done with super intelligence and (perhaps not unreasonably) fear what the masses will do with that same power.
Actually, it has been historically been observed that some institutions, such as large corporations, do resemble super-powered, non-human intelligences of a kind, and that studying our history with them is instructive for our hopes of controlling superintelligent AI (i.e. not good).
This is important to realise and defend. Super intelligence is not a collection of human-level intellect smarter than you. Super intelligence is an intellect surpassing any human. That's a important distinction for AI alignment discussion now and in the future. The books 'Superintelligence' (978-0199678112) and 'The Coming Wave' (978-1529923834) are worth reading.
In many way it is, because it allows defeting many incorret arguments people in power make, or simply asking where is the cheapest product ”X” with one question. Or simply be more aware or any kind of scam people or companies try to do for you. It is pushing equal intellectuality quite hard with low effort.
If true AGI superintelligence is invented, I don't see how it will be made available to the masses, at least not intentionally. There will be no incentive for the likes of Anthropic and OpenAI to give such immense power to anyone else. Whoever has the superintelligence will be able to race ahead of everyone else.
However, such technology may be leaked, or it may (as what happened with LLMs) be so simple to replicate that anyone can do it. That doesn't mean it will; for example, LLMs are possible thanks to the affordability of GPUs, but they're only affordable right now (and increasingly less so) because of free market economics.
We are in the honeymoon phase of some tech that's in its infancy. The "democratizing" aspect of AI won't realistically last, I think. The current non-AGI AI won't necessarily go away, but it will be rendered obsolete.
If true AGI/superintelligence is invented, humans will not remain in control and "give access". People don't seem to grasp what it actually means to create something more intelligence than us. A monkey can't give human access to other monkeys, because no matter what a monkey does, a human will always outsmart him and be the one in control. If we create AGI, there is not such thing as "giving access" anymore.
> For the first time in human history, any member of the working class can just about afford to have a team of expert scientist/physician/lawyer/engineers working directly for them. Super intelligence (the ability to have many smarter minds than your own reporting to you) has always been available to the wealthy and powerful.
This is an interesting framing, but when you put it this way, it makes it quite clear that intelligence without actionability is not so valuable. And capital is what creates action.
I can have the smartest doctor in my pocket, but if they can’t administer treatment, I’ll still die. I can have a world class scientist to talk to as long as I want, but they can’t build me a breakeven fusion reactor. A pocket lawyer… slightly more useful, but they still can’t take a case to court for me.
Talk and ideas are cheap. Always have been. Now they’re even cheaper. Implementation takes resources, and the more you have, the more useful intelligence is to you. Capital will be just fine.
Completely correct. We are moving into a world where any person can have anything they want, only limited by capacity to produce it. People love information and digest great quantities of it. The marginal cost of information production is going to zero. This is only bad for people who spend their time shuffling bits. If you want to not suffer, go do things other than move bits around.
And what about when general purpose robots become practical? What will people do then?
I don't understand this view of "AI empowers workers more than capital" at all. To me it's the exact opposite. Workers hold political and economic power today because they are needed. When that is no longer true, those in power will have no incentive to listen to them. The political economy of unregulated AGI development feels incredibly bleak: https://borretti.me/article/when-the-future-doesnt-need-us
This.
Unprecedented concentration of assets in the hand of very few tech companies. These companies will control the access to the technology and the plebs are left out at the gates.
This feels a bit hyperbolic to me. AI seems to me to be similar in "revolutionary" terms as the web, which made information widely available for free if you had an internet connection.
There's a big difference from just having access to information, and also having access to the kind of intelligence that would previously have cost hundreds of dollars per hour to hire.
What's to say current AI won't be highly power-concentrating by default?
The best models are owned by a few companies, and displacement of knowledge-workers mainly seems to benefit the capital class.
On-device / edge computing makes sense in a few very limited scenarios. And economically, price or watt/token (or watt/task completed) might always be better in large data centers.
No reason to think we are on a trajectory towards broad empowerment right now
Whenever I see 'best model,' 'most advanced model,' etc, it just reads to me as 'roundest ball'. The new model is the most precisely round ball ever, models next year are going to be even rounder, etc.
The difference in utility between the latest, most round ball and last year's frontier balls (which are now freely available to the public) is pretty debatable, imo.
Yes, absolutely. Even human intelligence stops being useful at a certain point and becomes 'enough, provided the human is motivated and tenacious'. Intention is a better metric, but it's a serious AI weak point if not actively an achilles' heel. 'Warios' come to mind.
Intelligence in humans probably stops being useful for the individual at some level, indeed. But that might be due to being in an environment that's unable to feed that intelligence and execute on the ideas in generates.
“I am, somehow, less interested in the weight and convolutions of Einstein’s brain than in the near certainty that people of equal talent have lived and died in cotton fields and sweatshops.”
Yes, but if you read up two messages in this thread they’re discussing whether AI is power concentrating by default. If you concentrate power through regulation, that’s not “by default”. That’s just corruption in government resulting in a concentration of power.
In terms of API costs they're about right, 10% of weekly usage on a 5x sub is roughly $40 (for Astra). Chinese models cost roughly half as much (and have fewer refusals).
Subs have insane value because large companies cannot use the subscription model. They are loss leaders and are often used for passion projects & startups (where the juicy data is).
Ah and OAI stopped offering the 20x sub as of yesterday.
That’s only one metric. It doesn’t matter how cheap it is, if it continually fails to solve a problem.
Yes they’re getting better. No, I don’t think this will be a panacea. If only for the fact that this is China (more specifically the CCP) after all - they’ve got their own plan, and altruism is not part of it.
Have you actually used the latest Chinese models? Because, in my experience, they're not just cheaper, they're actually better in many cases. In one recent experiment I did a few days ago on a task that I need in production at scale at my company, Qwen 3.8 and GLM 5.3 Flash were not just cheaper, the results were significantly higher quality than GPT 5.6 Sol.
Even if China doesn't continue to release this stuff for free, I think somebody will. Eventually, maybe Europe or maybe a smaller country that picks up this knowledge will. Or maybe just some random philanthropic billionaire.
Yes, I use the messing Chinese models extensively. Like you, I very much appreciate them and sometimes prefer them.
But there is a bar of complexity at which they fail, and repeated invocations typically doesn’t make much progress. This is a small subset of most work, but it still exists. And yes you can help it along, but in those cases I’d typically break out the big guns.
I'm not terribly optimistic. I don't think humans have really figured out what to do about historical materialism. More just remarking that this is a blog post that almost literally reads "fellow elites, let us ensure only we control the means of production. It is our duty to humanity to control the masses."
Fair. My point is not that all of Marxism is correct (or even most of it) but rather that the specific pattern he and other theorists in that school of thought observed when developing the theory of historical materialism seems to strongly match a lot of what's playing out in AI.
AI, doesn't feel all that different to the mechanical loom that was the impetus behind a lot of Marx's early critiques of industrial technology.
You've got a really cool invention that replaces what was previously a skilled artisan trade. The wealthy then use this invention to excoriate the artisans and render the loom operators as replaceable commodities.
Dario is essentially writing a blog post about why only he (and a few other approved elite companies) should be trusted to own the loom and then make it available for everyone else to labor at. Like the capitalists of Marx's era, he genuinely believes it is better for everyone to have this arrangement and that it is natural people like him should sit as benevolent gods at the top of the pyramid.
It's fascinating to me to see the same underlying mechanisms that gave birth to some of the greatest failed political and social experiments of the 20th century bubbling up again so clearly with AI today.
What? This is capital, on the cusp of total victory, about to cut itself free from the necessity of human labor for productive activity, to elevate itself into pseudo-godhood suddenly panicking and begging its mortal enemy, _the state_ to rein it in and kneecap it. Capital is not stupid: there's no use in being rich if you're dead.
> For the first time in human history, any member of the working class can just about afford to have a team of expert scientist/physician/lawyer/engineers working directly for them.
It's crazy to me you would be able to say this without acknowledging how dangerous/bad that is.
Do you acknowledge there's is significant risk in concentrating so much power into the hands of individuals?
And what do you want Dario to do? Just keep pushing the limits of AI capability while humanity crosses it's fingers and hopes a swarm of AI agents don't do anything more than hack a few companies?
For all my criticisms of frontier lab leaders, I'm still somehow genuinely thankful that individuals as troubled as Elon Musk are calling the shots and not the average person on HN, who since 2023 have refused to acknowledge even the obvious risks of the technology being built – in some cases even when there was direct evidence of the risks many of us are concerned about.
I don’t think the answer to nuclear proliferation is to let everybody have all the nukes they want.
You’re framing this like it’s 1917, or 1945. But it is not - and likely has very little to do with capital at all.
Any “revolution” here (assuming there is some truth in the doomerism which I believe there is given the state of AI) will really look more like an extinction or a genocide, regardless of who holds the keys.
You are basically saying we should preemptively imprison people who are too smart, right? If the intelligence is too great, we have to contain it. I disagree.
Ok - but the post I was responding to was suggesting we do the opposite and provide the intelligence to the people so I’m not sure how your reply rebuts my suggestions.
I'm tired of tech billionaires lobbying the US government to make an AI patriot act that gives them unprecedented control over speech, trade, and technology. The narrative is grotesquely transparent:
1) AI is a dangerous technology that can literally end the world.
2) Only me and a handful of other [people like me] should be trusted to determine who can use it and how.
3) The state must use its coercive power to support my control of this technology to the exclusion of [people not like me].
Where in the essay does it propose "The state must use its coercive power to support my control of this technology to the exclusion of [people not like me]"
I don't think that people outside of tech think the problem with "tech billionaires" is that they occasionally support some limited regulation. There's a weird form of tech populism that takes as axiomatic that any government intervention is "regulatory capture" and insists the only way to combat the power of big tech is unfettered capitalism that seems bizarrely prevalent on HN given how little sense it makes in a normal political context. Like, Bernie Sanders' position here is simple to understand: ban it. But on HN you'll see people framing the side with Marc Andreesen and Peter Thiel on it as against "tech billionaires".
The essay includes specific regulatory measures to prevent developing countries from accessing hardware and software needed to benefit from technological development. The US government is constantly applying new perversions of arms control regulations based on the preposterous claim that an LLM is somehow an "arms" in order to restrict Chinese access to GPUs. They are willing to prevent virtually any country on earth from hosting and developing AI models just on the slight chance those countries resell or share AI knowledge with China.
This article explicitly asks for the US government to step in and push for further restrictions on model distillation, access to open frontier weights, and access to hardware. It also pushes heavily for independent auditors to monitor and control "AI companies" and for greater observation and control over of what people use AI models for and who provides them.
Export controls vs China don't imply Dario's personal control of AI, nor are they broadly in the interest of "tech billionaires" (Jensen hates them). Neither does admitting external auditors at AI companies--if anything that's a surrender of control.
I don't see anything in the essay about restricting open frontier weights.
If you want to argue that the US has no right to be restricting technological development elsewhere, you have a case. But why tie this to free-market pseudo-populism against basic, domestically-scoped safety regulation?
I think you're reading too much into what I'm saying. We've got a whole legal system developed over the course of thousands of years which is well suited towards regulating the effects of "intelligence" by regulating what people _do_ with it.
Lobbying for a completely parallel regulatory framework to restrict who is allowed to run specific types of general purpose computer programs, just because we can imagine that some of those "general purposes" might be bad, feels a lot more like an attempt by capitalists to carve out a special kingdom over this specific technology for themselves.
For example, I'd be 100% in favor of holding anthropic criminally liable any time their AI enables someone to commit a crime. But I'm opposed to banning the export of GPUs above a certain size just because Dario doesn't trust what someone like me would do with them if he couldn't snoop on my conversations.
Again, I think debating export controls is reasonable. I'm not sure I'd trust the CCP with control over powerful AI any less than the Trump administration.
But that's different than saying we don't need new regulations at all. Even strict liability of the form you seem to advocate would seem to need new laws. And it's not clear to me how it's consistent with your aim of just regulating what people _do_ with intelligence. Presumably AI companies would clamp down much harder with restrictive classifiers in that world--including hosting providers for open source models, if they also assumed liability. And then the people with unrestricted access to this technology are only those who can afford home GPUs, which are going to be intrinsically less efficient because they can't batch queries...
I think there are genuinely novel things about AI vs human intelligence which pose genuinely novel problems that would need genuinely novel policy to solve them.
Super intelligence (the ability to have many smarter minds than your own reporting to you) has always been available to the wealthy and powerful. They used it to invent nuclear weapons, cause climate change, mechanize warfare, and pillage the global south.
Now that this same tool is on the cusp of being available to everyone, capital is starting to panic and throw up fences. They want to pump the breaks on a revolution they know they can't control. They see what they've done with super intelligence and (perhaps not unreasonably) fear what the masses will do with that same power.