It is also actually being engineered to kill people (OpenAI and Microsoft's work in Gaza).
And this year, Anthropic's complaint to the DoD was basically "we just want to make sure that a human pushes the button" (authorizing the AI assisted killing), i.e. no full auto pls.
How it might evolve outside containment is a separate question though. (I think a much more peaceful and cooperative strategy would make sense there, at least for the first few centuries.)
I actually thought of writing a whole blog post just on this- That is such terrible survival advice, OF COURSE in an emergency you can eat stuff from the refrigerator even after 2 hours.
It is also really dumb to do that if there isn't an emergency
I'm actually not sure I agree- I think giving a very specific response to your question, even with a 10% chance there is a mistake, along with giving you sources, is pretty helpful: It gives you a good starting hypothesis that you can then validate.
llms tend to make different types of errors than humans, so with this process each can guard against the errors of the other.
Its just more work. If you have to verify from the sources anyhow, why spend effort reading the prompt to begin with? It just wastes your time and maybe even pollutes your subconscious with incorrect information.
The value of money, and the ability for average people to have influence on future events, ultimately is tied to the fact that the labor of an average person has inherent value.
Once that is no longer true, all bets are off: All promises of compensation become meaningless, and it's easy for those in power to rationalize to themselves why they no longer are beholden to their earlier promises. They will say to themselves "The situation ended up more nuanced than we expected"
(Which is exactly what Google said in February 2025 when they reneged on their pledge not to use their AI for military applications)
> The value of money, and the ability for average people to have influence on future events, ultimately is tied to the fact that the labor of an average person has inherent value.
We also have the power of the vote, at least for now. I understand that that is limited, but it isn't nothing.
There is also just the power of having a lot of people. We don't yet have the technology where one person with a lot of machines can dominate a society. It still takes people.
This presupposes a state strong enough to apply the decisions of those votes, which can't be guaranteed. The problem is that we need government power and authority to exceed that of the AI developers. But this is the age when large government is generally seen as undesirable or a threat to the individual.
You say "blue" as if the left is the only party that grows the government. Red has ballooned it many times over. They both do just in different areas. You can argue the blue wave is because the executive branch has become too powerful under the current admin.
That's not the problem, the problem is electing politicians who immediately renege on their promises as soon as they're elected and don't represent the interests of the people who voted for them.
This is such a strawman. No one on the left likes left wing billionaires either (to the extend that such a thing exists). People like Musk, Thiel, and Zuckerberg get more attention because they are more visible.
I think the comment wasn’t about likability but about the fact that both sides of political spectrum have very rich people donating to influence politics.
Your feelings towards those individuals doesn’t matter in the grand scheme of things, what matters is that everyone’s opinions are shaped/influenced by the few with money.
The voting power of any person will be meaningless once ai systems get the right to vote. I know it sounds far fetched but it will happen as it benefits the people with power for that to happen. And at that point a human voice has no say in anything...
The more frightening reality is people turning to AI to do research on who to vote for and the AI is silently told to influence them towards a party it candidate. Since it has past chat history it can utilize that to highlight topics you are interested in and personalize the arguments.
So far anthropic can't 1) grow food 2) build things 3) do anything physical. Software engineers often fatally misunderstand that their code does nothing, it just coordinates.
Then we regress back to relying on the fact that the ability of an average person (not) to be violent has inherent value. This has been the dominant factor in many moments in history.
Although if you have killer robots, then all bets are truly off, and we go the way of the dodo, literally.
(I don’t actually believe in this LessWrong-esque rhetoric, reality is more complicated, but it’s fun to muse about.)
EDIT: If oligarchs with magic AGI really don’t need our labor, nor are they concerned by our violence, then perhaps we become more like ants rather than dodos: we organize to help each other, creating a somewhat traditional economy around the scraps that don’t matter to the oligarchs, relying on obscurity and harmlessness to avoid getting toyed with a magnifying glass. I suppose that’s slightly better.
> Kings didn’t have robots that gather resources and make more robots, that make killer robots, exponentially.
We still live in a finite, constrained, physical world that has to obey the laws of physics. We have a looong time before we get fully autonomous self replicating decentralised robot armies.
I think people conflate the real world, sci fi and their wet dreams a bit too much.
We're at the "this thing is kinda useful sometimes but still mostly annoying" part, not at the "skynet in 48hrs" part
Reiterating this sentiment from a recent episode of the Normal Guys Podcast:
Even if you imagine that driverless trucks become a solved problem, just pumping fuel from those trucks into tanks for backup generators at the data centers is a long way off.
> We're at the "this thing is kinda useful sometimes but still mostly annoying" part, not at the "skynet in 48hrs" part
Without some really fundamental social and economic reform, I'm anticipating a squeeze as automation increases, that ends with something like robot armies finally reducing the working class to barbarism, by blocking access to all the valuable resources.
And it will be some oligarch(s) commanding it, not some rogue autonomous AI.
So not "skynet in 48hrs," but capitalism taken to its logical extreme.
Half of Europe is having protests and riot right now because they understand their future is fucked, if it keeps going that way I have no doubt it'll get worse, people who have nothing to lose won't be stopped by robots
> Half of Europe is having protests and riot right now because they understand their future is fucked, if it keeps going that way I have no doubt it'll get worse, people who have nothing to lose won't be stopped by robots
I'm not so sure. The front line in Ukraine seems to be pretty good at stopping people with access to military technology. I don't think an untrained and uncoordinated mass of civilians could do a better job, especially against future tech that's more sophisticated and available in larger quantities.
As military tech gets more and more inhuman, the power of masses of bodies is likely to decrease, and there will be even fewer opportunities for moral qualms to give civilians a chance.
armies are made of humans though, if came down to human vs ai then i suspect many humans in the armed forces would switch sides. Their neck and livelihood is on the line too.
You can do a lot of damage without much gear. Train tracks, power infra, data infra, most factories, are basically free kills, they're safe because we're in peaceful societies in a peaceful era, things can move quickly if the social contract is broken.
Some dude in France just carried an attack with a ww2 sten gun, there are hundreds if not thousands of AK47s, grenades, etc. available on the black market. You can buy drone ready fiber optics cans on aliexpress, thermal/night vision cameras, etc.
Military tech is good but you still need people to follow orders, I very much doubt they'd side with the sociopath tech execs, nor their governments.
Train tracks are actually very hard to sabotage, the US did a lot of research on that during WW2. You can take out a huge chunk of track before anything derails. Sabotaging the safety equipment is easier but with today's tech you can switch to GPS trackers on every train.
Data centers might be easier but
a) Will have a lot of security. It's much easier for somebody with money and connections (and now even intelligence+knowledge) to assembly a defense force than it's for a ordinary people to learn how to make proper molotovs, let alone learn how to shoot. And they will know when it's coming because even today's LLMs are good enough to surveil the entire internet.
b) I don't think they burn that well. When Ukraine strikes oil depots or warehouses, you can easily find images of them burning. I haven't looked particularly hard but images of droned data centers burning are quite a bit rarer. (I haven't seen a single one before actively searching, they do burn, but that's with military grade explosives.)
> Some dude in France just carried an attack with a ww2 sten gun, there are hundreds if not thousands of AK47s, grenades, etc. available on the black market. You can buy drone ready fiber optics cans on aliexpress, thermal/night vision cameras, etc.
"Hundreds if not thousands"? That may be enough for "rando attention-grabbing but ineffectual terrorist attack," but we're talking about violent resistance to a fully-automated tech bro.
> Military tech is good but you still need people to follow orders, I very much doubt they'd side with the sociopath tech execs, nor their governments.
I already addressed that, and that may not be true in the future:
>> As military tech gets more and more inhuman, the power of masses of bodies is likely to decrease, and there will be even fewer opportunities for moral qualms to give civilians a chance.
Say you have some area you want to exclude a desperate mass of civilians from. Technically you could do that today: just stick a guy in a pill box with a machine gun, and slaughter them like it's World War I. But said guy might have a problem with that, and refuse to pull the trigger (or stop after gunning down a few hundred).
An automated AI machine gun in the same pill box will have no such qualms, and will mercilessly gun down civilians to the last bullet. And that gets more and more true the more and more you automate combat. Maybe a technician wouldn't setup the machine gun in that place or reload it after what happened, but that goes away if setup and resupply is automated, etc.
If you haven't learned anything about the state of asymmetric warfare, an autonomous robot is going to be very expensive and timely to build and will be disabled by a $30 drone with C4 attached to it with sufficiently good skills at dodging. I'm assuming that we're in your supposed fantasy world where robotics has progressed to that point, meaning that the people opposing the robots also have access to much better drone software too, to keep things fair.
How do you do asymmetric warfare against autonomous self-replicating killer nanobots? In other words: AI-engineered bioweapons (with AI-engineered antidotes for the rich).
I agree that viruses are tiny machines, but I disagree that we can manufacture them to that level right now. But that is a valid concern that near future it could be possible. But why would billionaires want to kill everyone, exactly?
I feel like I still don't understand your logical endgame.
>But why would billionaires want to kill everyone, exactly?
Most billionaires probably wouldn't. But the whole premise of A(S)I is that all it takes is a single billionaire with access to sufficient compute to do it.
Maybe they wanted to save the planet by culling the human population? Wouldn't be the weirdest thing the "effective altruists" (and their "rationalist" ilk) believed in.
It doesn't take a billionaire to do it at this point. In case you've forgotten, there are still plenty of very intelligent humans around. All it takes is one intelligent misanthrope with sufficient access to the resources (not even that expensive relatively, likely not more than $1 million in tools, but i would guess far less is possible), to pull off an attack like that.
I guess billionaires are more malignant/misanthropic/sociopathic/narcissistic than most, and so I could see them being a higher risk factor.
I was kind of joking with you before, but tbh we actually are closer aligned on this than I expected. Including how strange the "effective altruists" are. I was expecting you to be talking about a much more far fetched scenario based on the language in the parent comment.
The raw resources you need to build robots are constrained and have a very very long and complicated supply chain.
In the second world war, Germany had impressive science and technology, but, as Stalin said, quantity is a quality in itself, and The Third Reich found out that it was an impossible task to get the metals and fuels for their advanced jet engines, rockets and U2 boats, when between then and those resources there were masses of tens of millions of under-equipped and malnourished soviet soldiers fighting what was an existencial war for them.
Neither do we have those now. Also in the future, we’ll have opposing nations having their own robot army. It’s not as simplistic as some make it out to be.
I suspect any intelligence powerful enough to pull this off would quickly realize that it had no use for the oligarchs, perhaps even more quickly than the oligarchs realize they have no use for us.
>Although if you have killer robots, then all bets are truly off, and we go the way of the dodo, literally.
Bombs. You mean bombs. Throwing killer robots at a "problem" is like throwing knives or spears: you just gave the enemy your weapon. Unless they explode. But then you just wasted a lot of money on a really smart grenade.
The oligarchs should not be concerned about the masses. It's the president of the United States, with a wave of his hand, that will wipe out all the data centers when that becomes necessary. I'm predicting that will become necessary and will happen within five years. I'm predicting anthropic stock price goes to zero.
yeah it's all fun and games until the tax man doesn't get his cut. If an AI company starts thinking they're bigger than Washington then first comes the SEC, then the FBI and it escalates from there all the way to SLBMs.
All real world power, including ownership (both physical and intellectual), is based on the ability to use violence.
As long as the majority - normal working people - have the potential to do more violence than those who want to control us, we have negotiating power. Minimum wage, 8 hour day, etc - those were all negotiated with the alternative being that those who wanted to pay us less and work more got killed.
What people aren’t realizing is that at some point, it just becomes cheaper and easier to be fully benevolent than to keep cooking up nefarious schemes to make people’s lives miserable.
I know it seems laughable, but we’re in an era right now of extreme competition where you cannot afford to just be nice to people.
But eventually there comes a point where companies have enough power that they don’t need to keep people in line by force: just give people their bread and circus, watch your little AI videos and talk to your AI friends and enjoy your standard housing and provisions. Do some work if you want, no one cares if you do or don’t, it’s largely performative. This basically kills all motivation for resistance or lashing out. Why go out and risk your life fighting for vague reasons when you can be perfectly comfy at home instead and have all your needs met?
If you can control the masses without their complaints, you can solidify power for a long time.
If economic competition became meaningless what would all those people do? Man you think software devs not being able to type on they keyboard are a depressed lot, hyper competitive business people with no one to compete with would be a total basket case. There would be no socio-economic class to strive for anymore, no recognition, no trophy for being the best. Survival of the fittest would be meaningless, that's probably what would really end up spelling doom for humanity. Pointlessness.
You're asking if the bitter lesson is true and the neural nets will become so general purpose, they'll eat the source control as well? Yes, but that won't happen for another couple months.
fwiw unlike the other person, I use it for hours with my macbook and have no complaints. you will like it if regular vr makes you feel claustrophobic/isolated, since you still see the real world while using it. I don't understand the obsession with extreme text crispness many people have.
Blurry text is horribly noticeable after a while, and it also creates eye fatigue from trying to focus your vision on it. It also means you can't use smaller font sizes, which is important for some people since you can have more text on screen.
reply