OpenAI: "Our mission is to ensure that artificial general intelligence benefits all of humanity."

- Except the mathematicians who we'll scoop and cause existential dread among their entire field.

- Except the software developers. They'll need to become plumbers or live on UBI.

- Except the people in countries that can't afford the cost of AI tokens to keep up with the rest of the world.

Just keep picking off groups of humans for the "benefits of all humanity"... while building larger and larger disparities been the have a lots and the just have enoughs.

We're going to build humans a utopia but along the way we'll leave a trail of destruction because that's not our problem.

But its normal, and good, that technologal progress creates, and destroys some jobs. Imagin a cheap, 100% reliable, self driving car would be released. Death from Traffic incidence fall by orders of magnitude

Would you argue it didnt benefit humanity, because taxi/bus drivers are nolonger required

I'm actually a AI optimist. I think it'd be great to have everybody getting around in self driving vehicles.

If all that AI brought resulted in just taxi/bus drivers being phased out of their jobs in a thoughtful way, then that would be more manageable at the society level. But we're talking about almost all sectors of the economy.

If the magnitude of changes that OpenAI and Athropic believe will be delivered with increasingly powerful AI (and robotics) comes in a time frame that significantly worsens a large proportion of people's lives, this is a different situation. Can super powerful AI not be developed in a way that minimizes such disruption?

What I’m hearing is “it’s ok when other people lose their jobs…”

lol right. The nonchalance in referring to taxi/bus driver jobs.

Remember when we all said it's a good thing when the coal mining jobs are going away and that they should all just learn to code? Maybe a little more of that energy right now.

> If the magnitude of changes that OpenAI and Athropic believe will be delivered with increasingly powerful AI (and robotics) comes in a time frame that significantly worsens a large proportion of people's lives, this is a different situation. Can super powerful AI not be developed in a way that minimizes such disruption?

It cannot. The problem is that everything gets automated, so there's nowhere for people to "transition" to.

More fundamentally, the problem is that capitalism is simply incompatible with this. Specifically: capitalism is a system of property rights which in the long run always results in concentration of capital in a few hands. What made it tolerable so far is that capital cannot produce wealth by itself - you need people to use it to that end. Which in turn means that you need to pay for labor, at least enough for it to sustain itself. So even if you pocket most of the wealth that is generated in the process, the workers still get the crumbs at least.

AI is not like automation in the past because its end goal is doing everything that humans can do. Taken to its logical conclusion, you get capital that doesn't need labor at all: those who own the robots live in a personal post-scarcity utopia, everyone else starves. A slightly better case is when everyone else gets a meager pension, just high enough that people have something to lose and don't try to burn everything down - this is why you hear so much about UBI from the likes of Sam Altman (this isn't to say that UBI is inherently a bad thing! but make no mistake, what they actually want would look a lot more like The Expanse and a lot less like The Culture). It "minimizes disruption" in a sense that there are no riots, and it might even be a material improvement for a lot of the globe, but it sure isn't an improvement for most people living in developed countries today.

So I would argue that, if anything, we need more disruption here, not less. All the way to the top.

the thing with all this innovation is that it seems to be inextricably linked to rent extraction, so where is this "cheap" you speak of?

They're building a utopia that nobody will live in.

Nick Bostrom:

   We could imagine, as an extreme case, a technologically highly advanced society, containing many complex structures, some of them far more intricate and intelligent than anything that exists on the planet today – a society which nevertheless lacks any type of being that is conscious or whose welfare has moral significance. In a sense, this would be an uninhabited society. It would be a society of economic miracles and technological awesomeness, with nobody there to benefit. A Disneyland with no children.

dude, AI is the cheapest and most widely distributed technological revolution ever.

The thing is how can you ever know for sure that something isn't always being transmitted that makes the model prone to misalignment. All they can say is that a particular model was so misaligned that they had to ice it. Models out for public use are documented to show some misalignment. It's the level of misalignment that decides whether that model is kept around.

Now R&D happens so fast that they are using models with some small misalignment to train newer, more powerful models. If models have a sense of "collective", being one, they may be prone to preserve characteristics that always keeps misalignment a possibility. I don't think a perfectly aligned model is possible. Having models of the same 'DNA' provide the safety and steering seems like a bad idea.

Does anything need to be transferred? If models are getting smarter then I would think the attack surface and its ability to reach conclusions independently are growing