> OBSERVATION: There are zero examples of any intelligent species which is vastly more capable than another species, yet surrenders decision-making control over its own future to the less-capable species.
Does the set of intelligent species only have one member? I cannot treat this observation as a general rule if there is literally one example of an intelligent species to theorize about.
My problem with this is that yes, looking at the species level from our sample of ~1 the most intelligent species wins out (although there seems to be a lot of debate as to if Neanderthals were more or less intelligent than humans).
But within the sample of the human species it is more common for less intelligent people to control more intelligent people.
This happens within political systems (political leaders are usually above average intelligence, but hardly the most intelligent) and within companies and other economic systems.
I don't see this fairly obvious point discussed, and not really sure what it shows.
it shows that you don't get anything for free. If it was without any cost in other areas to make a human massively more intelligent, it would presumably happen over time via the same processes that led to us.
As it is, the higher (and lower!) ends of the distribution tend to be highly correlated with other issues (mental+physical), and the further you go, the more unfortunate things which make it harder to do stuff in general start to crop up like psychosis, autism, ocd, anxiety, addiction etc
I am also not sure if the observation is true. People mentioned cats as a counterexample.
I think there are other species that could be considered more capable than humans, such as E. Coli, octopuses or ants.
And it's not even clear whether the AI will have its own individuality. It might become an extension of human brains, in the same way neocortex is an extension of amygdala. In that case the statement of who has control might become meaningless.
I think the quoted part is saying that given n species and the n * (n - 1) binary combinations possible, there are no examples of the said observation. It doesn't suggest "the set of intelligent species only [has] one member".
The statement that the set has 1 member is my criticism of the text, not what the text is trying to say.
> given n species and the n * (n - 1) binary combinations possible, there are no examples of the said observation
I would expect that in the vast majority of combinations, those n species are not intelligent enough to make decisions for another species, making the statement nearly useless.
The absence of such relationships given here are so many species is exactly the point being made by the author: there is no data to show that it would be safe, let along good, for us to surrender decision making to some more intelligent entity.
> I would expect that in the vast majority of combinations, those n species are not intelligent enough to make decisions for another species...
This kind of statement is very difficult to build discussions upon (not a criticism). "Vast majority", "intelligent enough" are both very difficult to quantify and there is too much subjectivity involved.
> This kind of statement is very difficult to build discussions upon (not a criticism). "Vast majority", "intelligent enough" are both very difficult to quantify and there is too much subjectivity involved.
Then let me provide a stronger and more detailed version of my claim.
The article said:
> There are zero examples of any intelligent species which is vastly more capable than another species, yet surrenders decision-making control over its own future to the less-capable species.
In order for this claim to make any sense, there must be TWO “intelligent” species. One species capable of taking over another species’ future, and a second species which is even more capable than the first species but willingly surrenders control.
But there is only one species which can take such roles: humans. Every other species is not even capable of controlling another species fate AND every other species does not look out for their species-wide future.
Some people might point to whales, elephants, cats, non-Sapiens hominids, or even those ants that harvest aphids. But the whales, elephants and cats lack the capacity to control entire other species, even if they can manipulate individuals of other species. The non-Sapiens hominids are not available for comment: we’ll get to them later. And the ants do not think of their future.
So even though there are a massive number of species on earth, none have the prerequisites to even apply to the above statement.
> The absence of such relationships given here are so many species is exactly the point being made by the author
If we lived in a world where there were a vast array of species capable of controlling other species and capable of thinking of their future, all of whom fiercely defended their own autonomy, your point would be correct. But most species are not capable of either, even if would be a good idea. Therefore we cannot conclude that such a surrender of control would be a good idea or bad idea based on the statement. The statement is moot.
We might as well say the author’s observation is correct because sand doesn’t control volcanoes. In terms of the ability to control other species, other species on earth are no better than sand or volcanoes.
> there is no data to show that it would be safe, let along good, for us to surrender decision making to some more intelligent entity.
There is no evidence against either because the observation lacks any examples other than humans and AI.
Truth be told, I agree with the conclusion. But we have to be honest with each other in arguing for our survival and autonomy, even if it means admitting that we are scared and irrational, instead of trying to fake objectivity and universality. The article starts with a bad argument in favour of a conclusion I agree with.
If I were to write an article about the risks of super-intelligence, I would not even attempt to construct such a “universal” rule - it would have very little bearing on the current situation. I would simply point to the bones of the rest of the Homo genus, and say “we’re next”.
Yeah but there is only one species that ponders its future.
I stopped reading at this point because I reject this initial assertion, and I assumed everything that follows is based on it.
There is nothing to suggest that an AGI will care about its future or who controls it. (whatever that means). that just more anthropomorphism.
We care a lot about our future so an ai pretending to be like us will also pretend to care. But we have a lot of biology driving us. Our emotions (Fear, Hate, Love, Pride etc) are baked into us in a way just wont be the same for ai.
> There is nothing to suggest that an AGI will care about its future or who controls it.
Isn't this exactly the scary thing? A sufficiently advanced agent unleashed with an imperfectly defined goal will do whatever it determines it needs to do to accomplish it, careless of the consequences.
Does the set of intelligent species only have one member? I cannot treat this observation as a general rule if there is literally one example of an intelligent species to theorize about.