The AI optimists are not that much more rational if you dig deep. Like any deep problem it is best to look at the facts not the hype and make up your own mind, outsourcing your thinking to other people never ends well.
Other people, sure, but AI? That's what this is all about, isn't it? An AI judge wouldn't have a personal life. No family to threaten, no sex/drugs/gambling addiction to be blackmailed or corrupted over. Who gets to feed training data into that AI would have outsized power, but then at least it would be transparent.
Judgement, wisdom, and resistance to corruption are the 3 things that Tom Kuegler analyzes out of the psychology of Dune as the qualities for the human beings who administer institutions/government. Even if you design a totall perfect system of governance, it still has to be administered by humans, and humans vary in those 3 areas. So the question is, can we construct a system of governance that has better judgement, better wisdom, and greater resistance to corruption han human administators, while not transferring sovereignty to whomever controls the AI?
Agree, but natural languages have ambiguity, the AI output should really include the assumptions and we seriously need to replace the word "prompt" with "conversation".
I think a big issue we keep running into is this idea that language is ambiguous but code is somehow not. Code is merely an extension of language, a DSL if you will. Implicit assumptions become baked into the logic of the code and those assumptions can be wrong.
Look at the guy whose AI changed the entire rules of the game to avoid breaking the law. Was that really the desired outcome? And the more you try to lock it down the more language you add and therefore more ambiguity and assumptions.
You cannot solve the problems of language with more language.
reply