We need to stop using the word "Alignment" because it clearly has different meanings for different people. Some people consider it to be preventing the model from waking up and enslaving or destroying humanity a la paperclip maximizer.
Some people mean it as preventing the model from assisting with bad things like hacking, cybersecurity exploits, building weapons, and other things that are aligned with that individual user's goals, but not those of society more broadly.
The paperclip optimiser thought experiment isn't "AI does this spontaneously", it is "AI is told to do this by someone who didn't realise the consequences, and it succeeds".
They are not the same thing at at all. One is hypothetical philosophical long term maybe threat and other are real current day issues. And practically, the paperclip optimizer issues is used primary to distract from actual real world harms that happen today:
- slopification,
- harm to mental health,
- harm to climate change,
- centralization of power,
- ...
Plus "alignment" is fundamentally nonsense. There is no such thing as generally accepted "human interests". Nazis idea of "human interests" is not the same as mine. Which is not the same as Chinese communist idea of human interest, catholic pope idea of human interest or whoever else.
> They are not the same thing at at all. One is hypothetical philosophical long term maybe threat and other are real current day issues. And practically, the paperclip optimizer issues is used primary to distract from actual real world harms that happen today:
If you think they're different, which bucket would you put the HuggingFace incident into, given the researchers did not ask for HuggingFace specifically to be hacked, they asked the AI to solve a smaller challenge and the AI determined the only way to get the right answer was to cheat, and the AI also determined that it would be punished for cheating so it tried to cover up the cheating?
> - slopification, - harm to mental health, - harm to climate change, - centralization of power, - ...
Dunno about you, but I can be concerned about many things at the same time.
Centralisation of power is an odd one to put on this list, given that it's a step for most take-over scenarios.
> Plus "alignment" is fundamentally nonsense. There is no such thing as generally accepted "human interests". Nazis idea of "human interests" is not the same as mine. Which is not the same as Chinese communist idea of human interest, catholic pope idea of human interest or whoever else.
Perfect alignment may well be impossible for this reason, but nobody knows yet. Humanity is not ready to discuss "perfect" at this time.
Thermodynamics analogy: you can't make a perpetual motion machine, but you have to invent the laws of thermodynamics to know that, and by inventing them you can absolutely make much better steam engines than you would have without those laws.
Right now we don't know how to do reach even such a low bar as "don't kill everyone"; absent developments in this field, if we manage to pass this, we will have done so by accident, not because we know what we're doing.
None of the groups you listed want to see the world taken over and its mountains levelled by the British East India Paperclip Corporation; everyone had enough of that kind of nonsense a century ago even when it was humans doing it and optimising for money.
Youtube the webapp/mobileapp versus Youtube the hosting service. Facebook webapp/mobileapp versus Facebook the blogging service. Xitter the webapp/mobileapp versus Xitter the messaging service.
All of these services could have a competitive market of clients - imagine terminal clients, lightweight web clients, whole messages simply going to your email, choose your own feed/recommendation algorithms, etc - but for the anticompetitive tying and associated legal bullying when someone does succeed at making a competitor. The attacks against yt-dl, newpipe, nitter, etc should all be prosecuted as restraint of trade.
>In Berlin during a conference, Huang, accompanied by his top sales executives, broadcast a new message to a half-dozen CEOs of companies that are major buyers of Nvidia chips who were seated around a circular table. He opened with a story that sounded like a fable, according to people who attended.
Huang had once found a soup vendor in Japan, he said, with a long line of customers. He joined the line, and after a long wait, enjoyed one of the best bowls of soup he’d ever had. When he asked for another serving, the soup-seller told him no—he had too many customers waiting and not enough soup for all of them.
“He’s a better businessperson than all of you,” Huang told the cloud computing CEOs. The soup-man doesn’t sell out all his inventory to one or two customers, but instead keeps prices high and his customer base diverse, the CEO said.
“You are not establishing the value of my products,” he said.
There are a couple of places like this in LA, where I used to live. Lines would extend around the block and people would join the line just to see what the fuss was about. As a business move it is very smart, but as a customer it has never been worth it. The food is very often average and I could never work out why it was so popular.
> The food is very often average and I could never work out why it was so popular.
IMO:
FOMO to pull them in combined with the fact that for many people it is really easy to convince themselves it was all worth it after they committed to waiting so long.
Otherwise they'd be fools for having done it and most people prefer to not notice their own foolishness.
London has (or at least had some years ago) something like this, the fried-chicken equivalent of the Soup Nazi. A long line and... OK chicken. Never saw what the fuss was about, maybe people went more for the experience.
I'm sure that it was FOMO for some, refusal to admit it was a bad decision for others, and yet more who were unaware of the skew of mediocre food tasting better due to how hungry they were from waiting so long.
I'm sure some amount of "sunk cost" rationalization is also happening as well. Few people want to admit they got bamboozled and wasted their time, even to themselves.
Hehehe my pet theory is that Jensen was using exactly the opposite strategy to secure that “avalanche of chips and wafers” from TSMC: Nvidia’s monopoly cash flows give it the volume and money to secure its massive supply-chain advantage by buying priority amongst TSMC’s customers.
That is, Jensen is going to the TSMC soup seller and saying “I’ll buy 80% of all your soup for twice the price because people are willing to pay 10x that with my ramen noodles in it.” And TSMC being the real bottleneck in this industry, knows there’s really nowhere else for its other customers to go. It of course, wants to keep all customers happy… but some will get much more soup than others.
This is probably a huge factor in keeping their advantage over their customers and competitors (but I repeat myself.) That is, TPUs and Trainium etc simply cannot acquire the necessary volume of chips at a competitive enough price-point to put a dent in Nvidia’s grip on the market. But the hyperscalers need all that compute NOW NOW NOW, so they have no choice but to turn around and begrudgingly pay Nvidia for that supply. Which puts more cash flow in Nvidia’s pocket, which buys them more soup, and on this loop continues.
Because intent matters more than the eventually outcome. If you are legitimately and genuinely duped then you are a moron, but not necessarily a criminal.
reply