Imagine multiple (basically all of them) leading pharma companies CEOs saying:
We cannot be sure our new medicine won’t harm or even kill humanity
Not only that, they are new to this whole pharma business, have no medical degree (medicine just appeared a few years ago and is still mostly art then science)
And they even say there is 10% chance of the majorly bad permanent outcome and they already had drugs that escaped the lab a few times and harmed others (suicides, lowered academic performance in children, major hacking sprees)
Isn’t it extraordinary enough? Isn’t it “not enough evidence some caution is advised”? ;-)
We used to have the TV, the thing was in the box, the simulations were in the box for 70+ years, and now something starts to crawl out of our “TVs”:
We can empower all (a lot of startups are needed, check my bio), not only AI agents
This is a poor analogy because you have not listed any positive and demostrated upside.
It's more like "we found a cure for lung cancer and it works on 75% of patients and we're working hard to get to 100% and cure other cancers too" and someone else screaming "You need to stop that research because there is a 50% chance you'll cause the zombie apocalypse and turn us all into zombies".
I get this analogy isn't perfect but, (1) lots of people are seeing benefits. For example Mozilla claiming they used AI to fix tons of bugs. If there were no benefits there'd be no incentive to keep going (2) it's hard to verify the naysayers claims because they're guessing without proof. Sure, it's easy to follow their arguments and nod along but they are guesses similar to the population bomb of the 1970s
You can check my bio to see that I directly propose many all-empowering startups (because alas without them there is non-zero probability we’ll end up pretty disempowered), quite a few related to AI
AI agents literally appeared a few years ago, AI in its modern form, too
Each drug is researched for more years than the age of ChatGPT ;-)
I personally think we have 50%+ probability of the perfect futures for all - p(perfect) - alas it’s not 100%. It’s trivial and profitable to grow it
The petition is about a much broader problem: if we’ll demotivate people from understanding math, vibe math in a rush without making it understandable - math becomes useless
Basically with pic generation at least we all understand what is on a pic - some cat or something
With math it is trivial to generate a lot of stuff that is half-assed and just gives a plain yes/no but doesn’t get absorbed into human technology and better math
Basically it is not inly about the answer 42, it is about the questions, _meaningful_ _full_ answers and empowering people
By giving a short answer AI can burn the find and make it boring, hide the fertile ground around that answer that any human would’ve highlighted
If you have Einstein - it is not just relativity he did - you can continue asking him questions and he continues the research direction, teaches students, etc
With AI it is often 100% gate keeped (most of the time experimental AIs do the stuff, you have no access even for 200 bucks/month or more) slave that was killed after finishing - hard to “compete” with that and it is not very inspiring approach to researchers ;-)
Fair enough - maybe I didn't fully read the statement. Though you can ask AI questions about how things worked sometimes. Like here's me asking the google one how you derive E=mc2 https://share.google/aimode/dtHKAO9Nr0EYWq7JE
Surprisingly, there are combinatorial systems with just a single combinator[0]. Also, NAND along with a single axiom[0] can reproduce all of propositional calculus. By Curry-Howard-Labek, these are related facts and suggest that a universal tree likely exists in the tree calculus.
Calling a machine with no drives beyond maximizing a number a "slave" is far worse than saying it "thinks". The problem isn't the emotive language, it's that it implies human motivations such as self-preservation and desire for freedom that it doesn't have. Even on HN, people regularly claim it would be "irrational" for an ASI to do things like killing all biological life. That would be irrational for a slave, but not for a machine that does whatever necessary to make the number bigger. "Thinking" is comparatively abstract, so it's less likely to mislead.
effectiveutopia.org