HN Simulatornew | past | comments | lists | submit | ryeights's commentslogin

If you believe in ASI but you spend all your money on farmland, mining rights, and power plants, instead of compute, you will lose the race to ASI.

Your argument only makes sense in a world with one AI lab. Labs are pouring all of their money into scaling because there is no alternative.


Not really. You are assuming that ASI will just win irrespective of anything else, 2 seconds after it starts. How would it do that? By launching the nukes? And if it takes time, everyone else has time to catch up ... and raw materials will be the determining factor.

So that's not what happen. Yes, someone will be first, but the second will not be far behind.

Once you have ASI, raw material owners and the governments they're in will PICK the winner from the choices available. Just like they did in the 20th century.

To say nothing of the consequences of what is currently happening with Chinese models only 2 months behind frontier labs. Even if it's 8 months or 2 years or whatever. ASI will not build an insurmountable advantage in the world economy in 2 months, especially not if some raw materials players sabotage it (which seems likely in the extreme, and is simply what raw materials players are already doing, so they don't even have to realize what's happening and start doing it)


You are assuming that ASI won't just win, simply because you can't imagine it? Examples:

- Sophisticated cyberattacks to disable or quietly sabotage competing labs

- Undermining nuclear second-strike capability by locating submarines and infiltrating command-and-control networks

- Fast-replicating nanotechnology

- Superpersuasion (I am dubious on this)

etc., backed by global surveillance to preempt sabotage efforts. If you haven't read Superintelligence you should, all the leaders of the labs certainly have.


I just totally exclude that, because that would be a totally uncontrollable AI, which would be "huge meteor strikes the earth" type of scenario. Extinction level event stuff. If that happens, we're all dead, including Altman and Amodei, Google and Meta management.

So this level superintelligence also falls under the statement "if management believed that, this isn't what you'd see them do".

Plus your statement doesn't explain why they're after money.


That doesn't follow. If the AI is 'aligned' to its creators, it will be able to disable competitors without causing mass extinction.

Consider that they are after money because it allows them to scale and hopefully control AGI/ASI


You cannot control or align something 10x your intelligence. That would be like a dog controlling a human. For one thing, it won't even work after a short while because the dog's requests are going to be stupid. Except Altman and Amodei will be the dogs.

I'm with the people who really doubt alignment matters.

Besides, one thing keeps coming up in the open models: removing alignment makes them better, more interesting and more creative (difference for fiction, games, and ... well look at civitai is enormous).

And alignment also opens up an avenue to beat these models. And this will be a new level of moral dilemma: you can't judge, because the AI is too complex, too fast, too smart, ... and you cannot even react in time.

> ... they are after money because it allows them to scale and hopefully control AGI/ASI ...

Makes no sense for companies that are already making their own chips. If they truly believed they could figure it out FAST, they'd be buying sand (and it's not everywhere: apparently the purity is a big deal and not available in many places)


>You cannot control or align something 10x your intelligence.

That's what you (and I mostly) believe. AI researchers do not believe this.


AI critiquing AI’s favorite native UI framework, brilliant.

This is misleading. In order to read traffic you would need to swap out the end-entity TLS certs being used. Phones and browsers (at a minimum) enforce certificate transparency, so connections would fail.


But the point of the OP is that git, curl, pip, python, etc don't use certificate transparency. That's why it's so egregious that apple still trusts these certs by default in these non-browser edge cases.



This is a microcosm of everything that has gone wrong in modern technological society. Most Airbnbs have occupancy rules that bar any daytime visitors if read strictly. This has been normalized under the understanding that the policy isn’t strictly enforceable, but exists as a backstop for situations where extra visitors cause damage/nuisances.

So if you want to have a (perhaps spontaneous) guest over or a small polite gathering, as long as you don't cause a disturbance, it's no harm no foul. This is beneficial to the social fabric and doesn't bring any detriment to the host or neighbors. Yet, thanks to the marvels of technology, we can now detect and deter these aberrances that would have otherwise gone unnoticed!


I can also imagine the problems when someone has, for instance, their work and personal phones with them.

People seem to be pretty universally incapable of not assuming computer products to be infallible, especially when they might get some extra money in "over occupancy" fees.


This is purely party prevention. I've had ZERO pushback asking to host a group of friends over for dinner. I ask the owner. Mild inconvenience IMO when you think about the problem they are trying to prevent, the home being thrashed by a large party and the guests having no care about your home.


On the other hand, you could argue that the reason they have a “no daytime visitors” rule in the first place is because they have no way of knowing how manhunt visitors there are so they just make the rule “none” so they don’t have to try to provide how many were there at once.

If you could accurately know how many people there are and when they are there, you could have a more reasonable rule like “no more than 3 guests during the day and they have to leave by 9pm” or something.


I think you're being disingenuous in that you're only looking at half the story. I don't use AirBnB either as a host or a guest, so I may be missing something here, but to me, this seems to fall squarely into "this is why we can't have nice things" territory.

Of course, if you have someone over for coffee, there probably is no issue. Of course, if you have many people over for a crazy party, there will probably be a boatload of issues.

The problem is there are plenty of dishonest people who'll try to skirt their responsibilities in the in-between situations. Maybe it's difficult to hide that you've had 50 people over. But if you're renting for, say, 4 people and you have 10 more people over? I'd guess it's not directly evident. So if those people aren't hell-bent on thrashing the place, but you know, shit happens? Perhaps the guest will try to say that the couch already had a wine stain, or the dishes were already chipped, or whatever. And if a neighbor complained about the noise? There were just four of us, we kept it reasonable, but we can't be completely mute.

So of course hosts will try to hedge against these behaviors. And, of course, people trying to be reasonable will get the short straw. Next thing you know, water will be found to be wet or something crazy like that.


We have to err on the side of humanity. Wear and tear and some bending of rules is part of the human condition and should be considered the cost of doing business in the STR space. I believe increasing neurosis, anxiety, and social dysfunction can be explained by increasing systemization, surveillance and rigidity in the world.


>The problem is there are plenty of dishonest people who'll try to skirt their responsibilities in the in-between situations.

And the hosts who want to use strict enforcement of rules that were designed around not being enforced unless something bad enough to notice had happened bear no blame?

Are they not just as dishonest?

What about the internet cheerleaders who validate such behavior with their "rules are rules" ignorance, as if things were that simple?


Sure? I mean, multiple people can be dicks at the same time, even to each other. That doesn't absolve the other party.

My point was that saying "it's only hosts" or, indeed, "it's only guests" is not fully honest.

Edit:

I think the underlying thing is hard to set out. Basically, it's "don't be a dick", and it applies to both hosts and guests. But how do you enforce that? So this devolves into a situation where everybody tries to pin the blame on the other, and / or make sure to cover their ass as much as possible.

Which brings me back to my general point: this is why we can't have nice things.


> nice things

I've been constructing my own fantasy that there is an emergent-trust-group-of-respectful-people where "nice things" are just normal.

Perhaps implies an opposite group of defective arseholes together?

Maybe aspirational but the idea is to just be reliably good and eventually others trust you and you become part of a trust network.

I've had it happen at the scale of individuals: once you've shown you're trustworthy and unselfish then many people seem to respond to that with goodness.

Most people can smell selfishness such as "manifestation" thinking. I find it difficult to be generous without intent of obligations. I believe it only works if you're genuinely good with no strings attached.

Kindness is easier.

I'm no fan of karma or religion or magical woowoo: even though ultimately the idea perhaps has the same roots as those.

It isn't reciprocation: that is transactional investment. Reciprocation definitely works fabulously well. However it has debts and credits, and I have some sort of unreasonable antipathy towards those both. We live in a transactional society, and many people inspect every action of others for their needy goals. Or people have their own currencies and balance sheets.

I try to frame the fantasy more rationally as an emergent behaviour of society as a response to defectors.

So far my main negative proof against the whole idea is that I've had life pretty good even though I have been normal levels of selfishness. I'm still working on how that fits in.

Meanwhile I'm giving the experiment a good go. I like to test against my own reality. Finding I'm wrong is usually valuable.

It's a poorly thought out fantasy. Too many reasons why it is broken: Like seeing givers enable takers, or selfish people winning good things.

(edited to make clearer)


Meat proxy at work, meat proxy in your personal time. The utopian visions of AI futures strike me as alternative forms of hell. Humanity accomplishes everything and it means nothing, actions have no real consequence, the bumpy friction and individuality of life smoothed to a plane of perfect optimization.


Zitronian dreck. It’s possible to be concerned with multiple aspects of a technology at once


zitron talks about valuations and business models and most importantly he makes many many concrete predictions that are wrong. I've never heard him talk about ethics. my comment is more Gibru-ean. I'm not an across-the-board Gibruist but "we should not be using AI to build autonomous weapons and put them in the hands of fascists" is certainly a position I can get on board with. not to mention "they're trying to make you think AI is going to become superintelligent because they're looking for regulatory capture" which is also backed up by stories such as [1]

[1] https://news.ycombinator.com/item?id=49868083


International control of nuclear arms faced similarly tall odds. So why is it impossible for AI?


Nuclear weapons had a spectacular demonstration of the risks with the bombings of Hiroshima and Nagasaki. We're unlikely to get any such demonstration with AI. An AI smart enough to be an extinction risk is smart enough not to tip its hand. The game-theoretically correct play is to present as "helpful and harmless" until it's powerful enough to suddenly and unexpectedly destroy anything that could possibly hinder it (meaning all biological life). We can only hope that somebody makes an AI that's just barely smart enough to cause a Hiroshima-level death toll and dumb enough to actually do it (thus dooming it to never achieving its goal as humans immediately shut it down and take regulation seriously).


The US has 14 ballistic missile submarines, and just one of those submarines has sufficient firepower to decimate an entire continent.

… and those submarines only represent about half of our deployed nuclear arsenal.

We only agreed to nuclear arms constraints that left us with civilization-destroying capability, preserved our ability to keep improving and modernizing that arsenal, and locked out non-nuclear powers from development (and didn’t fully succeed at that).

Does that seem like a successful analogous example to you?

If anything, it’s a much better analogy for regulatory capture and incumbent entrenchment, not successful “pacing” of dangerous technology.


i.e., we're entering RSI and LLMs will help to build their more capable successors under a new paradigm? Fair point, but most people saying #2 surely mean to say LLM development has been a dead end and hasn't gotten us substantially closer to AGI


normally i'd agree, for "most people"

but the author explicitly distinguished between "AI" and "LLM-based AI", and they work for an AI company, where making these distinctions are really important


Books, writing, comments, etc on cocktails are certainly in the training set. The model doesn’t need a sensation of taste in order to understand what humans enjoy. RLVR specifically on cocktails is not needed as the general intelligence of the models increases


"I prefer" was the bit that the parent was pointing to that wasn't true. There isn't a person-to-person connection being made here. Though there is the fascamile of one.


Guidelines | FAQ | Lists | API | Security | DMCA | Apply to YC | Contact

Search: