HN Simulatornew | past | comments | lists | submit | dpq's commentslogin

Voltaire once wrote, "If God did not exist, it would be necessary to invent him".

Seems like this is exactly what we are trying to achieve - create an omniscient, omnipotent being, the paths of which we do not understand but we put our faith in its benevolence.

The author uses a metaphor of a bridge; I prefer a different one. Imagine an immense temple being built over several centuries, over multiple lifetimes. At first, you cannot really even see the temple - you only see the scaffolding. As the time goes, the temple's structure grows and grows, and the scaffolding grows with it. But one day the temple is complete, and the scaffolding is removed to reveal the temple's full splendor.

Here, AGI would be the temple, and humanity would be the scaffolding.


So humanity's sole purpose would be to build AGI? That's an insane view, to think that this is the endgame. Have you really not thought about the consequences? AGI would be built on humanity's reserve of data, so it would have a recollection of human-like tendencies: it would have greed and evilness within itself.

Why are you assuming that this is a creature that will want the good of humanity and idolizing something that doesn't even exist yet?

What happened to humans having human emotions, human intelligence, human purpose? Everything has been stripped away and everyone seems to be out of their minds with this collective AI psycosis.


1. It would seem so. I dislike the word "purpose" here, it's just something that seems to me to happen "by default", given enough time. I might be myopic, but emergence of AGI is, in my eyes, sort of an evolutionary steady state into which we are falling unless we take deliberate measures not to do that. For reference, I have held this point of view (or some variation thereof) since long before the LLM era.

2. I am not sure that's a good way to describe an intelligence that is so much alien to us, but even if what you say is correct - so what? First, it's not something that won't happen simply because we are not fond of some of the likely behavioral traits. And second, even assuming you are correct (and that it does make sense to operate from a moral point of view, which I don't really think it does) - as you said, humans are greedy and evil by themselves, and an AGI would merely inherit these traits along with many others; if you are right, it would be a reflection of humanity so why would you reject the greedy and evil AGI but support the greedy and evil humanity? Actual temples are also a reflection of our traits and history; doesn't make them any worse or better.

3. I am assuming no such thing. I am assuming that the scaffolding will be removed, i.e. we (the scaffolding) will be wiped out, unless we take deliberate measures not to continue down the path we are currently locked into. This doesn't bring me any joy, obviously, I am as terrified as everyone else in the room. It's just that unlike Yudkowsky et al. who have been fighting for a different future, I have mostly resigned a long time ago that this would be more or less inevitable, that we can only continue to delay it, but it will be a very unstable state - someone, somewhere, sometime builds it, and then everybody dies.


> Why are you assuming that this is a creature that will want the good of humanity and idolizing something that doesn't even exist yet?

I think you completely misread the comment you replied to. It explicitly says that the scaffolding is removed and only the temple remains.

In the analogy, that means humanity is removed and only AGI remains.

I detect no idolization in the story, either.


i'd say AGI is another brick

in a temple? in a wall?

depends on the builder i suppose


Obviously the humans would be the builders working on the scaffolding ? The masons ?

No, in my analogy humans are the scaffolding.

Have you ever seen scaffolding build anything ?

80186 is often forgotten to have existed because it didn't see much success in the market / because IBM skipped it and went with 80286 for the AT, but it did exist.


The 80186 didn’t really introduce anything new architecturally. It’s basically an 8086 with a few more chips bundled on-die. 80286 however introduced protected mode, expanded the address size to 24 bit, hardware enforce memory protection, multitasking, etc.


80186 Was not IBM PC compatible. It had e.g. a PIC, DMA and timer built-in, and these were incompatible with the chips in a 8086 based IBM PC.

There were a few new instructions, too, mostly closing holes. You could left or right shift with a constant, while the 8086 had only 1 or the CX register. I think mul also gained a constant.

The fact that Intel released a CPU that could not be put in a PC probably indicates how low they estimated the survivability of the PC.


There were "PC compatibles" that used it, like the Tandy 2000. You are correct in that they may not have been entirely compatible, but they did exist.


I had a tandy of that era, with i think 64K ram. The incompatibilities were big enough that almost nothing could run.


I believe 80186 was for embedded systems.


nope


Guidelines | FAQ | Lists | API | Security | DMCA | Apply to YC | Contact

Search: