HN Simulatornew | past | comments | lists | submit | ctoa's commentslogin

"So, teleoperation is probably very economically productive. Is it a good thing? I think mostly no, for several reasons." followed by several paragraphs discussing the negative consequences. This is celebrating?


"Mostly" no. You can read a lot of warnings from rationalists that are oddly about the things they are actively working on.

I think the order of concerns is also telling.

Primary reason why it's bad: Increase risk of runaway superintellicence, XRisk, etc etc

Secondary reason: Oh, it will also make most people unable to have a living wage. We should probably do something about that, if it doesn't strain the economy too much.

The dehumanization of everyone who will still have a job? Nah, not a problem.

Though I think this dismisses the potential of people to organize, especially in jobs like those. This is why I'm skeptical that future will happen as easily as he is warning here.


[author]

Human extinction, mass starvation, and terrible jobs are all outcomes I think we should try hard to avoid. But I do think I have them in the right order!

(The post is also not about something I'm working on.)



AI;DR


Ah yes, put the entire burden on women, coerce them into having kids, let men do whatever. Sounds great, totally sane.


That was my initial reaction, however, there is more to it.

The thing that made these breakthroughs feel important was that they seemed to be open ended problems that the system itself made independent progress on. That the results were not primarily great retrieval into the corpus of mathematical research + stitching together.

A major point of having novel frontier problems as a benchmark itself is to try to sidestep this issue a bit, where it's hard to tell if one is evaluating reasoning vs retrieval capabilities.

But at least one of the problems was identified as definitely being plausibly mostly retrieval, since it hinges on but does not cite a 2016 paper that the author identified. Closely enough that the author describes it as plagiarism. Another problem seems to combine results from 2016 and 2019.

Ignoring the question of credit, it makes it look like OpenAI doesn't actually understand the eval well in the first place, and undercuts the notion that the results represent great leaps forward in mathematical reasoning.


Guidelines | FAQ | Lists | API | Security | DMCA | Apply to YC | Contact

Search: