My effort to help scientists navigate the AI era:

On ChatGPT: Settings -> Data controls -> Turn off “Improve model for everyone”

On Claude: Settings -> Privacy -> Turn off “Help improve our AI models”

They can scrape your work after you publish it.

Will they even abide by these settings? They did not respect IP...

Highly doubt it. Remember how cursor would never save or train or customer’s sessions and then got acquired by SpaceX because of their large coding training dataset ?

ah man, I wanted this domain but I'm pleased to see it's at least being put to use instead of squatted

I agree with you, but to play devil’s advocate: what if it is on the same level as Amazon drone deliveries? Is intelligence really a bottleneck in society? People in MENSA with high IQs don’t often accomplish anything in proportion to their relative intelligence. The most intelligent people are rarely even in charge of making decisions. There are a lot of intelligent people with intelligent ideas that nobody listens to. It seems like the real bottleneck in society is building trust, consensus and constructive incentives between groups of people to do the things we already know are smart, not a lack of intelligence.

Interesting idea, but I'd have to disagree. MENSA is basically a group of benchmaxxed people, they are super good at a set of testing but that doesn't mean they have the ambition or other social skills to make a difference in the world. When you align all those factors, you find the greatest contributors to science and engineering. What happens if you completely eliminate the need for someone to be inherently intelligent? Suddenly, the number of important difference makers in the world increases by an order of magnitude.

I agree with the sentiment though, intelligence is not all you need. We're not going to "solve science" through RLVR, people are still needed in the loop until we get superhuman construction robots and superhuman communicators.

Good points. Maybe a more general version, removing constraints on one resource is usually not hugely impactful because you simply hit the next constraint.

Concrete example, "coding is free" (TBC, but let's assume), and now we're code review constrained. Even if code review becomes free we probably just become product discovery constrained, or something else. Making all software free will just cause us to hit on hardware constraints, and so on. It's still progress, and more progress will bring more benefits, but there's a limit to how much better the world as a whole can get from any one change like this.

MENSA selects for puzzle solving abilities.

Reinforcement learning trains task completion abilities.

MENSA selects for puzzle solving on the basis that those are correlated with "being smart".

Reinforcement learning selects for models that appear to complete tasks, on the basis that appearing to complete a task is correlated with completing a task.

Both have problems. The latter is probably a better correlation, but by no means flawless. If we just take at face value that RL is training models to complete tasks, we're not engaging deeply enough with the process enough to see where it falls short.

MENSA also selects for people who want someone to certify their intelligence. High iq people who have done a lot have much less desire for a MENSA membership. We should be looking at the non-MENSA members with high iqs and seeing what they're doing.

It wasn’t always like this, it changed in the last 10 ish years as the industry became more lucrative. Or maybe it’s been happening more slowly over a longer period of time, but I do think the earlier founders were more idealistic

> I do think the earlier founders were more idealistic

The original founders created YCombinator, the most prestigious startup incubator, and Hacker News used to be named Startup News.

What has it giveth? Or which one’s which?

It would be cool if there was a new kind of language we could use where there’s no ambiguity and we could define rules and procedures

you mean like those old programming languages of the days of yore?

things were so much easier back then, weren't they?

i was there, 3000 years ago...

Yeah, and you would also want to make it so that you get the same exact result every time, so that once you get the language right, you can be confident in the result. Shame that we never invented anything like that though, it would be super useful.

Enshittification is inevitable once a company takes VC funding

That's the correct answer, the rest is cope.

They won’t find anything because it’ll have rusted and disintegrated within a few hundred years

Monocrystalline silicon (silicon dioxide was my brain hiccup) seems to be sufficiently stable to last hundreds of thousands of years. [1] notices constant dissolution rate of monocrystalline silicon that will keep its rectangular shape recognizable for a long time.

[1] https://pmc.ncbi.nlm.nih.gov/articles/PMC5830089/ "The observation that hydrolysis of Si NMs proceeds by surface reactions only, in a predictable manner without diffusion through defects or pin-holes"

He did not say LLMs are not useful, he said they’re valuable for software engineering and could justify a hundred million or billion dollar valuation but not a trillion dollar valuation

This is latests refactor, in older videos he stated more than once that llms are basically crap (and at the time he said that he was probably right, but it was very short sight)

> As OpenAI bides its time, the company is engaging in early stage discussions with investors about a potential new funding round. The company could raise around $30 billion, CNBC previously reported, but that figure could change.

I predicted last month when they launched Luna that they had raised more funding and I suspect this tidbit dropped to CNBC is just prepping the public for a fundraising that’s already happened. There will probably be an announcement this month.