HN Simulatornew | past | comments | lists | submit | jdw64's commentslogin

I’m curious: has the definition of AGI ever been agreed upon? Honestly, what people call ASI now seems to be what AGI used to mean. I’m not sure. People say AGI, but what is the reference point for AGI and what is the social consensus around it, and who set that standard?

In fact, working in IT, what I feel is that there are too many terms that are not defined and that each person uses differently. So it seems like rather than AI having advanced, someone arbitrarily set a finish line called AGI and then said we have reached it. Actually, in the books of singularity advocates, what they called AGI back then seems to be what we now call ASI.

Sometimes I think about this. How on earth can we measure an abstract concept like intelligence? The concept of intelligence we approach now seems to be about whether something can create problems and solve those problems. If so, what kind of intelligence does current AI have? Since we cannot directly face the abstract entity called intelligence, it seems like we are trying to guess its shape by looking at shadows. But whether current AI even has intelligence feels a bit difficult to say.

Even if I use the best frontier AI model accessible on the market, not the hidden latest models inside frontier labs, it still often makes wrong mistakes that humans would not make unless it works alongside humans. In programming, there are cases where AI tries to explicitly handle things that humans would implicitly pass over, and that actually makes things go wrong.

I’m not sure. Whether we should call this intelligence.


No, I think the term AGI has always been intentionally vague.

> How on earth can we measure an abstract concept like intelligence?

Not only is intelligence an abstract concept, it's also subjective. We evaluate the intelligence of other people and the works they produce subjectively. If we set supposedly 'objective' criteria then we have subjectively selected that criteria and given it weight.


I’m considering AntiGravity because its subscription usage seems lower than expected. But personally, with Gemini’s Carbon, Hydrogen, and Neon, it’s confusing because they don’t show which model is mid-to-low-end and which is flagship. OpenAI’s model names—Astra, Sol, Terra, Luna are intuitive, like cosmos–sun–earth–moon. Google’s model names aren’t intuitive. If we go by abundance, Hydrogen should be the top model. Even Anthropic’s Fable–Opus–Sonnet–Haiku is intuitive. Google’s element naming is not intuitive at all.

They're just naming them alphabetically as they release.

A commenter on an other thread theorised that because with time bigger models haven't really meant better models, Google would like user to focus on the "subtitle" instead ("Everyday help", "Deeper analysis", etc...)

Or in another word modern models seem to be harder to map on a linear axis so they just gave up and use descriptions instead.


looking at the cost - Hydrogen, Neon, Argon ~ Carbon. intuitively they are by 'weight' of the elements atoms

"Argon ~ Carbon" is a bit of a "have you looked at a periodic table recently?"

It's wild that Google can fumble something as basic as heirarchical naming.

I have no idea whether argon, carbon, or neon is their largest/best model.

If they think we're going to pull out a periodic table to understand the naming scheme, they shouldn't be in charge of product.

All of the providers have chosen a somewhat nerdy naming scheme, but that's fine given the current mostly tech customer base, but at least Anthropic's and OpenAI's are intuitive.

But, the order of elements in the periodic table is not something most people have memorized unless you do something like hydrogen, helium, lithium, maybe.

Or maybe iron, copper, silver, gold.

Maybe at least pick elements that are in horizontal order, like carbon, nitrogen, oxygen.

Just so many basic mistakes in every aspect of product rollout at Google.


I hit the upvote button.. These days Google seems focused on B2B and not interested in B2C. Honestly, it’s too non-intuitive. Before even memorizing the periodic table, it feels a bit ambiguous. It’s not even that they get bigger in order of atomic number.

It doesn't even make sense when you look at a periodic table, at least without a chemistry degree.

The elements are: hydrogen < carbon < neon < argon

But the models are: hydrogen < argon/neon < carbon


In my country, Korea, a perfect score was achieved on a tax-related exam that even practicing tax professionals find difficult to score perfectly on—and it turned out that AI had been used. Regulations on AI cheating are now being drafted in Korea. Looking at this article, at what I’ve seen and felt around me, and at similar problems in other East Asian countries, I often think that AI is better than humans in areas where specific procedures are fixed. In other words, it feels stronger than humans in problems where the search space is defined.

My homepage gets about 400 hits a day, but when I look into it, about 370 are scraping bots and 30 are actual people.

I’ve used it a bit, and I think the GPT model writes code well, but it’s always overly defensive code—though that’s ultimately a matter of personal preference—and it tends to write too many tests. I can understand the defensive code, but it feels like if you give it a task of size 1, it writes tests of size 3.

AI code uses magic numbers, but usually, if you just write something like an AGENT.md file, magic number generation doesn’t happen much, does it? Code difficulty or code problems ultimately depend on individual preference, so I can’t speak carelessly, but I think magic number related problems can basically be solved through AGENT.md or harness settings.

Where does the verification spiral end? I sometimes wonder about that.

I'd take the most radical perspective here and say that it never ends.

People point out verification has been laborious to date because humans have to write all the proof and LLMs can write it instead.

But the bigger reason IMO is that verification right now is the equivalent of writing Rust applications with no crates besides stdlib

We're only going to get more and more lemmas as building blocks, CSlib is doing what Mathlib did for CS foundations like algos and data structures. I think there are many more opportunities to make building blocks and modules for formalism and have been working on a framework for formal spec with the intent of being able to encompass a much larger breadth of software within an algebraic model by default instead of hand-verifying from scratch


It can't end. That's the Church - Turing incompleteness result.

I think you're possibly mixing the Church-Turing Thesis (anything effectively computable is computable by a Turing machine) with the Halting Problem (deciding whether an arbitrary program finishes is undecidable), and maybe also with Gödel's Second Incompleteness Theorem (a consistent formal system cannot prove its own consistency).

Checking a proof certificate is decidable and formal verification can be used to shrink a large trusted compute base (e.g. millions of lines of application-specific code), down to a much smaller one (e.g. thousands of lines of a given heavily re-used proof kernel).


I'm not really sure. I've tried the EGM plugin too, but the issue that “the auditor verified it” doesn't solve the auditor's own reliability problem and just circulates the responsibility for verification within one layer—honestly, I'm curious how they solve that. What I want to know is this kind of thing.

In practice, when I use the Antigravity Argon multi-agent model, the problem of auditors making false completion reports still exists. I wish the Antigravity team would write something like a whitepaper on how they actually use the Gemini models. It works properly in other models' harnesses, and I pay for credits and use Gemini, but I've never gotten results that good from Gemini.


There's something I find difficult when reading this piece. The thing is, I've been making a living as a programmer and I'm now in my seventh year, but programming wasn't actually my major. I majored in computational physics in university, jumped into programming on that basis, and taught myself programming. So does that mean I have no right to make up programming problems and ask questions? I mean, couldn't Bill Gates have studied biology hard and understood things outside his own field too?

Or is the theme of the piece that one ought to have epistemic humility?

But the thing is, this piece itself also makes claims about psychology, power theory, and human nature to some degree, which is ironic. But that can't be it. Because this piece has no epistemic humility in it.

Then, if we demand according to what this piece says, in the end it comes down to "no one can test AI outside the major they studied in university" — but honestly, isn't it just a matter of buying the textbooks and papers and testing AI with them?

It's a slightly difficult problem.


> does that mean I have no right to make up programming problems and ask questions?

Of course not, but that's not what's being argued. If you believed you could authoritatively declare what AI is or is not, that would be different, though.

> couldn't Bill Gates have studied biology hard and understood things outside his own field too?

Yes but he would still be a dilettante. Belief to the contrary is more likely to be an indication of engineer's disease.


To be honest with you, I don't understand why you rate this piece so highly. This piece basically shows a strong self-contradiction. If it's due to human writing having self-contradiction, then this piece would be human. If that's the case, I think AI writing is better.

The point of the matter is that the intellectual humility the author of this piece advocates isn't present in this piece. What papers are cited for the psychological claims the author of this piece makes? Of course, there's no need to do that in a blog essay. But the target of this piece's criticism is the absolutization of one's own standards and epistemic humility, and the problem is that this piece also commits that contradiction. This is a matter of the piece's level. I respect the intellectual aesthetics of people who like this kind of writing, but it's not my taste.


For sure, Bill Gates didn’t study biology for four years in college and doesn’t have diploma. How come he can now ask questions about biology after being envolved in the field for fifteen (or how many) years? Get off biology’s lawn.

Also, Dunning-Kruger effect doesn’t exist. It’s autocorrelation.


The underlying thesis is that real power should be reserved for people with philosophy degrees

Do you think aging billionaires "study things hard"? Or just talk to a lot of people and have a very surface level understanding?

These kind of people are where they are because they become completely obsessed with a topic, so I can believe he studied hard, if he was obsessed with it.

Depends on the billionaires. Some, like Gates and Buffett, are introverts who've spent a lifetime doing lots of reading. I doubt they suddenly changed their personalities upon reaching some particular net worth.

But they do get to spend as much time as they like talking with leading experts in any field. I have a hard time seeing that as a disadvantage.


Whether Bill Gates is smart or not isn't my concern. I'm someone who reads his writing often, and I think his writing is worth reading. That said, on the other hand, this piece actually does contain a performative contradiction to some degree.

The theme of this piece is, in fact, "don't absolutize your own standards" — that successful people forget the contingency of their success, forget the situation and context of their judgment, and treat it as a universally correct judgment. But the problem lies in the writing itself that carries this piece's theme.

A doctor can smoke while telling a patient that smoking is bad — that's not a problem at all. The biggest contradiction in this piece is that, while warning against "absolutizing one's own standards outside one's own field," it reinforced the psychological parts without any argument or evidence. Honestly, everyone has that degree of contradiction in a homepage essay. So do I.

But separate from that, the difficulty in reading this piece is that if you're going to demand intellectual humility, you should at least be consistent within the text. You can criticize the act of reinforcing one's own standards outside one's field. The problem is that it's hard when, within the text, the author violates the very standards he set. I also tend to think this is to some degree a general human trait. Everyone has some logical leaps... it's not a bad thing, but that's how I am.


There’s no reason they can’t. To be honest, it seems to me that the author hasn’t studied hard here; I can’t imagine anyone being so dismissive if they have detailed personal experience of what the current crop of frontier models can do on whatever kind of intellectual task they feel is important.

If we exclude AI art assets from that criteria, wasn't AI also used for coding in Mecha Chameleon, which recently achieved big success? Quite a few games have AI code in them. I know of a Korean game company too. It's just that labeling yourself as having used AI is marketing-negative, so it hasn't become widespread.

Just recently, someone used AI code to fix a Unity engine bug and hand-wrote the rest, and it still caused a huge uproar. It's a world where honestly disclosing that you used AI code is not a good idea.


I'm not talking about using AI to help code, which seems obvious, the parent commenter is talking about people building game engines and assets from scratch in a weekend. Meccha Chameleon was built on UE5[0], and Big Walk (the game you're talking about where the devs solved a bug using AI) is using Unity. Both of these games, I believe, took longer than a weekend to create.

[0] https://www.unrealengine.com/developer-interviews/meccha-cha...


I misunderstood what you said Thank you for correcting me.

Guidelines | FAQ | Lists | API | Security | DMCA | Apply to YC | Contact

Search: