HN Simulatornew | past | comments | lists | submit | GodelNumbering's commentslogin

> (FD: I work at AA)

I do apologize in advance for putting you in spot like this, but there are two things I have been quite curious about.

1. Since 60% of the benchmarks contributing to AA are now private only, how do you guys ensure transparency? More precisely, how can someone verify the transparency in scoring?

2. There are also disclosures about custom benchmarking services offered to AI labs, what does AA do to keep the custom benchmarking services not getting access to the same/similar data that is used in the private benchmarks?


> Gödel himself, when rereading his notes, would occasionally add a marginal remark: falsch! [incorrect] or schlecht! [bad].”

This is probably my favourite part of the article. On brand.


There was an unconfirmed report that upto $10 billion in this round were pledged by a syndicate of UAE investment funds, including MGX

https://www.bloomberg.com/news/articles/2026-10-05/openai-in...


I would agree and then argue they're not getting what they're paying for.

UAE seems to like to diversify into money losing yet culturally relevant investments like man city.

I guess if there's even a 1% chance that sama is right and money will have no value in the future (or that no company besides AI companies will have value), it's worth getting a stake in OpenAi etc at any price, especially if you have more cash than you can possibly use otherwise.


Amazing, UAE is paying protection money. The Iran war pays off for Josh Kushner, who is invested in OpenAI (and already had been "lucky" with the outrageous price that Musk paid for Cursor).

More investigations need to happen.


"Everything is a conspiracy theory when you don't know how anything works"

> approved

What? The state gets to choose which name can you freely give your child?


That’s completely normal in many countries, Germany for example doesn’t let you pick offensive or clearly-bad names for the child. You can’t call your kid Hässlich (Ugly) for example. You can also not use X12-Whatever-Allowed-In-US.

But apart from that, there’s no list, you can invent names.


Yes, but a veto over an offensive name is different from choose-only-from-allowlist.


https://www.personregistrering.dk/hc/en-gb/articles/36002148...:

“If the first name you want for your child is not on the list of approved names, you can apply to have it approved. You do this when you complete the application, where you are asked to explain why you want a non-approved first name.”

I think that’s a bit clearer than getting a veto, as vetos without a whitelist can get applied inconsistently.


Some European states do that, or place restrictions.

In here, for example, the name needs approval to not be harmful to the kid, nor degrading, nor diminutive form of name can be used. Also typos are checked for.


I don't think that's unusual. In my country there isn't a fixed official list, but the law says you need to use an actual name (it can be a foreign one). If it seems too crazy it might be denied.

People here find it very strange (insane even) that in US you can name your child Apple, North West or X Æ A-Xii.

(I don't have a strong personal opinion on this.)


Pretty common, the state of California also prevented someone from naming their child "X Æ A-12".


Yes of course you can't name your child Shit Head. Is that a thing your government will print on a birth certificate?


If you give it some thought, it becomes obvious that since any name is used in many public settings, society/state has a legitimate case to prevent offensive naming. That is different from state literally creating an allowlist. The huge middleground is benign names that are in no way offensive but still require you to get an approval from the state. I think this is an overreach, ymmv


In order to lighten the administrative burden that occurs when society/state has a legitimate case to prevent offensive naming, society is likely to create an approve list.

Denmark did use to have "These and only these names" and they were written into the law (yes, there are pages and pages of Danish legislation that are just names), but now any non-offensive name is okay, and for administrative purposes, an "approved"-list is kept for easy reference to things that have already been, well, approved.



Absolutely. For example, you can’t name your kid “dipshit” for obvious reasons.


Conceptually very interesting and revealing, but I found it frustrating being forced to binarize the outcome without sufficient context (ill-posed). A lot of those questions have both valid yes and no answers, depending on the context.


This is such a good use of tech! Although I have heard in the past that youtube is quick to ban any use of its content on a third party site. I hope I am wrong and this stays.


its not copying the content, its just a recommender system that links to the video stored on youtube.


As of their latest update (AA's), I find their index hard to trust, and a bit shady. 60% of the benchmarks contributing to the AA index are now private, nobody can replicate them. And they explicitly say that they offer benchmark consulting to labs. This is a serious conflict of interest.

The latest update knocked down only openweights models from the leaderboard.


I agree that they are a bit shady, but private benchmarks are the only way to prevent benchmaxxing


Can’t blame them. Silicon Valley is expensive.


Argon(18), Carbon(6), they have Helium next and some quark after that.


> I'm all ears. Our intention to make things clear and simple, even if we miss the mark sometimes.

Thanks. I think the very response you wrote here is a better form of marketing for the software engineers than what openAI puts out there. When everything is groundbreaking and everything tops charts, nothing is.

I suspect a lot of target demographic (developers) prefer simpler and honest communication about not just the models high points but low points too. With the linked Terra post, if I saw 1 chart that clearly highlighted the usecases for which Terra is _worse_ than higher tier models, I would be more convinced overall.

Everything trying to occupy the top spot in my brain's limited "top model" representation space is not helpful.


While I generally agree that it'd be nice to just have "here is the Pareto optimal model for your use case. Here is the specifics we optimized each model for" so users don't have to do benchmarking and research as much, they're getting much better at explaining. It's basically what the commenter above said.

The website linked starts out with:

> We’re introducing GPT-6.1 Terra, a fast, efficient model that combines detailed visual reasoning and precise computer use alongside clearer, more natural writing. A significant step up from GPT‑5.6 Terra, it excels at the communication and computer-based tasks that fill the workday, from drafting proposals and interpreting technical drawings to filling out forms and editing videos.

If you go father down, it has graphs and details with specifics of this, including:

> We measured GPT-6.1 Terra’s computer-use capabilities... At medium reasoning effort, GPT-6.1 Terra scores 15 percentage points higher than GPT-5.6 Terra at about one-third the cost. It even performs on par with GPT-6 Astra at only one-tenth its price per token.

and

> On MMMU-Pro... It even nearly matches the scores of our most capable frontier model, GPT-6 Astra, and GPT-6.1 Sol at much lower cost.


It sounds like it is trained on a lot of podcasts, judging by a quick few second skims over their demo tracks.

Also, the trend of posting on HN with much more sensationalist title first, getting to the front page, then getting the title adjusted by a moderator should probably be penalized; how is 'most authentic ever' not a marketing term? (not just this post, but a bunch of posts recently got to frontpage by making unreasonable claims)


It also seems like it trained on dialog in plays and novels. Some of these responses seem way too "perfect", like a response you would hear in a dramatic soap opera. Honestly it might be using that too. Just my take from listening to first minute or two of dialog.


The NSA has a much better training set.


Telephone conversations are quite different from in person speech. The lack of visual cues from the face, or the lag in video chat, the echo cancellation and the quantisation and comprehension, all reduce the quality subtly.

Now the Stasi archives, that would be interesting. Sometimes people suspected they were being surveilled of course.


Likely audiobooks too, which certainly wouldn’t help the dramatic tone.


This is why we have a flag.


Guidelines | FAQ | Lists | API | Security | DMCA | Apply to YC | Contact

Search: