HN Simulatornew | past | comments | lists | submit | rotis's commentslogin

If The UNIX-HATERS Handbook should ever get updated, this deservers its own chapter.

He has already IT experience then. Surely he learned something out of it. Securely using apps. Discretely handling deals, etc.

Anyway from their webpage: > We work directly with heads of state, ministries, national cyber authorities, and the senior officials responsible for executing national strategy.

So I presume he has and provides contacts. Just the usual one hand washes the other. Nothing nefarious I'm sure.


LMAO, you mean he learned to install Signal and turn on disappearing messages after all of his chat messages, obviously implying corrupt political practices, were read in a special government committee? The guy doesn't even know what DHCP is, "I'm sure". I almost feel bad that company is getting traffic from HN. Kurz is such a disgusting little worm.


Matryoshka you say?


thinking_effort: max Reasoning trace: This is the classic pelican-on-bicycle SVG test.

Opus 5.5 had similar response on max: This is a classic test request

https://tools.simonwillison.net/markdown-svg-renderer?url=ht...

I think your test is already embedded into the models. You should search for new frontier tests to subject the models to. Maybe they should now try to unify the standard model and general relativity in physics. I'm pretty sure this is nowhere to be found in any training data nor shared in any chat between a scientist and a LLM ;)


The fact that they've heard of the test doesn't seem to help them draw a good picture of a pelican riding a bicycle.

That said... here's "Generate an SVG of an armadillo in fishnet tights jaywalking on Mars" on xhigh for comparison: https://tools.simonwillison.net/markdown-svg-renderer?url=ht... (and here's the same thing from other models: https://tools.simonwillison.net/markdown-svg-renderer?url=ht...)


Why even respond to these comments anymore?

Every new model you make this post, every time there’s someone who posits it might be trained on, and every time the answer is “maybe but probably not” it’s not a worthwhile conversation to have at this point, either the models can do some arbitrary thing or they can’t.


> every time there’s someone who posits it might be trained on, and every time the answer is “maybe but probably not”

The answer is “they are definitely in the training set”.

They all generate the same composition and style because this has become a self-reinforcing loop. They seem to reproduce a specific solution from memory instead of designing from scratch.


I'd turn that question around: why bother with the pelican tests at all? Simon explained why he started them in the first place:

"I chose that because a) I like pelicans and b) I'm pretty sure there aren't any pelican on a bicycle SVG files floating around (yet) that might have already been sucked into the training data."

That's no longer true. Every new model gets a blog post with its pelican SVG in it, and that ends up on the web like everything else. Haiku 5.5 and Opus 5.5 now say "this is the classic pelican benchmark" in their reasoning traces. People keep pointing this out and it keeps getting dismissed.

I, for one, don't see the point anymore. The reason for running the test is gone. I just don't get why we still treat the results as meaningful.

https://simonwillison.net/2024/Oct/25/pelicans-on-a-bicycle


I continue to do the test because I still learn something new from it every time.

This time, just seeing the difference between Haiku 4.5 (a year ago) and Haiku 5.5 (today - and 1/10th the cost) was worth it alone.

Same for Mistral the other day - the leap from Mistral Large 3 (their previous best model) to Mistral Large 4 was similar to the Haiku 4.5 to 5.5 jump.

I wrote some more thoughts about what value we can still get from the pelican test back in July - https://simonwillison.net/2026/Jul/16/kimi-k3/#what-can-we-l... but I've actually become MORE confident in its ongoing value since then. Using it to compare reasoning levels is proving particularly useful at the moment.


> because I still learn something new from it every time

Can you share in what ways these learnings affect your decisions or behaviors?


I get a good initial intuition about how much they are going to cost, how much the reasoning efforts affect their output (and their duration and cost), and how much they have changed since the previous release in the same model family.


You get intuiton on capability which is contaminated by your blog post being ingested to training corpus, wihout strict comparision analysis on not pelicans, not animals, not svgs you cannot state any general capability with your prompting


My only concern is that it might actually decrease the quality of the output. So many are outputting the pelican leg on the "wrong" side of the frame (along with the left pedal if you were facing same way as the pelican) it may be reinforcing future models to draw it that way.


You are not alone.

The hn community is diverse. Of course some people will be tired of it but if enough people are engaging with the pelican test I would say that is likely because it still has some relevance.


Yeah, I haven't been seeing the point of this exercise for a long time (apart from driving traffic to Simon's blog if I wanted to be negative) and find it boring by now.

But apparently we are in the minority since the pelicans are always upvoted to the top, so if others have fun with it then whatever, fair enough.


I too will repeat the same comment that appears on 100% of these threads, and say that actually I love seeing the Pelican and that at this point it has become a lovely tradition.


The benchmark is not saturated. They are in the training set in a way where the models have heard of it, but not in a way where all the models get a perfect score on it.


It’s just fun.


Why even respond with a great new test? :)


Are you saying even these recent pelican pictures are not good? Vector graphics aren't meant to be photorealistic I don't think. For what they are they seem pretty good already to me. You can easily describe what is in the picture.


Yeah, all of the Opus 5.5 ones are totally fine and usable for vectors.


Whoah!!! gemini/gemini-3.8-flash is what I've been hoping to eventually see with the pelicans!


Maybe they're as sick of it as some of us are?


Some of you might be but some of us appreciate joyous absurdity and look forward to the pelicans with great pleasure and anticipation. Chacun à son goût.


Just skip it and move on I think there are more people sick of the people saying they are sick of it


Fellow pelican here. Since you host on CF, maybe you can turn on their AI Crawler Block Feature? Maybe helps a little.


LOL, I love that it inferred that this particular armadillo would be pretty smug


Make this the new benchmark please. The prompt is funny, the result is even funnier. I think the great part is there isn’t even a right or subjective way to draw it.


Frog on a unicycle!


Lol... can't believe this armadillo thing caught on.


LLMs hallucinate and pander. Calling something "classic" is a common LLMism


So, just to confirm: you’re saying that it’s hallucinating “classic”, because you don’t think it’s in the training data, even though it correctly is a classic test?

Please show me your personal reasoning chain how you reached this conclusion because I think you’re hallucinating that the models are hallucinating the word “classic” here.


You're making a lot of assumptions.


I think it’s quite a stretch to say that when a model correctly calls out something to be a classic problem, that it’s probably hallucinating.

That’s quite a big assumption from your end as well.


It's not a classic problem though. It's a very niche, novel benchmark that can't be more than 2-3 years old. That doesn't fit any definition of "classic"


It will be two years old on the 25th of October.

I think it's credible to call it a "classic" though, in a field that moves this fast. Name another comedy benchmark for LLMs with the same traction.


Turing test


right? "im having this niche problem i find no information on anywhere, any ideas?" and you get "ahh this is the classic "


Or “of course, super niche thing you made up is a problem everyone’s had” - they used to do that but not sure if anymore.


There's 8 billion people on this planet, and you're probably failing to imagine something sufficiently niche to be not actually very common anyway.


I meant “made up” as in truly invented - this was years ago now, they’d use a boilerplate sometimes with “everyone’s dealt with ”, but see what you mean.


Okay, but then "years ago" in AI is a different era entirely.


not all 8B have computer access, and even less access the computer as a device they control. Our crowd is actually rather small.


Pelican on a tricycle?

Two pelicans on a tandem?


Pelican writing on a forum about bicycles


About pelicans on bicycles.

And then just and recursive loop


Azerbaijan is a former soviet republic. So them playing role of a proxy for Russia or stealing secrets to sell to Russians makes sense to me.


Maybe it's just a secret cabal of rogue agents downloading LibreOffice for no particualr reason whatsoever? I mean there is a precedent already: those fellas hijacked some poor wiki.


In many (most?) companies it also takes nontrivial amount of time to patch potentially affected systems. In my company half IT department got recently an email that we need to urgently patch all Windows servers because of some critical CVE. I checked it. The description was maybe 3 lines and it said this affected domain controllers only. I raised this with security team and they said that in this case we don't need to patch everything. Now if we had to do this, it would force us to spend nontrivial time on unplanned patching. That this in itself disorganizes our operations can not be ignored. So comparatively me doing some legwork and checking the CVE was trivial effort and saved a lot of busywork for the department as a whole. That time we got lucky.


If something is hard do it more often so you get good at it.

While this time you avoided something, it sounds like you need to figure out your patching process ASAP. There is every reason to believe a major issue will affect everything in the near future.


Later on he gives an example of Blackboard software that was very disliked. So I don't see what was unique in his perspective either. To me this article is not very focused and feels mostly like rambling. The core idea that we get accustomed to bugs/quality in software we use is sound. Just the article needs streamlining or better a complete rewrite.


This site's trustworthiness when it goes for pre-aquisition data was disputed here already:

https://news.ycombinator.com/item?id=48086786


Thanks for sharing this, had no idea. This information changes my perspective a lot.


Reminds me of this story from Windows land by Bruce Dawson (and similar on his blog):

63 Cores Blocked by Seven Instructions: https://randomascii.wordpress.com/2019/10/20/63-cores-blocke...

Looks like Linux isn't immune to these types of problems.


Guidelines | FAQ | Lists | API | Security | DMCA | Apply to YC | Contact

Search: