He has already IT experience then. Surely he learned something out of it. Securely using apps. Discretely handling deals, etc.
Anyway from their webpage:
> We work directly with heads of state, ministries, national cyber authorities, and the senior officials responsible for executing national strategy.
So I presume he has and provides contacts. Just the usual one hand washes the other. Nothing nefarious I'm sure.
LMAO, you mean he learned to install Signal and turn on disappearing messages after all of his chat messages, obviously implying corrupt political practices, were read in a special government committee?
The guy doesn't even know what DHCP is, "I'm sure".
I almost feel bad that company is getting traffic from HN. Kurz is such a disgusting little worm.
I think your test is already embedded into the models. You should search for new frontier tests to subject the models to. Maybe they should now try to unify the standard model and general relativity in physics. I'm pretty sure this is nowhere to be found in any training data nor shared in any chat between a scientist and a LLM ;)
Every new model you make this post, every time there’s someone who posits it might be trained on, and every time the answer is “maybe but probably not” it’s not a worthwhile conversation to have at this point, either the models can do some arbitrary thing or they can’t.
> every time there’s someone who posits it might be trained on, and every time the answer is “maybe but probably not”
The answer is “they are definitely in the training set”.
They all generate the same composition and style because this has become a self-reinforcing loop. They seem to reproduce a specific solution from memory instead of designing from scratch.
I'd turn that question around: why bother with the pelican tests at all?
Simon explained why he started them in the first place:
"I chose that because a) I like pelicans and b) I'm pretty sure there aren't any pelican on a bicycle SVG files floating around (yet) that might have already been sucked into the training data."
That's no longer true. Every new model gets a blog post with its pelican SVG in it, and that ends up on the web like everything else. Haiku 5.5 and Opus 5.5 now say "this is the classic pelican benchmark" in their reasoning traces. People keep pointing this out and it keeps getting dismissed.
I, for one, don't see the point anymore. The reason for running the test is gone. I just don't get why we still treat the results as meaningful.
I continue to do the test because I still learn something new from it every time.
This time, just seeing the difference between Haiku 4.5 (a year ago) and Haiku 5.5 (today - and 1/10th the cost) was worth it alone.
Same for Mistral the other day - the leap from Mistral Large 3 (their previous best model) to Mistral Large 4 was similar to the Haiku 4.5 to 5.5 jump.
I wrote some more thoughts about what value we can still get from the pelican test back in July - https://simonwillison.net/2026/Jul/16/kimi-k3/#what-can-we-l... but I've actually become MORE confident in its ongoing value since then. Using it to compare reasoning levels is proving particularly useful at the moment.
I get a good initial intuition about how much they are going to cost, how much the reasoning efforts affect their output (and their duration and cost), and how much they have changed since the previous release in the same model family.
You get intuiton on capability which is contaminated by your blog post being ingested to training corpus, wihout strict comparision analysis on not pelicans, not animals, not svgs you cannot state any general capability with your prompting
My only concern is that it might actually decrease the quality of the output. So many are outputting the pelican leg on the "wrong" side of the frame (along with the left pedal if you were facing same way as the pelican) it may be reinforcing future models to draw it that way.
The hn community is diverse. Of course some people will be tired of it but if enough people are engaging with the pelican test I would say that is likely because it still has some relevance.
Yeah, I haven't been seeing the point of this exercise for a long time (apart from driving traffic to Simon's blog if I wanted to be negative) and find it boring by now.
But apparently we are in the minority since the pelicans are always upvoted to the top, so if others have fun with it then whatever, fair enough.
I too will repeat the same comment that appears on 100% of these threads, and say that actually I love seeing the Pelican and that at this point it has become a lovely tradition.
The benchmark is not saturated. They are in the training set in a way where the models have heard of it, but not in a way where all the models get a perfect score on it.
Are you saying even these recent pelican pictures are not good? Vector graphics aren't meant to be photorealistic I don't think. For what they are they seem pretty good already to me. You can easily describe what is in the picture.
Some of you might be but some of us appreciate joyous absurdity and look forward to the pelicans with great pleasure and anticipation. Chacun à son goût.
Make this the new benchmark please. The prompt is funny, the result is even funnier. I think the great part is there isn’t even a right or subjective way to draw it.
So, just to confirm: you’re saying that it’s hallucinating “classic”, because you don’t think it’s in the training data, even though it correctly is a classic test?
Please show me your personal reasoning chain how you reached this conclusion because I think you’re hallucinating that the models are hallucinating the word “classic” here.
It's not a classic problem though. It's a very niche, novel benchmark that can't be more than 2-3 years old. That doesn't fit any definition of "classic"
I meant “made up” as in truly invented - this was years ago now, they’d use a boilerplate sometimes with “everyone’s dealt with ”, but see what you mean.
Maybe it's just a secret cabal of rogue agents downloading LibreOffice for no particualr reason whatsoever? I mean there is a precedent already: those fellas hijacked some poor wiki.
In many (most?) companies it also takes nontrivial amount of time to patch potentially affected systems. In my company half IT department got recently an email that we need to urgently patch all Windows servers because of some critical CVE. I checked it. The description was maybe 3 lines and it said this affected domain controllers only. I raised this with security team and they said that in this case we don't need to patch everything. Now if we had to do this, it would force us to spend nontrivial time on unplanned patching. That this in itself disorganizes our operations can not be ignored. So comparatively me doing some legwork and checking the CVE was trivial effort and saved a lot of busywork for the department as a whole. That time we got lucky.
If something is hard do it more often so you get good at it.
While this time you avoided something, it sounds like you need to figure out your patching process ASAP. There is every reason to believe a major issue will affect everything in the near future.
Later on he gives an example of Blackboard software that was very disliked. So I don't see what was unique in his perspective either. To me this article is not very focused and feels mostly like rambling. The core idea that we get accustomed to bugs/quality in software we use is sound. Just the article needs streamlining or better a complete rewrite.
reply