I think lego is reaching levels of absurdity that I didn't know were possible. I get that you have to target another demographic with prices like these, but I'd love to hear why. Does no kid play with lego anymore? Are these prices justified or are they now a luxury type of brand for collectors? What is the deal with lego these days?
You seem to be missing some critical context, which is that this set is coming from a site where fans can just propose new sets and if they get enough votes LEGO will consider making an official version. So yes, this is largely a set for adults and the price is set as such.
Yeah, a couple of months after I graduated in 2015, it was dead, 5 years after that, same as profile images everywhere, nothing changed, immediately abandoned after the forced school interactions lmao
I go off of marketing obviously, but how is every new announcement of any vendor always on top of all others and then cheaper, while next day the next best and even cheaper model pops up? Is there anyone who independently verifies this? Its kind of pointless to comment on this announcement, much like how its pointless to comment on all the other announcements...
There are hundreds of benchmarks (all noisy), dozens of model families, and dozens of modes and variants within each. Announcements are crafted by publicity teams and involve cherry picking a chart that looks compelling.
As a prospective user, none of of the benchmarks details will matter to you, but the family of models and settings they peg themselves against can give you a hint of where you might want to evaluate them within your own workflows.
As an enthusiast or lay researcher, the more interesting thing is to look at their techniques and any innovations they're exploring in training, inference, etc. Those are usually going to be details tucked below the fold and won't be summarized in a benchmark chart.
As anybody else, why do you even care about this in the first place?
This doesn't just apply to AI. You should know better than to take marketing announcements at face value, nor trust that most "influencers" or "journalists" or "commenter" repeating and discussing those announcements are doing a lot of work as they buzz over what's said.
Don't be a victim to PR spin, but also don't discount potentially interesting work just because it got channeled through a marketing team. Just look for the information you need amidst the noise or move on if there's none there.
Because they selectively choose which benchmarks to highlight and which competing models to compare against; they don't always compare against the SOTA or the Pareto frontier of performance/price, a lot of times you'll see them compare against the last generation of competing models, or the like, so that they look better if you don't carefully consider what they're comparing against.
For example, here you see they are comparing against Opus 5 and GPT-5.6 Sol, when Opus 5.5, Sonnet 5.5, GPT-6 Astra, and GPT-6.1 Sol would be the latest SOTA models from those labs to compare against.
Also because a lab's own benchmarks tend to run about 6.4% better than third party runs of the same benchmark. Source: https://whatmodel.sh/claims This can happen due to a number of factors, and doesn't necessarily indicate anything nefarious. It could just be that they run multiple times while fixing issues that aren't related to the model, and have a selection bias towards stopping when the score is on the higher side. But since you see that sometimes their own scores are lower than the independent runs, some of the variation is explained just by variance in the model solving problems, since LLMs are nondeterministic there can be variation from run to run and you don't see labs always cheating, just potentially a bit of selection bias about when they stop tweaking their benchmark setup and publish the results.
And yes, there are independent third party runs of benchmarks for just this reason. The most popular with broad coverage of a range of benchmarks is https://artificialanalysis.ai/ but there are a number of others as well.
Being pedantic - LLMs are deterministic software, that is chaotic and typically deployed with a PRNG seed. An LLM ran using the same seed will produce the same output consistently (theoretically - in practice this isn’t always the case because of changing factors wrt the execution environment given how sensitive/chaotic models are).
In many cases they are not deterministic even with a deterministic seed because many runtimes embrace races that interact with floating point rounding to improve latency.
in practice, as actually used, LLMs are nondeterministic.
It is possible, if you control the execution environment sufficiently, and are careful not to use certain kinds of parallelism which can lead to nondeterminism, and you set the seed deterministically and ensure that the prompts are byte identical, you can operate them deterministically.
But no one actually does all of that. Almost everyone uses cloud hosted APIs where the provider could change any number of variables at any time. Everyone wants faster cheaper inference so uses inference engines which may trade off a certain amount of determinism for more speed. Different versions of harnesses and chat templates and benchmark software may introduce small differences in text.
And even though the cloud providers have in the past offered seed and system fingerprints to try to give people a way to control or detect non-determinism (see https://developers.openai.com/cookbook/examples/reproducible...), that has all been deprecated and removed in the newer APIs.
So yeah, while in theory it's possible that if you control the whole stack you can get deterministic results, that's not how anyone is using it, and there is no way for third party benchmarks to ensure deterministic results of hosted models, the latest APIs don't even support it at all.
This whole "well, actually, LLMs are deterministic if you hold them right" is a red herring. In any actual real-world use, they are basically always non-deterministic.
And of course, there are reasons that providers have dropped these attempts at providing deterministic results; because the results are chaotic, perturbed by small changes in input, it's almost never useful to try to make them act deterministic or detect when non-determinism has happend. Instead, it's better to think about them non-deterministically, and look at the distribution of results, rather than just considering a single pass or fail on a single test to be meaningful.
With the old GPT-3 api in the olden days, you could set the seed and get deterministic outputs! Interesting results could be drawn from it, as for instance if you use the prompts "a blue bird"/"a blue bird in the sky" the blue bird would have the same look in both with similar seeds.
Fair. I wasn’t advocating for “holding them right”, I feel it’s an important nuance to keep in mind when conceptualizing them, i.e. it’s procedural, determistic, yet both inherently and in practice extremely chaotic and therefore not practically deterministic software. So, in essence - software.
You have to read between the lines (or wait for someone on HN to do it for you).
It's never "cheaper and faster and more intelligent", it's pick 2. Sometimes (as was the case this month), it was "cheaper per token, but reasons five times as much", so your costs go up.
I run a benchmark and independently check all new models. They are very spiky.
If you are doing coding only then official benchmarks are mostly right, but as many people are starting to do for instance marketing work with LLMs now there’s very little correlation between coding and creativity for instance.
I've got a corollary question, which is "how big is the market for these never ending model announcements?" ... I guess its mostly people coding with agents. Most general AI users are just asking ChatGpt to word an email for them or whatever. How many AI users are on here looking for the next model announcement and working out if the price per token is better? Are we talking like 50,000 developers? More? Less?
Openrouter does >600 trillion tokens a month and the top models shift on a daily basis. A lot of people are paying attention to new releases and even pre-releases (which tend to be free)
It's always cheaper (*than the next more expensive model) and better (*than the next worse model).
And often they also don't mention how verbose it is, that the same text splits into more tokens than in competing models, how slow it is, that the pricing is temporary, that the cost per task actually isn't that good despite being both cheap and capable.
I have this idea that the big labs internally have much more capable models which are prohibitively expensive to run though, and the competition is mainly around getting the price down by releasing models that are quantized/otherwise made more efficient at the cost of a lot of performance.
I'm on the wrong site for this, but isn't it interesting how we don't ask what is the most interesting field I can join, but rather ... how do I stay employed with this for the next 44 years? Seems to me we missed the entire point and just accept it as reality and that it has to be this way. I'm not asking this to be snarky or smart, I'm just genuinely curious why everything seemed easier back in the day.
Not every field has an equal number of vacancies. There are a few people who study Art History and get a job as an art gallery curator, etc. There are many more who end up as baristas.
It's fine to follow your dream, but just make sure that your dream wants you too.
It also helps to have rich parents, if you grow up poor you understand the consequences of choosing the wrong career path can be devastating. Some people have the freedom to experiment and still get a soft landing, other people need to get it right the first time.
If AI came about 40 years ago, it would have been much more restricted and advanced much much slower.
But politics does not care for the average worker anymore. Every democratic country is slowly turning geriatric, so due to how democracy works, the will of the OLD is what runs societies. And job security is not one of those things, when the biggest voting block does not even work.
I have to say, Epics vibe is just malicious and toxic. Steam is silent and just lets things speak for themselves. What is the value of an idea if it does not work in the real world?
I just wish the people at Epic collectively took some vacation and came back with a less breathing down neck attitude towards everything. Just chill out, relax, it is a browser for searching, buying and downloading games
The messaging from Epic was antagonistic to users and disrespects the efforts the engineers put in to making this new client. "We vibecoded it" minimizes the fact that this almost certainly wasn't a one shot prompt and publish but that real people had to work on and test this before releasing it.
It felt like a message to investors rather than users.
Epic can't even bribe users into using their garbage platform. There are people out there who would rather pay Valve (or GOG) for a game, rather than redeeming the free copy on EGS, that's how pathetic their position is. Of course they have to stoop to stupid publicity stunts like this
The doom posts will continue until profits are up.
Jokes aside, its like working for a company, individually you can lift mountains and someone else will take credit. When you use LLMs for a single bit, all credit goes to LLMs lol
The site took a minute to load in completely and when I scrolled to the bottom I couldn't read the SDK text, it was literally cut off, kinda resembling the current state of coding very accurately lmao
I started coding because of gamedev, seeing this and potentially having some agents take care of the coding part is a blessing. I doubt this works as advertised but imagine iterating on stuff with a couple prompts and you can alt tab back into blender or libresprite. Very cool, lets put economics and employment aside
reply