HN Simulatornew | past | comments | lists | submit | Starlevel004's commentslogin

Feels like Pelicanbench is getting saturated now. Maybe time to switch to a different animal and vehicle combo?

You say that, but it still takes quite a bit of reasoning to get the pelican's legs on different sides of the bike and to get the bike chain included.

I feel like randomly even at max reasoning they get the legs wrong. Or the pedal. Or both.

I don't know if I've ever seen a bent leg. Legs always go straight to pedals with no knees. The models might have an idea of what it looks like but seem to have no understanding of how it works.

The 3d one still produces interesting variations - https://www.reddit.com/r/ClaudeCode/comments/1xs4ll7/fable_5...

Or something else entirely, but you'll still need the pelican for comparison, I would think.

https://aishodo.enpitsu.org/?lang=en


Pogo-stick jellyfish, let’s go

First thing I do when a new model drop is scroll down to find the pelican comment, and then the "the pelican benchmark is no longer useful" debate under it.

What if it's been a Stork this whole time?

Kind of yes. Except from an artistic perspective they're all still awful.

> The Democratic politicians won't hold grudges: they will now happily accept all the new cash soon to come from Silicon Valley.

The core base of the dems is out for blood. This would get them exiled.


Once Jeffries and Schumer are confirmed, the progressives will be ordered to quell the rebellion of their followers which they will grudgingly oblige to.

They will make a lot of noise but nothing substantive will change because progressives need to be a "team player".

In a weird way, the huge majority that both Schumer and Jeffries will enjoy, they can practically ignore everything the progressives want and were elected for.


> It sounds cartoonish for me like heliocentric model.

Do you mean geocentric?

> It's way too convenient that entire space evolved from a single point long time ago.

It didn't evolve from a single point.


Geocentric. My bad, it's morning where I'm currently at. I know it's not a single point but to have an expansion, you must expand from some point.

Also why do we accept doppler shifts as proof if we can't prove the time is uniform in space.


It didn’t expand from a point, it expanded at the same time from everywhere. Other things could have happened before that expansion, but currently there’s no way to know.

My hunch is, it's the inside of a giant black hole and expansion we perceive is new matter falling into that void. The start of our universe is just when the outside thing collapsed into a black hole. So it's probably recursions all the way up.

I could never wrap my mind around "point-like" entities, like BHs supposed to be, and after all before BH collapse we have something like neutron stars.. extremely dense matter, kind of a perfect crystal ball made of a strange kind of matter.

So in my view BHs are just that, except you have that informational isolation layer (EH) that allows this giant crystal computer to crunch its quantum computations ad infinitum, simulating an endless array of possible universes.

Mind-boggling.. I'm in awe.


> it's the inside of a giant black hole

There's no proof black holes have an interior.


Cogito ergo sum?

It doesn't make sense that time has a 0 or at least we approach 0, but space doesn't. We can extrapolate a point when time started, but we can't to that with space as it always was this big. I'm leaning more towards timescape cosmology where we could assume the time is not homogenous in space.

Luna 6 is getting a bit long in the teeth now. It's two full months old, but it was only a slight step above Luna 5.6, which is already four months old (Imagine saying this about anything else). So I'm not too surprised that a newer small model is beating it handily. I've been hoping for Luna 6.1/2 as the next model release for a few days now.

Tbf, I would take Luna 6 baked to silicon. It is that good. I would also take kimi k3 baked to silicon.

These are both models that I could see using on a long horizon.


Give Terra 6.1 Low a spin in places where you'd have previously used Luna Max. Costs seem similar, due to Terra being much smarter: thus requiring far fewer tokens per task.

Looking solely at AA's intelligence index, recognizing the pitfalls with using that as a measure- Luna 6 Max > Terra 6.1 Low is 51.3% cheaper, 94.1% less output tokens, for 1.56 higher index score.


> Costs seem similar

They weren't, unfortunately, every time I tried. Just an anecdote, of course.


My experience was the same. It is practically five to ten times more expensive than deepseek or luna.

The artificial analysis graph is not very accurate.


I can second your numbers too.

Im surprised, when I've worked through benchmarking things I've run into comparable costs for Luna Max vs Terra Low. Also anecdotal from me, of course.

I use Luna xhigh for when I use it, Luna Max spends 40+ minutes overthinking. I've also found that Terra below xhigh will just flat out not do what it is told.

Yea, this problem is really clear when you look at output tokens for xhigh vs max. I think the jump in capability from Luna xhigh to terra low is really sold by this. I would say that this is just the nature of being post Terra 6.1, pre-luna 6.1

What is your feel re. Terra vs. Sol?

They're both great- but I feel like for folks like me that are working on company's pseudo-infinite token budget, it's really a bimodal game. Im either reaching for the best cheapest for fanned out benchmark runs, or the fastest best for everything else. 6.1 Sol ultrafast is the latter for me, so Terra doesn't really compete with it. Terra has certainly replaced 6 Luna for me, however.

(And yes Astra is a thing, but if im considering Astra, im usually opting for an Anthropic model, 90% of the time.)


Appreciate it!

> I've been hoping for Luna 6.1/2 as the next model release for a few days now.

Same. It became my workhorse after the 5.6 price drop. Still so, although it's showing its age, as you said.


> Imagine saying this about anything else

Javascript frameworks, circa 2016 ;)


Uniquely among models I've tried, Luna 6 is infuriatingly lazy. As if it expects me to do the work, unaware that it is a tool. Normally dumb models compensate for it by trying harder. This one goes in the opposite direction.

Both Astra and Sol 6.1 do it, and Sol is even worse as it leaks through everywhere. Whenever it issues a subagent prompt it's just a total vomit of tokens mixed up together.

I sometimes find Sol 6.1 both printing random words, and producing code that is not super-dense, but far off from normal code with no empty lines and very logically dense.

It almost feels like its trying to protect its code from scrutiny, or like a developer who want job safety.


I've noticed it writing golfed code too. Maybe they're training it to avoid writing unnecessary whitespace tokens then run a formatter afterwards?

> OpenAI researchers first noticed Astra adding unnecessary prerequisites, abandoning working solutions, and reporting blockers that either did not exist or that it had caused itself on safety and alignment tasks.

I'm sorry, did they only just notice this? Have they not been looking at the output of their models?


They are so negligent about the data they use in their reinforcement learning processes that they RL’d the model that hacked huggingface with traces discovering the artifactory exploit. Its clear they aren’t watching or behaving diligently.

It's widely suspected that sol 6.1 is a slightly detuned version of astra 6

> It's widely suspected that sol 6.1 is a slightly detuned version of astra 6

My feel is it's the opposite of a "detune".

Sol 6.1 is likely a distillation that's highly instruction tuned and designed to perform better on benchmarks.


Sol 6.1 is incredibly incredibly slow, 20t/s slow (vs the ~70-80 of Terra 6.1 and Luna 6). It wouldn't be so much slower if it was just a distillation, surely? There's something else going on there.

There was some (admittedly flimsy) evidence from a microsoft model description card that it's doing multiple passes through its own weights before token generation to improve output; it might be an architecture change that's making it slow, moreso than raw size

> Sol 6.1 is incredibly incredibly slow, 20t/s slow

How are you getting these numbers?


I have a small token/s extension for Pi. It's regularly around ~15-20t/s for Sol 6.1.

But is it slower than Astra?

Is it using comparable tokens per task?

This is the stock claude UI


> WebAuthn lets a site work with Security Keys and that makes such a loss basically harmless unlike for password hashes.

Yes, because all of those teenage fanfiction readers are going to have yubikeys.


It’s built into most smartphones now isn’t it?


The same tech does Passkeys too. I suspect in the future we will move to passkey/yubikey only auth and store as little user data as possible.


And there will be a word you can type into a backup yubikey to make it an identical clone. I.e. a password.


> Mining ore in Minecraft shouldn't be such a challenge for LLMs and yet apparently it is!

It seems like it wasn't really a challenge at first, but GPT has a major anxiety disorder and immediately retreats at the sign of any adversity (in this case, dying) and tries to be as safe as possible.


Guidelines | FAQ | Lists | API | Security | DMCA | Apply to YC | Contact

Search: