The price numbers look weirdly specific, I wonder where they came from.

Personally, the thing I'm most excited about is the FrontierCode 2.0 score, which is a benchmark I trust a bit more - 31 seems to be roughly on par with 6.1 Sol and Opus 5.5 medium

I have essentially no knowledge in this field, but another concern may be that if the model has minimal cybersecurity knowledge, it may also implement insecure systems as a side effect.

My initial thoughts too, it's going to write insecure software. I recall early LLMs having problems with trivial stuff like SQL injection.

Definitely going to give this one a skip.

> LLMs need an insane amount of text. Dessa learned language from hundreds of times fewer words (in fairness, LLMs also have to learn C++ and whatever the hell Angular is).

Now that a larger proportion of text is AI, I'm appreciating human-written text more and more; this got a good chuckle out of me.

>and whatever the hell Angular is

I've definitely seen LLMs compose jokes with this structure a lot where "whatever the hell ___ is" is in parenthesis. Not saying this is AI generated tho

I wouldn’t be so sure that that’s human text

I can reliably get my custom models to output a 95% perfect replication of my tone and voice

As somebody who attempted that recently, I seriously doubt that percentage :D

Maybe you just have a voice that's, shall we say, in sparse latent space? ;)

New insult dropped!

I have an insane amount of writing output as training data going back a loooong time so it all comes down to how much training data do you have

Many of us can do it on just our hacker news comments (been there, done that).

What do you mean by your custom models? Do you mean custom prompting?

Couple things:

I have a “style guide” markdown that is pretty expansive

I run an RLHF style loop with me in the middle for all my writing now where I give a voice rant that is over expressive (aka redundant samples of concepts that I want to express clearly) in order to manage lossy compression steps

Then I will manually clean up that last 5% after a half dozen rounds

That sequence goes into the style guide to be referenced as a reasoning chain of thought to map rant > output

It’s been three years of tuning I’d suspect at this point

How did you train your style guide, though?

I have a somewhat expansive style guide, and it still reeks of Claude.

My cleanup is much closer to 40%, which still makes it worth the effort, but sometimes when speed matters, I'll apologize for the slop.

I've recently been using gemini argon as a voice judge. so it goes claude ( with style guide) -> voice judge with argon -> claude adjusts. I do that in a few loops before it hard stops if it can't reconcile, and pushes to me for review.

Guessing - finetuned? Or LoRA?

EDIT: Nevermind, see below. I would definitely go with a LoRA though.

That’s if I was going to build my own model on myself, which I’m not planning on doing at this point

Inconceivable!

Super interesting read; I especially found this letter from "P791" fascinating:

> Hi, Wonyoung, I map North Korea to help North Koreans.I select areas that look like they might be good points to escape. In particular, the detailed mapping around Phungso-ri (풍서리) is almost entirely my work.I mostly map along the Tumen River, both on the North Korean and Chinese sides. It is important to map the fences, roads, and towns on the Chinese side.

They chose to cross anyway knowing the risks, and it's still preferable to death by firing squad.

"Beyond Utopia" is a harrowing documentary about a South Korean pastor who helps people escape NK, filmed on the trail side by by side with along some actual refugees. It illustrates how complex, dangerous and utterly corrupt the process of getting out is, and also mentions the role of local gangs: they detain refugees and then do a bidding war between the authorities and relatives to extract every penny.

Interesting - I wonder if it's because coding doesn't use the specific token probabilities, while decision models do

Yes this is the reason. Transformations like quantisation preserve the rank of outcomes much better than probability mass.

FWIW, my tests used only ranking. I used Kev to drive E2E test runs using agent-device, where Kev had to decide the next action - like pressing a button, or scrolling down etc, to reach a certain goal (UI state). It went surprisingly well, but 4B BF16 was significantly better than 8B Q8. The latter made some wrong decisions in every run while the former was almost at Luna precision.

Fascinating insight. I'm curious, how do you know this?

I think it also helps that speed and latency aren't as vital for coding and we can afford to let the model think for longer.

I think, considering the size of this model, it's closer to a Pro than a Flash on everything other than speed