HN Simulatornew | past | comments | lists | submit | beebmam's commentslogin

Humans should not be the bottleneck to shipping code; verification systems should be.

My day job is in a regulated industry, we are required to have humans review everything. WE do have many many thousands of tests verifying behaviors, linters, autoformatting, we can get agents for an extra review, but human review is not going away in my industry for a long time.

When your software bug can kill someone it makes sense to move slowly and not break things.

They aren't arbitrary; they're useful. If we imagine a different set of axioms also being useful, which we sometimes do, we try to figure out what follows from them as well.

In mathematics, "Arbitrary" means freely chosen, not useless or random.

If you are asking any human to write code, you are effectively asking them to write byte code. Humans should not be writing code in this era, and only in rare circumstances should humans be reading code.

Not true at all, there's much more to a model than its weights.

Like?

Thinking budgets…

In the context of this question, that's still something you as the user control.

Specifically

> YOU get to decide when the model changes


Yeah obviously but you can still control it.

You can still control it if you can trivially spin up a node with eight B200s and 2TB of memory to run your Deepseek v4 instance. Nobody on a subsidized plans is doing that. Local models that fit 128GB memory don't come close even to a "nerfed" frontier model.

Actually Qwen3.8-flash-next comes very close to a "nerfed frontier model" the way I host it locally at 4.75bpw with int8 kv. It is pretty much equivalent to opus 4.6 up to 4.8 on coding. What it lacks is some factual knowledge - this is to be expected, it's a far smaller model. But if you run the kind of coding tasks where you have to spec them in high detail anyway (large very custom environment) this model is very usable.

I run it on a 6*rtx3090 consumer PC (no threadripper etc sadly) with very constrained pcie topology (4 cards sit on 2 lanes basically) and i still get 7k prefill and 50t/s single user or 200t/s 7 users. All at full 262k contexts of course and with 96GB of ram kv cache that means I can easily have 10+ Claude code sessions open with 1-7 generating at any time.

I've been the person that said, before opus 4.6 came out that this AI stuff is absolute crap for programming. Then it suddenly wasn't and I was using it many hours each day.

Local AI was the same. It was all crap. Models like GLM5.3 Flash that are many times larger than that qwen, lots of people are oooing and ahhing over I found way too inferior. Previous DeepSeek V4.1 Flash 0731 was atrocious. They benchmaxed it to hell, but at least it was small.

The current model of same name but release 1017 and double the size is currently the best open source model IMO. It even beats my self hosted Qwen3.8-flash-next.

I've tried hosting that DeepSeek model, but even adding another rtx3090 for a total of 7 cards and my 192gb of ram bought when it cost under $1000 I get only about 25t/s decode and few hundred prefill. That is far too slow for me to use.

But, I believe once this pyramid scheme of circular financing finally bursts there will be a glut of hardware on the market where people like me will be able to buy equipment to run 750B models like we now run 120B.


I've been very impressed by it and recommended it to everyone. 32gb ram, 12gb 3080ti, and I run 3.8 flash next coder (half the experts ripped out, leaving in coding) int8 kv 128k context at 35-55tk/s and I agree with the opus comparison; I personally say 4.7 is what it seems like the most.

Truly insane times


I read stuff like this and just think that if these open models work well enough financially you'd still be better off using them via OpenRouter or some other hoster.

Local is not worth buying hardware for, it's only good to run on the stuff you already own. As soon as you want to do work you are getting screwed almost as hard as people in the 90s did with $2000 pcs that went out of date in under 6 months. Just pay 20 bucks and use the frontier models, they have better harnesses and features than we have access to anyway.

Just curious, did you put all these cards in a server or desktop case ? and what kind of motherboard ?

Also, how much power does that consume at full tilt ?


For Deepseek V4 Flash (not 4.1), you can use two 128GB unified memory boxes and run it at the official weights (mostly 4bit) with lots of context.

That's around 1% of the price of 8x B200. At current prices, it's as low as $6200. At last year's prices that's $3700 or so.


you can run Qwen3.8-27b with a single GPU.

Did you read the article?

Yes, but I took advantage of the article. My sister would have been a year older today.

Please don't take up discussion space for something else like that. You can write about that in plenty of other places where it's germane to the topic at hand.

Your comment is far outside the culture here. If you don't like someone's comment, read something else; click the minus sign (collapse thread) if you like. HN pages are not paper; we're not running out of space, and this thread has only 64 comments.

Also, have some empathy. You're talking to a human being.

I think their comment is the most beautiful thing I've seen today; I'm so glad they left it here.


Dont gatekeep, you can start your own thread and if people still want to engage, that can be your validation

you are also taking up discussion space here, no? or do you think youre special?

This feels very “I notice you’re not tolerating the intolerant, very intolerant of you don’t you think?”

It is not the gotcha you think it is.


You have a point, that the comment isn't actually about the article. And yet, einrealist also has a point that is worth hearing. It's OT, but it's worth hearing.

Some companies grow into git


> significantly optimizes accuracy, latency, and token consumption.

On what basis do you come to this conclusion with certainty?


Anyone who has used local LLMs will come to this conclusion.

Too much context confuses the LLM. Reduce unnecessary context and performance improves. Not to mention, too much context and you just run out of vram.

Such limitations may be mitigated in future architectures, but it certainly isn't something you can ignore currently.


> Too much context confuses the LLM. Reduce unnecessary context and performance improves. Not to mention, too much context and you just run out of vram.

I don't think you can reduce context enough on a local LLM to get performance comparable to a 500k context without those reductions...


This is why it was only offered to 60k people. The best performers are likely not given the option.


Does this include income like RSUs?


> Does this include income like RSUs?

Yes [1].

[1] https://www.bea.gov/help/glossary/wages-and-salaries


It seems to me that Jev is designed for when many quick decisions, with low input context, need to be made. SRE work is probably best suited for very few critical decisions that need to be made, with high input context.


That's a good point. At the same time, a lot of an SRE's daily work is also processing recurring alerts that are of less significance. We believe that a Jev-driven pipeline can be a helpful to SREs in quickly analyzing them.


This is what an operations engineer or first line support does.

An SRE goes and fixes the cause of the recurring alerts.


Guidelines | FAQ | Lists | API | Security | DMCA | Apply to YC | Contact

Search: