A negative datapoint - We have been evaluated prior Deepseek Flash models (both the original 4.1 flash and the smaller 4.0 flash) and they have been overthinkers. Our workload is well defined but moderately difficult coding tasks and among open weight models GLM5.3 flash performs best with Qwen3.8 flash next a close second. DeepSeek 4.1 flash fails at lower thinking levels and spends a huge amount of tokens (an order of magnitude more) at the highest thinking levels where it is able to successfully complete the tasks. I will evaluate this newest release since they specifically claim "the model reduces task execution time and output length", but keeping my hopes low.
Artificial analysis shows this: underperforming Qwen3.8 Flash Next and GLM 5.3 Flash, both of which you can run on a single B300 . I think Meta has a bit of catching up to do in the open-weight space.
As an aside, it's really unfortunate that artificial analysis only provides comparison at the highest reasoning level, which is mostly unusable for high-volume production use cases. I would really like them to start providing comparisons at low and medium reasoning levels across these models because those tend to be the ones that actually get used in production.
Been using GLM 5.3 Flash via Ollama Cloud basic subscription:
if this is the only model I had to use for the rest of my life, I would be cool with that. It's good at a wide variety of tasks including reverse engineering, design, working with ESP 32 etc.
I think Muse Spark 1.3 is better but GLM 5.3 Flash is an excellent daily driver.
It is unreal how good GLM 5.3 Flash is. There's an ~4 bpw EXL3 quant that fits on 2x DGX Sparks. I'm not sure it's worth it now ($7k/Spark) or even when I bought them ($4k/Spark) but there's something about knowing that the whole internet could go down and I could still have this model running at 50-60 tokens/s with 1M context, indefinitely. It's just so good. TensorFold hits 100+ tokens/s in aggregate with 2-3 M total context pool. It's like having Claude at home.
I'm using Flash and the full 5.3 model side-by-side, and for coding I'm hard-pressed to find a difference. Flash is slightly worse at steering in longer contexts, but writes code that's just as good as anything I get out of full 5.3. You could totally build a SaaS or video game with nothing but 5.3 Flash.
The full 5.3 is only really needed for super deep bugfixes/profiling passes, or ridiculous reverse-engineering in my experience.
Yeah, MiMo Pro is the first Chinese model I can really clearly identify as an upgrade over GLM 5.3 Flash. K3, Qwen 3.8 Pro, GLM 5.3, DS4.1 Flash all feel around equivalent to me. Oddly lots of people have derided MiMo but I have found it able to tackle slightly tougher problems. For instance, I tried GLM 5.3 Flash, DS4.1 Flash, and MiMo Pro on making a new map layer for my Garmin watch, and MiMo was the only one that found the actual best method (.img file for OSM).
Garmin.img thing is the interesting, but not really on my Experience.
Been running the same stuff on GLM 5.3 Flash, Deepseek v4p, Qwen 3.x model and Mimo v2.x pro model. Mimo faster yeah, but its hallucinate more and losses the thread easier. And when the context window is not limited, when usage of >200k token, is loss context. even he didnt know what im sayin.
Also, sometimes he Had a Loop over and over, for the same tool call (on my case is Grep and Read. >15x looping) until is burning a bunch of context for nothing. GLM and Deepseek doesn't do that (but the deepseek Hallucinations is huge asf too).
In my experience, like most models, it requires a ton of supervision. It is extermely verbose. Flash loves to create some wildly unecessary abstractions/spaghetti code (like most llms). It wants to implement things in the least effecient manner and needs to be reminded constantly not to do things a certain way.
Am I the only one experiencing accidental chinese tokens, wrong grammar, and other sloppy mistakes in almost every output when using GLM 5.3? I feel like using it through any kind of subscription service (z.ai, Opencode Go) just serves me aggressive quants.
On some days it's usable, but on most days it feels more like having to trust a pretty smart intern that may or may not get a small stroke every couple of minutes.
Never had that happen at all and using since launch as well, but I have a very custom harness running.
My local machine went down however and I used their web API to repair it and talking to that felt much stupider, I think harness really matters Alot.
I do get the really annoying LLM "that's A, not B" and "load-bearing" from time to time, those two tells stay no matter what I do even though the rest of the output is pretty amazing,
I was running DS731 until Qwen 3.8 Flash Next (so what, like a couple weeks? The pace has been torrid). I switched to Qwen for image input, then GLM 5.3 Flash. At the time, Qwen felt like a minor step down from DS731, GLM 5.3 Flash feels like a major step up from Qwen, so I'd say slightly better than DS731. I realize that's not exactly what the benchmarks say, but DS731 definitely flailed around on some of the more challenging things that GLM mostly gets for me.
I've been using GLM 5.3 Flash as the planner and DS 4.1 as the task worker over the last month and the results have been very good, predictable, ridiculously cheap.
Why this model split? And are your work volumes so high that you prefer these cheaper models over a subscription or are you simply trying to avoid the 200/m? (which will only go up).
Genuinely curious, why not use GPT Sol 6.1 high which has a better cost per task and intelligence ($0.32, 50) or Mimo 2.6 pro? ($0.13, 46) over GLM Max's $2.01, 45?
Seems like you should flip your list and save time.
I'm not sure why everyone is so ideological about tools.
If it works well I use it. Refusals are a non starter.
Imagine if my table saw refused to cut out certain shapes. I mean once they are internet connected some brand will probably try that...but just why put up with it at all?
They're not using gas generators to run the other models?
Gas generators are common in new data center buildouts. I'm not a fan, because they are insanely loud, but they do prevent the data centers from weighing down the grid and raising rates for all the neighbors.
This is absurd nonsense, and needs to be quelled wherever espoused.
Dams aren't as a nuclear explosion. They don't burst into appearance, in a way where animals cannot walk away from the rising water. Birds lose nests all the time (wind knocks down trees, for example) and they merely build another. And they only need nests to rear young, and those fall out of trees so they... build another.
What happens in reality, and not in fantastical, hyperbolic renditions of the truth, is that one ecosystem replaces another. Where before was river, is now lake. And this is an entirely natural process. There have been cases of earthquakes causing river blockage, and of course the reverse!
And of course our friendly beaver blocks rivers all the time, sometimes to the extent that it can be seen from space. Rivers being blocked is a natural, normal, common practice.
But I suppose you hate all the beavers too? Do you raise fist to air, shake it in emotive angst, and rail against the horrors of natural geological processes too? Do you hunt the noble beaver, too?
How about instead of dismissing, deriding, and debasing one of the most environmentally friendly methods of power generation, you get behind it? How about instead of suppressing a hydroelectric dam, which means people will build Ng generators, or use coal, you support them.
Or would you prefer nuclear, for that stable, steady, backing, of which windmill and our friendly sol does not provide? Or would you prefer massive acres of batteries, instead?
I beg you sir, please cease this mindless chatter!
(no I don't normally talk like this, but what the hell, a but of levity may bring less conflict here)
This was intended to be absurd nonsense (or as I call it, irony).
I mean, if you think about this, burning stuff is also a part of nature, wildfires happens regularly, volcanos active right now, etc. Burning fossils like gas was(from caveman burning wood) and is still most popular way to generate energy among humans. I don't get what's so special about datacenters people find compared to any industrial facilities to fuel hateboner on most absurd nitpicks.
Arsenic is a part of nature, but it’ll still kill you. Burning the massive amount of co2 stored underground during the carboniferous period is not the same as random forest fires that burn co2 fixed a few decades ago.
Pretty sure you're not able to use many models, besides local, if you need to guarantee where the energy produced powers them. I don't align with much of what's going on in frontier AI today. And I don't actually use it as much as I had been as I've invested in local hardware. But I'm still curious and continue to test frontier models on different tasks. My limited experience with Grok is that it's boundaries are much less rigid than Anthropic and OAI, IME.
But if the planet is the only consideration here you can't use any in of these models when pointing a finger at one. They're putting them wherever they can. And I'll be one of the first one to cheer some of these systems being dismantled after the fallout / when the US has some political leaders that have a spine again.
I don't buy that - you can't distinguish. This sounds like a convenient way for people to say "my choice is better for the planet" with basically no basis.
You've moved the goalpost and declared victory my guy. I literally gave you an option, that isn't meta, and is run on renewable energy. The fact that it's hard to get clear information doesn't negate the fact that there are differences, and we should seek to find the best option rather than giving up and accepting whatever bad option is nearest or shiniest.
Let me clarify my perspective here. You put the frontier providers at the top. That's probably representative of >80% of the total, global, inference. The other ones you mention? They're likely showcasing "green AI" as a differentiator, but comparatively, they likely represent a percent of global capacity.
And you don't get to put a feel good landing page from Meta and "declare victory" (I didn't) just the same. It means nothing because there isn't any accountability. Nice try.
Not OP, but I've been trying out Sol 6.1 medium with hermes agent, and I have not been able to match usage costs to what I've been getting with glm 5.3 flash high.
Really don't know why, since I've offloaded thinking to the LLMs in this area of my life. Though even at medium, 6.1-Sol is noticeably better at most tasks I throw at it.
My experience has been that the "cost per task" estimate wildly underestimates the cost of top models on most real work, because the tasks (as I understand it) tend to be isolated bits of problem solving or code generation, which rewards writing code that works the first time and not generating too many reasoning tokens. Sol is great for both. However, a lot of real work involves tracking down bugs, working through tests, examining many parts of a codebase, refactoring... even with subagents and good tool use this tends to bloat context before the problem gets solved. Deepseek's absurdly low cache price really shines in these instances, but even GLM 5.3 Flash easily beats the top models on price in real world usage. Maybe that's different if you factor in subscriptions.
Been very happy with it although I will say Muse Spark 1.3 being free on OpenCode is my main model these days. If I run out of free credits on OC (which almost never happens) then I drop into GLM Flash 5.3.
V4 isn't the newest DeepSeek, and GLM 5.3 non-Flash isn't the same ballpark
- DeepSeek V4.1 Flash has marginally better coding, and smaller experts (8B-In/16B-Out) with more compressed attention and KV cache, so it's faster but will have less expertise per task. It's a 552B model, so while it theoretically may contain more trained data it also requires more RAM (the experts are smaller to compensate for speed loss) making it harder to serve without the distributed KV cache DeepSeek hosts.
- GLM 5.3 Flash has marginally better agentic work, and has video input (DS doesn't), and its experts are 18B so there's more intelligence per task. It's a 320B model so it requires less RAM which will make it cheaper to serve. Considering it's a much smaller model, the higher intelligence indicates their architecture/training is better.
- GLM 5.3 (non-Flash) is far more intelligent than both; it's a 744B model with 40B experts, but it's text-only.
Model sizes aren't directly comparable, and they aren't even that relevant. I care about result quality, cost, and speed, not size. I switched to deepseek because it was giving me better coding results than GLM, and for far less cost. GLM is still better at planning.
In OpenCode, it shows up as "Deepseek V4 Flash Latest", so I assume it is the v4.1, so that's where the confusion comes from.
However, I disagree on the technical specifications. For most projects, a single prompt will get me what I want with good quality with Deepseek vs GLM 5.3. This is from real world experience with JS and Elixir. Perhaps for other tasks GLM 5.3 might be good. But, my biggest annoyance with most non-flash model providers - including GLM is they will dumb down the models based on load (like switching a non-flash model to flash model). That sucks because you expect a certain level of intelligence from them. Not a coin toss on "here's me hoping it works" and you expect a certain model only when you explicitly specify it by name.
> But, my biggest annoyance with most non-flash model providers - including GLM is they will dumb down the models based on load (like switching a non-flash model to flash model).
Where are you seeing this? This would be sufficient for me to stop using that provider entirely, immediately.
GLM 5.2 was the first open model that I was happy to use as my main. It's a fantastic model. 5.3 cost a lot more per task, was slower and didn't get that much better results for me, so I tried 5.3 Flash for a bit, it was ok, but didn't feel as magical as 5.2 and I have now landed on DeepSeek 4.1 Flash as my main.
It's not just you-- the claudisms in 5.3 are pathetic and drive me crazy, I was very sad to see it go in that direction and consider it a big step down in that respect.
BUT OTOH the performance of 5.3-flash are impossible to argue with-- it has the same pathetic claude behavior full of "Honest" and "lever" and "load bearing" tripe... but it's scortchingly fast on hardware that is pretty slow running 5.2/5.3.
As a result I use 5.3-flash a lot, in spite of feeling kind of gross about its language. I don't enjoy using it the way I enjoyed GLM-5.2, but its pretty effective.
Stupid question probably because I’ve gotten out of sync with the new releases but is Flash better or worse than vanilla 5.3? I’ve been using 5.3 extensively (and maybe ironically have used flash only for viewing images which I have as a skill in Pi because 5.3 vanilla seemingly doesn’t support vision). I find it generally competent but still very far behind opus 5.5. I’ve been trying to avoid Anthropic but the lift of opus vs GLM is enough that I end up coming back to it. I feel like currently there’s no comparison.
anyway, tldr I would not be happy if stuck with vanilla 5.3 so is flash better and should I move to that?
GLM 5.3 Flash is better if you need native multi-modal capabilities, higher speed, and lower costs, but worse than vanilla 5.3 in raw reasoning and for complex programming.
You don't need to directly compare the same reasoning levels. Today you can compare Sol on Low vs Haiku on Max, for example, but you can't look at GLM 5.3 High, only Max.
For some tasks I genuinely feel Opus 5.5 does better on low than medium or high. The trouble, as always, is figuring out which.
Edit: I realize after commenting that you're not suggesting medium is useless, but just arbitrary and differs between models. Personally I don't want to directly compare mediums between models, I just want to be able to compare across effort levels.
i run opus on medium, sonet on high, and haiku on high for implementation based how complicated is
for planing and specing and review opus is on high usually, except if if the task is contained.
did not try to run it on low, will have to try it.
Opus 5.5 on low is good if you are relatively sure a given task is straightforward and time spent gathering background information is purely wasted or possibly even detrimental. Opus 5.5 low seems great at doing the most obvious thing based on the prompt without thinking much about it.
I agree - Opus 5.5 on low is a really great model if it has some direction and the tasks aren't complex.
If the tasks are complex cranking Opus up to high or xhigh and getting it to delegate to Opus low subagents is pretty effective and much faster than running everything at high.
It shows the time to complete a task versus intelligence index, which is much more important for my use-case as I do quick back and forths usually. Someone on here suggested it to me a while ago. It also captures speed and token efficiency nicely, since both of these decrease time per task.
Yep, just benchmarked it against GLM5.3 Flash on my own local bench and it's slower in TTFT, tok/s and concurrency. It also scored lower.
Tried different reasoning levels and didn't really see any difference but GLM 5.3 flash only really works high, max or none so I couldn't really compare the lower efforts which appear to work on Spark.
this is properly bad as you can run glm on 6000pro, at (in my eyes) pretty good quant, and a q2 on a 5090. im suprised at how good 5.3 flash is at some parts. i still think ds4.1 flash is better but it is more demanding. i hope meta pushes something in a range of 300-400b with a 15-20b active parameters. i know its a crowded space. but it would be fast and probably mostly good enough.
> As an aside, it's really unfortunate that artificial analysis only provides comparison at the highest reasoning level, which is mostly unusable for high-volume production use cases. I would really like them to start providing comparisons at low and medium reasoning levels across these models because those tend to be the ones that actually get used in production.
The frustrating thing is they started doing this for the big labs—you can see their benchmarks for all reasoning levels of Sol 6.1 or Haiku 5.5. They just don't seem to do it for any of the open weight models.
I feel like reasoning is a useless metric, and we should instead focus on results, and time to achieved said results. That is a more useful metric overall.
> Artificial analysis shows this: underperforming Qwen3.8 Flash Next and GLM 5.3 Flash, both of which you can run on a single B300
You don't need anywhere near a B300 to run Qwen 3.8-Flash-Next in its Q8 largest variant published as a GGUF by unsloth. It fits very comfortably in a 256GB system.
My immediate reaction is that it’s weaker on coding than other open-weight models and that coding is a primary use case for running locally (not the only one, but surely a major one). That said, it takes a lot to get full Kimi and GLM running locally.
very true. when configuring custom agents for enterprise, we almost always default to none or low reasoning knobs. The added latency from medium or higher reasoning levels compared to the improvement in response quality is a diminished return.
it’s quite suspect to claim that a model is ‘underperforming’ so resoundingly, especially after so little time since release.
there’s this objectivist approach of reducing llm quality to a single dimension, furthermore one with measurability bias.
my thesis is that the chinese models beat the american models on benchmarks, on quantitative measures, but probably do worse in subjective measures. to put an example, compare a chinese phone with an iphone, the specs are great, but the OS? the materials? the privacy posture?
so will chinese llms have great specs and benchmarks, but will miss the mark when it comes to the actual quality in practice when your input stops looking like the benchmarks it was trained on.
America number one, it’s the chinese that play catch up with clones, not the other way around
You're asking a question, which if people follow up with searching may lead them to the implied point. Just post the numbers if you think the numbers are interesting ¯\_(ツ)_/¯
Um a 512GB m5 max Mac Studio is what 25k? A B300 GPU and the computer to use it is what? Double? Triple that? Apples v oranges my friend. Apples v oranges.
I seriously doubt their ability to double API prices on Dec 1. Token prices are in a clear downward trend and Google should maximize revenue by competing for volume. Currently OpenAI is the only US frontier lab trying to seriously compete with open-weight models that you can run yourself for an order of magnitude lower than what frontier labs want to charge
I'm a little skeptical of going below 4-bit quants due to the potential for significant degradation in quality. I'm running 4-bit quants on an RTX Pro 6000 rented for approximately $1/hour and getting about 1.2 million tokens out and 40 million tokens in per hour with caching. The quality of 4-bit quant is good enough for difficult but well-scoped coding tasks. Here is the inference stack I am using: https://www.reddit.com/r/BlackwellPerformance/s/FrKwk3GoDK
So strata is a little slower, but keep in mind that ninfer-3090 is very optimized for a Qwen 3.8. Standard Qwen 3.8 runs at 20 t/s, this modified version can do 50 t/s (but it's extremely long in it's thinking, it just goes on and on.
This is on a 3090 that will crash unless power capped, with a Zen 2 CPU, 64GB DDR4 with a PCIe that refuses to go higher than 8x (basically pretty crappy all in all).
Yet with some tweaking and optimizing I still manage to get strata to run at 40 to 60 t/s.
That strata has been optimized on my Oh My Pi conversations. So when I'm using it, it's probably faster and closer to ninfer in speed than during those unoptimized benchmark tests.
They don't but you can still format things in an actually readable way and be nice to people.
People also often us _italics wrong_, just coz it works that way elsewhere. Works totally fine with a "*" instead on here. Just go with the program and you can make beautiful posts even on here. Heck you have AI now so you don't even have to remember anything any more. There's even a `help` link next to the input box and just to write this I double checked it and I was even misremembering and using two too many spaces for monospace! I really do wish `these would actually work` but hey, you really can't have everything!
Unrelated... just use your just locally installed qwen 3.8 and instruct it to figure out why you arent at 16x.
For me, I didnt know I was on 4x and I was able to, giving it enough permissions, with a good harness like PI, to figure things out and go 4x -> 8x -> 16x.
It probably will take some opening the case and switching ssds around, but it's amazing how powerful these models have become.
Not as fast for me. I was able to snatch a spark before the horrible price hike, and I have it at 40/50tok/sec. Before I had the great 3090 setup which is what led me to get the spark. That 3090 is now relegated to creating videos of my kids doing dumb things.
I now backordered two more sparks, before the hike... hoping they will honor the agreed price and dont cancel on me. They should arrive in a month,
Its evidently clear the frontier labs won't keep on giving us cheap inference for much longer. I also watch in disbelief how people say a sub is cheaper, which probably is. But you loose on so many other things (privacy, predictability, censorship...)
Ill check for this in my setup. I thought it came with only one thinking level causing the “meandering” as it refers to its own long CoT.
I’m doubting the quality of the model, even comparing with luna 5.6, it is giving such bracingly wrong answers. It claims other tools have changed the code when pressed why it made a mistake. That code or tool didn’t come close to working on the project. It is scary.
Anecdotally, I've found that 27B at 4bit hallucinates a lot more than QFN at 3_xxs, particularly for less technical reasoning.
I've tried to use both for online comparison shopping. QFN not only seemed less delusional but also made useful observations and problem solved ways around many different website access issues.
For the 1 dollar per hour, may I ask (without trying to incite things) why you would not just buy a ChatGPT Plus subscription or other 20 dollar subscriptions? Do you like the privacy?
People think claude costs $200/month, it actually costs $200/month and all your labor to train their models, improve their product, and all your best ideas and work that they will feed back into their company. Its very expensive
You can opt out of model training with a checkbox, though I doubt most companies respect this completely. At the very least, I imagine they find loopholes to exploit.
But on the other hand I'm giving them my best and worst ideas and the value I get in return outweighs what I'm contributing to them. I also don't have the problems of initial HW cost and maintenance.
Yer I also use service with zero data retention promises but it is hard to really trust them for me...
They trained on basically all of the existing material, legal or illegal already and are starved for true human input.
They are also in a heavy arms race were basically only a handful of companies will survive.
Why do you care? It's their legal/compliance problem, not yours, if it turns out they didn't (though I don't understand the urge to deny them this in the first place, it's not like it hurts you or deprives you of anything).
"Whoops my bad, you are right, your data may have been in the training pool. They are now forever in weights that'll be reused to train every future model. Too bad you can't do anything about it."
Their own TOS will still include your "anonymized" data in their "research", with no explanation as to what that means. You need an enterprise account to get real zero data retention.
Yes, and? Why would I care? Why would I begrudge them? I'll get the improved product, too, that will benefit from not just my data, but everyone else's to.
Don't get the greed and the Dog in the Manger mentality so ever-present around AI detractors.
Access to models is currently being priced below it's actual value. Largely because the market hasn't settled yet. Once it does the prices to access AI will increase -signifigantly.
It will be less of an issue for people who have already got the equipment and the practice running locally than it will be for people who have simply been relying on ChatGPT.
While this may be true, their throughput is practically unbounded. this has been a primary challenge to using local models at work, where its normal for me to have 1-6 sessions churning away at once. So while 100t/s on a single session is very fast, theres no way you could get a cumulative ~500t/s from a consumer setup like this, do the bandwitdth bottleneck on a single card/cpu/ram/ssd (tends to be gpu ram bandwidth is the limiter in my setup, but they all have respective limiters to this kind of throughput)
I think Opus 5.5 is now fast enough with the inference stack improvements that they did. Sonnet 5.5 is around 140 tokens/s when I had measured it last time. A $20 subs on claude and using only sonnet 5.5 will last a lot
I was firmly wanting to move towards local llm, and since the Sonnet 5.5 and recent Opus speed increases, I'm putting that on hold until local inference speeds can be improved. Not sure how much juice there is to squeeze there, but I'm hopeful.
>I'm a little skeptical of going below 4-bit quants due to the potential for significant degradation in quality
There's literature on this actually, if you want to get that low the model really should have quantization as a pretraining target otherwise larger models that are quantized after collapse.
In fact, to do this effectively, you need somewhere near 50x chinchilla to do QAT on super tiny targets like 1-bit or even 2-bit. These large models already need an astronomical amount of training data that doing proper QAT that small isn't really feasible unless there are breakthroughs in the architecture.
4-bit models and smaller _can_ perform well but not through just naively quantizing the existing weights of a large model.
Yeah, there's a tradeoff. pre-training with QAT needs more data and a ton of hardware, so it's easier to just take an existing open weight model and quantize it. This works, but it'll underperform a model that had quantization as a training target.
I am renting spot VMs from Nebius. I've tried a variety of other providers like Vast and Spheron. Vast worked well for renting 5090s but I like the large memory and pricing I'm getting at Nebius for RTX Pro 6000. The extra RAM really matters because I need PLE to offload the ngram to RAM.
Yes, you can probably get the whole thing up and running in about an hour the first time. If you pause and restart, it takes about 15 minutes to load the models from disk into GPU memory, so budget for cold startup time.
How does it take fifteen minutes to read <100 GB into GPU memory? Shouldn't that be limited by SSD speed with everything slower than a minute being a terrible ssd?
A lot of cloud platforms have terrible slow network storage. They also might need to compile the GPU kernels fresh as they might not have a persistent CUDA cache.
this is not right. vllm can easily load safetensors model at >5GB/s if not faster when setup right. did you make the compile cache persist? if you use docker, you should bind mount the kernel compile cache, so they don't need to be recompiled each time vllm restart.
I have to concur with you here because I'm running 3.8-flash-next in the largest unsloth Q8 version on a 256GB system for daily use, and I've recently compared it head to head vs a Q4 that I have on a 128GB system and the Q4 is just plain not as good. The Q4 is given only more basic tasks.
I can't even imaging letting a Q2 anywhere near a code base that I care about. To generate a bird on a bicycle as a test? Sure. Actual things I don't want it to screw up? nooooooooo.
There is a magical quant GSQ-RCO that at 3-bit XS beats the full unquant. I have tested it. It is on par with the full unquant on DeepSWE. Give it a shot, it is magical. ITSA lab, they basically devised a way to selectively quantize parameters. I am a GSQ-RCO truther at this point... it is so fast on my V100s and I can't find an instance where it has acted weird or fell short of the full quant on agentic coding. I Have it 3d model in blender, all sorts of stuff. It's just... good.
I've run DeepSeek V4 Flash on DS4 on single DGX and standard model weights on aDGX cluster. There was some degradation going to the hybrid 2 but quant but really not much.
Could you please stop posting unsubstantive comments and flamebait? You've unfortunately been doing it repeatedly. It's not what this site is for, and destroys what it is for.
Apologies. I had recently read (I think from someone on here) that they were in some discord where those sellers advertise. Whoever it was mentioned that the service was incredible value for what they got, but admittedly it was for toy projects, there was no written agreements, and uptime was something like 98%. I should have found the source and linked, and will try to do better overall in my replies!
We recently moved from 27B to Flash Next. The quality is superior for coding. Our workload is primarily well-defined coding tasks that need to be attempted a few times before the model gets it just right. FlashNext is also better at finding issues in generated code than Gemini 3.8 Flash.
I'm on m1 max 64gb and went from qwen3.8-27B back to qwen3.6-a35b. Is flash next the move? I went from usable say 40tk/s qwen3.6 to unusable, like 11 with 3.8 and not impressed with the replies for the time sacrifice. pi (omp) and omlx but not with the recent 3.8 patch.
I've been waiting for a 35b of 3.8, I don't really know what the other versions are about. I'm on 5g so juggling 40gb of model files sucks. And honestly I'm sick of tweaking this stuff for no, very little, or break-it level improvements. Qwen3.6-a35b has been solid for work, just don't give it freedom to wipe your data.
I’ve heard a quantised version of flash next can fit in ~50 gb of vram (which needs a system level flag set to go over 48gb)
But the m1 cpu is itself a bottleneck on prefill compared to say an m5, there’s no real getting around it. And the 400mb/s bandwidth starts to hurt without MOE
Hoping these model optimisations can see us through to 2028 because for everything other than LLMs this hardware is still over specced and working incredibly well
One key factor not directly mentioned in the article is the simple alignment of financial incentives when electricity distribution was privatized. Before privatization, the agencies responsible for collecting electricity bills had little incentive to actually perform the collection. When coupled with rampant corruption, you get a situation where a lot of bills are either getting unpaid or not even being generated and just being written off as transmission and distribution losses. Switching to private distribution immediately aligns the incentives of the distributor to actually bill their customers for the electricity they are using. None of this is meant to take away from the value of the technical and policy work that led to improvements, and I'm sure all of the factors called out in the article also contributed to this change.
Pretty impressive to see the amount of performance they can squeeze out of the same hardware. I suspect the same process will play out for all combinations of LLMs, inference providers and hardware stacks. This should bring down the cost of inference for the providers by an order of magnitude in the next year and lead to fantastic margins for inference providers.
Back in 2008 Fujitsu has one of the best performing 10Gbps Switches. We were building 40 Gbps packet sniffers at Google (4x 10 Gbps NICs) and needed switches that could do things like mirror traffic across ports at line rate. Fujitsu was way ahead of the pack. I always wondered what held them back from building a meaningful networking business in the US.
> I always wondered what held them back from building a meaningful networking business in the US.
It was just timing. Their networking gear was built on some very impressive ASICs which became commodity chips a few years later before they could grow. Arista, Juniper, et al ate their lunch using other vendors’ IP. IIRC it was Broadcom and Fulcrum that released the chips that killed them.
>We were building 40 Gbps packet sniffers at Google (4x 10 Gbps NICs) and needed switches that could do things like mirror traffic across ports at line rate
I'm sure there were good reaasons for this, but man this just sounds bad. Like that's the spec I think NSA must give to their contractors outfitting 611 Folsom Street's Room 641A
Port mirroring is the network engineer's equivalent of using breakpoints in a debugger for a programmer. It allows you to inspect what happens on the wire to get a clue for what's wrong.
Maybe I'm being naive, but as someone who's spent a lot of time capturing and reading PCAP files, 40GB/s worth of mirror is gonzo nuts! You may already be looking for a needle in a haystack, but this time the haystack is 1/60th of the entire bandwidth of the internet in the year 2000!
It does sound bad the way OP described it, but packet or HTTP mirroring is a standard feature of A/B testing and blue-green deployments of a very critical code. Mirror traffic between version 1 and 2, then compare response body, response time and return response A to the end users.
I used to operate several dozen 10G/25G/40G/100G taps which fed a series of tools in the DC at a regional healthcare back in the day.
80% of the relevant outcome was typically to get a true-to-the-wire sniffer capture (switch mirrors won't always mirror 100% of packets for various reasons) for troubleshooting performance of whatever the complaint of the day from the server team was.
19% was for feeding a security monitoring systems which looked for abnormal flow patterns to let us know a server was compromised.
1% was for the call recording system compliance requirement for the emergency department.
0% was because I was a cool superspy tasked by the government to siphon info to them or trying to sell medical records on the black market or something. I mean, you can try to something nefarious with such tools... but one could say the same about a generic server, SAN, application, etc as well. People are just used to understanding what those would typically be used for so they don't assume it must be for the scary thing they've heard about.
That said, it doesn't rule it out either. But again, the concern shouldn't be sourcing from their usage of normal infrastructure tools it should be sourcing from... well, all of the user analytics Google very publicly does directly in the server.
It's super common and very useful for security systems to work like this. Instead of putting something in-line that might choke on a burst of traffic you put it off to the side and send it a firehose of packets it may or may not be able to handle. If it falls over, no problem.
These tools are very common in non-nefarious ways for troubleshooting networking, and while vendors set up to serve this space solely as troubleshooting tools, I've also built my own that can sit on a network link and monitor for problems.
In my case it was for mobile wireless signaling traffic (all the coordination for creating a mobile internet connection, handing the connection off between towers, etc), and I'd credit it as one of the reasons you're mobile internet connection is so stable. When LTE first came out, myself and many others solved all sorts of bugs in the equipment and protocols by using or building these sorts of tools.
There are companies that mirror their traffic to storage for a few days to be able to look back and troubleshoot issues. ALL their traffic, proactively.
For US federal government, "Office of Management and Budget" says to store 72 hours of packet capture, since 2021 (partially in response to SolarWinds attack). (M-21-31)
The XG series switches? Force10 sold them until Dell bought Force10 and continued to sell them (although the firmware changed a lot along the way). They were, AFAIK, pretty popular in the US under the Dell brand.
My home network is XG2000 10G core and a few XG0224 1G/10G dist. Very nice switches.
Ubiquity proved you can bust into the market from the bottom end if you want. Nothing was stopping them from doing the same but like many companies they prefer to focus on how much money they can siphon from the Fortune 1000 so I can't say I feel sorry for them.
I'm not sure why that's relevent, particularly since Ubiquity's approach wouldn't have remotely competed with Fujitsus custom ASICs if they were actual contemporaries, but you clearly have an agenda you're in love with, so...ok, sure.
AFAIK those sold relatively well, considering the TCO of both sides (NICs were very expensive and there were like a dozen companies vying for position once upon a time). They were also interesting because they were cut-through. CX4 was noticeably lower latency than SFP+. So a very fast, but feature limited architecture which is probably why other players like Cisco and then Arista (aka Broadcom) took over.
We have alliances with Japan and S Korea which predispose us to believing we might be able to get a clean supply chain from them if a new Cold War starts. Out other options don’t even pass a giggle test and that’s the state of the world until some manufacturing is brought back onshore for strategic reasons.
The same way Congress has been subsidizing Boeing and its competitors since WW2 to make sure we can build things that fly if another war ever starts. Or with drones, that we can build ones that will be flown BY US instead of defecting the moment they get in range of the other side.
> The same way Congress has been subsidizing Boeing and its competitors
Funny you should mention that. Boeing deliberately buys 35% of a 787's airframe from Japan (along with other equipment for the 787 and other models).
That's a deliberate play: we buy a lot of parts from Japanese companies, your airlines buy mostly Boeing airplanes. Even today despite some diversification Boeing has over 70% marketshare in Japan.
If you ever wondered why Airbus setup factories in the USA that's a similar deal. The US Gov and Mil want to ensure factories and workforce on home soil should the need arise. In some unrecorded meetings somewhere strong hints were no doubt dropped that it would be in Airbus' best interest to play ball. They obliged.
That said the USA is the largest supplier of parts for Airbus airliners so there's also a benefit to having aerospace manufacturing and R&D here.
Boeing proposes sth called the “Boeing Honeywell Uninterruptible Autopilot” to add drone-like remote control capabilities to its airliners (for security reasons, obviously). I'm not aware of a similar offering for Airbus but I have no doubt such a capability has existed in Airbus as well for perhaps two decades. I expected development of such a sensitive system for Airbus to be based in France, but your comment about “strong hints” and “play ball” has me thinking…
This is the first time ever I am hearing of Fujitsu being involved in any bribing/lobbying controversy in the US (whether as a victim or an offender), and I doubt I am the only user in this thread in this position.
Unless it is a very well-known controversy, I feel like there is a need for extra context here. I am not even asking for anything of the "proof beyond any reasonable doubt" nature, I simply want to understand what you are trying to reference.
It is great to see another player introduce a fully open stack. Nvidia's Nemotron is the only other prominent one I know of.
All that said, the headline claims do not match the self-reported performance. For example, the dense 32B model is significantly behind Qwen3.8 27B (chart towards the bottom of https://ifm.ai/blog/k2). Gemma4 31B is not in the comparison set. This is the most important sweet spot for self hosted open-weight models today and real competition here will be very welcome.
They have the 32B listed as "stage 1" with the note "final checkpoint to be released." So, not finished yet. Not sure why you'd release it if it's not finished, but that's the explanation.
