- Larger MoE models only need compute proportional to the number of active parameter, so Qwen3.8-Flash-Next (180b with 6b active) runs more tok/sec than Qwen3.8-27b on the same hardware.
- Larger models typically have higher quality output and require fewer thinking tokens to get to an answer.
Consequently, measuring time to complete a task with a wall clock, the speed difference is less than you would expect.
Large mixture-of-experts models, like Muse Spark or Kimi K3.
MoE models want a different ratio of memory/compute/$ than dense models like a Gemma4 31B or Qwen3.8 27B.
Since only some of the experts are active each round, you don’t need as much compute to get good tok/sec, but you do need a lot of memory to avoid streaming weights from SSD or system RAM.
You could run a full Muse Spark on one of these for $24k at probably around 50 tok/sec.
You’d need 5x RTX 6000 Pros at $80k for Muse Spark (albeit at much higher tok/sec).
On a $24k budget, you’re looking at a maximum of 96GB from Nvidia or 384GB from Intel or AMD. So you’re bottlenecked by PCIe bandwidth, and all those TFLOPS go to waste.
Other configurations of the Studio have similar logic for other MoE models. For instance, Qwen3.8 flash-next runs at a very acceptable 75 tok/sec on a 128GB M5 Max for about 1/3rd the price of an RTX 6000 or 40% less than 4x B70’s.
Iridium still has a monopoly on high latitudes, too, AFAIK. Their USP is being the only network that works literally anywhere on the surface of the earth.
A $2500 Mac Studio with M5 Max running Qwen3.8-27b or Gemma4:31b in oMLX gets about the same speeds you quoted... and it comes with a warranty and has enough memory not to choke at longer context lengths.
Gemma4:31b, which is the better model with more complex architecture (and thus can't be compiled into a megakernel), is gonna be way slower in prefill, and ~50% slower than a 3090 even with oMLX. You can't get away from much slower shared ram vs vram, and less parallelism compared to 3090.
The other advantage of an eGPU setup, you can simply add/swap performance. Dual 4070s give you 24b of ram and you can still get about 80% of the performance of a single 3090, and actually you get faster prefill due to more parallelism.
So in terms of money/usefulness, Mac always loses.
SSD token caching (like oMLX does) goes a long way for prefill. Typical miss rates are under 5% in long coding sessions, which is mostly the output of a turn's tool use.
iPhones have great cameras. Get a MagSafe mount for on top of the monitor and stick the camera to it. The feature is called "continuity camera", and it works with any Mac app that would use a USB webcam.
And it's almost impossible to find ones that get as bright as Apple monitors for less than Apple charges!
My absolute favorite feature of Apple displays is that I don't need to have the brightness cranked to max all the time, even when sitting next to a window. Some of them are, dare I say it, too bright for a normal office at maximum brightness (yet still dim for comfort in a pitch black room).
There are some good alternatives if your budget stretches to $1000, though: Samsung's 27" ViewFinity S9 comes to mind, with 2880p and 600 nits.
That's a common misconception. You can't make the calculation purely on the mass of raw material needed to get a X kg of Y% material.
Every enrichment process drops in efficiency as the enrichment increases, asymptotically approaching that process's enrichment ceiling.
That's why nobody uses thermal diffusion to enrich uranium: It's by far the easiest method (you can do it with 1800's technology), but the practical maximum enrichment fraction is too low for anything useful (other than providing feedstock to better enrichment processes).
That's also the reason a centrifuge enrichment plant uses warehouses full of cascaded centrifuges: The centripetal force inside the centrifuge is working against a diffusion gradient, so each stage only gives a small percentage decrease of the "derichment" fraction.
Once you pass 50%, you stop looking for U-235 needles in a haystack to keep and start looking for U-238 needles in a haystack to throw away.
Exactly. The definition of the meter was originally derived from the circumference of the Earth and the second is (now) defined as a specific number of vibrations of a particular state of a particular atom. If you want to change the meter to make the speed of light in vacuum a nice round number then you'll end up affecting pretty much all of the derived units upsetting the whole SI system.
And in the end it doesn't matter. You should just use the right number, which is arbitrary and landed on 7500 m/second and change short of being 'perfect' but it would have been an incredible coincidence if it did.
The reason that didn't happen is because the speed of light was one of the harder numbers to pin down and the first estimates and later refinements happened much later than the definition of the meter. And to complete the circle: the meter is now defined as a fraction of the speed of light! This has the interesting property that even though we may get better measurements for various constants the fixed ratio means that SI as a whole remains standing.
That would also break SI in pretty terrible ways. So we're stuck with some numbers that don't look pretty in base 10, but if that bothers you can look at all natural constants in hex going forward and then they'd all be equally ugly.
What physics constants look like is for the most part a consequence of a number of accidents of history:
- the fact that apes have five fingers on each hand
- the fact that we have two hands
- the order in which the various constants were discovered
- some of the base items used for early definitions (which made complete sense at the time)
All of these to some extent factor in to that odd feeling that things are not as elegant as they could be.
- Larger MoE models only need compute proportional to the number of active parameter, so Qwen3.8-Flash-Next (180b with 6b active) runs more tok/sec than Qwen3.8-27b on the same hardware.
- Larger models typically have higher quality output and require fewer thinking tokens to get to an answer.
Consequently, measuring time to complete a task with a wall clock, the speed difference is less than you would expect.
reply