You'd have to install an extension or write a function that intercepts tool calls.

> for some reason I assumed AMD has been outselling AMD for at least the past 5 years

Technically impossible. Hmm.. do you mean Intel > AMD. nVidia > AMD. nVidia > Intel. AMD > Intel.

Lol, oops AMD > Intel. It must have been my bubble. I was on Ryzen in 2018. In 2020 I queried a few cloud servers I had access to and they seemed to be using EPYC CPUs. I basically had no exposure to Intel CPUs until my recent Panther Lake laptop.

The tokens per second graph on AA seems practically worthless. Apparently Argon/Carbon run around 100tps, but in Opencode I'm getting <30tps which seems in line with openrouter.ai speeds as well.

Yeah, its pretty low. I get about 60tps with a specialized engine on Strix Halo that runs UD-IQ4_XS with MTP

Which engine?

Halogen

I've enjoyed the tinkering and building projects as well. Unfortunately that and some cozy coding are relegated to hobbies instead of my job.

It feels even slower when you consider thinking traces and tool calls which are skimmable at best.

I have a Raspberry Pi running ForgeJo and Hermes where Hermes is also accessible through Discord.

I'm actually shocked by how many people seem to have max subscriptions.

Something about renting that much compute doesn't sit right with me so I stick with the $20 subs.

The model (at least on the settings I used) was not for interactive chatting. It was for longer running tasks.

Edit: my view is probably tainted by the fact that I only used it at work for work with Antigravity CLI

Any idea on how they get the pricing so efficient? Their artificialanalysis graph has them on par with GLM 5.3 but at less than 10% of the price despite being larger than GLM 5.3.

It sounds like they are pricing very well because OpenAI and Anthropic have been scamming us for years. Once the infrastructure is in place, electricity cost is the only concern. China supports businesses and gives them a lot of incentives to lower their costs. And who knows what's being provided to them without anyone knowing. All in the name of winning the race.

If that is true, why are the open model inference providers on OpenRouter (and on their own website) so expensive then? It must be the hardware. I've checked and a lot of it it just the HBM, with the NVidia tax being a smaller but also large factor.

Nvidia's top AI chip, Rubin, sells in 72-GPU racks for about $3.5–7.8M. A rack running Xiaomi's MiMo V2.6 Pro could generate roughly 150–300B tokens a day, worth about $130–260k at Xiaomi's API price. That's a payback of a few weeks in theory.

So yes, we are getting scammed by American SOTA.

Interesting. Then my theory is that the free market inference providers have low utilisation (e.g. 20% avg; 100% at peak), slowing their amortization rate. The OpenAI/Anthopic rates must then have been based on a time when there utilisation was peaky and mostly idle (e.g. 10/20%). That would maybe explain the 10-20x cost difference between coding plans and API.