It doesn’t. Decode is by far the most dominant and expensive part of agentic tasks and once prefill hits a certain achievable speed decode dominates the time in ant agentic flows with reasoning. There is a reason input is so much cheaper :)
Isn’t input so much cheaper because it’s just less work? Prefill is usually 10-20x as fast as generation.
If you have 200k context, but only 200t/s prefill, then you’ll wait multiple minutes when continuing an average session from yesterday. Or compaction - might take 15 minutes.
No commercial service has 200 t/s prefill though. It is 1000s of pps for prefill. It is largely compute bound and very very fast on modern GPUs and model architectures. That is my point. It is an order of magnitude faster than decode, which is largely bound by memory bandwidth and just has a much harder limit. Prefill tends to look like a small slice in typical compute / token generation budgets. I am not saying it isn't important (I have been working hard on my own inference engine for flash next and am intimately familiar). I am saying it does not dominate. Also compaction and reading the whole prompt tends to be one of the few times you really hit peak pps because it is cached and you can just fill your data pipes for a bit. Also Deepseek showed you can make kv cache really cheap. Decode is still decode.
The math. It’s pretty brutal. You have to multiply insanely large matrices in memory. A dense model has to have and multiply across every parameter basically. MoE models work around it, sure, but you still have to hold a lot in memory and density of activated parameters matters. There is not much of a way around it yet. We are potentially getting there, but its not obvious yet.
Bench is laughably out sync with reality too. I run things like flash next on an rtx 4080 that it says “won’t run” and its rates are just all over the place. Pure slop.
What would you prefer, 30t/s 50k output tokens needed for solution, or 300t/s, 500k output tokens needed for solution?
This is why "tokens per second" alone is a deeply flawed comparator.
This is also why discussions of Chinese models focus on tokens per second- they struggle to compete on intelligence so they dramatically increase effort levels to compensate, meaning they require massive total output tokens.
For example, flash 1017 isn't benchmarked yet, but 0731 required 15.4M tokens to complete the artificial analysis benchmark, and Sol 6.1 high required just 2.8M.
I certainly don't think the self-reported single benchmark scores are the best way to answer the question. I waited for the Artifical Analysis run (which itself is still mediocre, but at least it's an order of magnitude better), and predictably the results tell a different story.
Sol 6.1 High: 50 intelligence, $0.32 per task, 4.4 min per task, 13k output tokens per task
Sol 6.1 Medium: 48 intelligence, $0.21 per task, 2.9 min per task, 8k output tokens per task
Deepseek 4.1 1017: 45 intelligence, $0.26 per task, 4.8 min per task, 89k output tokens per task
If you look at the actual pareto frontier across the full benchmark, Sol 6.1 remains firmly on the pareto line, while deepseek remains firmly under it, just as it was before https://i.imgur.com/N72p5NF.png
This result is exactly as I expected, and seems to run counter to the effusive praise of this thread. Sol 6.1 Medium is producing higher quality output at a lower price with nearly 10X less output tokens! And this is not surprising.
And finally, Mimo V2.6 Pro continues to look like a stunning release, clearly smarter, cheaper and faster than deepseek v4.1 1017
"Sol 6.1 Medium is producing higher quality output ..." -- When it is actually up and working. Even last night I still received constant capacity messages halting codex over a dozen times and I gave up and shut the workflows down.
I would prefer it work and solve quicker. Opus 5.5 has a reasonable pace to me. Sol does not. I understand efficiency I just don't like using it right now because it has literally hung or failed to complete tasks a dozen times in the last week due to inference capacity.
Ehm.. I have a model that runs on my older gaming desktop that scores as well (or better) as this model does on DeepSWE. This model does better than Flash Next in many areas and I am sure is much stronger overall, but hmm.
Gemini Flash 3.8 was heavily optimized for agentic coding. It is not an ignored use case. Flash 3.8 benches extremely well on agentic coding. It was very intentionally made good at coding. So was Argon. It is doing a lot of mission critical porting work for things like Rust. Does Google also build models for things other than coding? Yes. It has a lot of enterprise customers that need good general models too. IDK, I haven't seen anything convincing in Flash 3.8 or Argon that say "Google is ignoring coding" it just took a while to get there since Pro 3.1.
(Disclaimer: I work at teh Goog and spend... a lot of time with these models :)
Quota allocations are a massive decision that go all the way up to Sundar: what GDM organizationally wants (training) vs what GCP wants (Anthropic to throw it money hand over fist) are not aligned and a balance has to be made. For various reasons, GCP carved out a significant share; an eleven figure check over a quarter or two carries a real weight. This doesn't mean that GDM was starved for compute or that it wasn't making significant progress, but it absolutely could have had a more impressive H1 if given more resources.
Am I misreading the Argon numbers then? It leads on DeepSWE and Vibe Code Bench but lags on FrontierSWE and Terminal Bench. Should we be viewing Argon as a frontier coding model?
It is a very efficient model, which I really appreciate. It can definitely feel a little slow. I tend to not hang around for it to do its thing, but for orchestrating and taking somewhat vaguely defined coding goals it is pretty amazing.
Sol is just hilariously slow. Even Luna is slow. It is unusable compared to Claude. I was pretty unhappy with Opus 5 and really gave OpenAI a shot for personal projects, but their quotas are just too small and their capacity issues in the last few weeks have just been untenable. No good. Opus 5.5 is just a very good model, very efficient with usage, gets things done at a reasonable speed. Sonnet 5.5 is also a very nice worker improvement. It is probably on par with Opus 4.6-4.8.
The sense I get from your comment and the GP comment is that OpenAI’s pricing changes alienated individual users for the sake of the enterprise market. I live in the enterprise market, and was quite pleased with the changes. We don’t really use Sol 6.1, just Astra.
$500/month for 40+ hours a week of Astra usage, with ultrafast mode available when you’re near the weekly reset has made our jobs so much easier. Our product has some sciencey bits as well, ands Astra kills it as a rubber duck in this area.
As a consumer whose 20x account got downgraded, I am a bit sad. I can’t justify $500 a month on personal AI spend. As an engineering manager, I could not be happier with OpenAI’s services. Also, Codex App Server is amazing.
I'm a bit confused by this comment. I thought Enterprise customers didn't have access to fixed price plans like the $500/month personal tier and instead paid per token? If that's available to businesses, Astra seems like a no-brainer. But at least the regular Business plans only show me the $25 and $125/month options and the latter is a very recent addition.
Hey, account created 39 days ago. Did you really have to tag that on after all the other stuff? Could you make it any more obvious?
That said this thread is hilarious because it's just an open ai sales manual debating with an anthropic sales manual. It's funny, but it's not, because the truth is that you people are just shitting all over the commons. Someone needs to kick you out but that's not going to happen here.
This product has made my life substantially easier. I have no affiliation with OpenAI.
Claude refuses to help you build your own harness off a subscription, and OpenAI actively encourages it with Codex App Server. So you’re just in an AI IDE customized to your job and specific needs ALL THE TIME. You know that good feeling you got when your CI pipeline was finally working, back when that involved speed reading documentation and guessing, or staying at work until 10PM? You just get that all the time because life is constantly improving. And your tools are beautiful and riddled with Easter eggs you put in them. There’s so much more time to test and talk to stakeholders because you are able to distribute the thing you are actually good at -solving people’s difficult problems- while following best practices. You used to have to weigh those two every day and it was terrible. And everything was always late. You had to cut so many corners you were delivering circles.
Now life is fun and I’m over-delivering. Constantly. With super high code quality, better test coverage, easier audit prep, and I just explain what I want (technically) at a high level to AI. I still have to code review, and it sucks. Not because the AI needs it, but because auditors need it. I think I can make it faster and easier in the near future with another custom tool.
I actually have time to make architecture diagrams. I have never had enough time to build that level of documentation.
With Fable, I felt the promise of this coming. With Astra, I feel like we finally had the resources to fully professionalize and supply our software team.
We actually need to hire more people next quarter.
I am with the creator of Rails on this. It is a time of joy. Now is the time to become more ambitious.
:shrug: I spend a lot of those token make local models better, resurrecting old hardware to make it more accessible and usable. Frontier models and AI is cool. This is my lived experience doing real systems engineering projects. My real sales manual is: https://stealthgoat.com/deepswe-flash-next.html local models are kind of awesome now and everyone should be using them in addition to frontier models. I will build the stuff I want which includes optimizing local inference engines until they are at my hardware ceiling. I use this capacity in my house to do reversing, security research, and internal OpenClaw type stuff I find useful.
They are all very slow at the moment. Some tasks are taking hours to even days. These are DevOps related tasks in which I create hundreds of servers in an agentic way. It works. But it's slow. That's still annoying.
reply