Public benchmarks are easy to cheat, if I am typesafe I would also release a public benchmark to distract otherwise competent people in overfitting to a benchmark instead of making something actually useful.
Diogo very much is against public benchmarks ;)
You take Qwen3.6 35b on a 5090rtx, and here you get a higher score than Jev, for 2sec more latency on average.
So yeah it's not subsecond, but I am sure that if I had VC money, I could too get within 500ms too!
reply