It is strange that they also benchmarked an M2 Ultra, but they are specifically not releasing a Mac runtime today. Maybe that is still under development.
I've also gotten a lot of similar spam emails from people scraping popular GitHub projects to find contributors' emails, then referencing such and such work that you did on XYZ project.
Please stop. This is not an acceptable way to perform cold outreach.
If you're going to use OpenRouter to test reasoning levels, always make sure you are locking to the official provider instead of third party providers.
Pretrained dictionaries have never been intended to help with book sized or bigger compression. zstd automatically learns the most efficient dictionary it can within a few kilobytes. Pretrained dictionaries are only useful when you're independently compressing very small records.
Please run GLM-5.3 and GLM-5.3-Flash. I would love to see how they do. On the smaller end of things, Qwen3.8-27B and Ling-3.0-Flash would also be interesting.
In the benchmark, have you considered instructing the models to build their own SPICE simulations to test their work? Simply asking them to write and run simulations could improve performance, even without telling them what to simulate.
FunctionGemma never worked well for me (without fine tuning). Liquid has released 230M and 350M models that work far, far better in my testing: https://huggingface.co/LiquidAI/LFM2.5-230M
I really look forward to a hypothetical LFM3-230M, because LFM2.5-230M is so close to being usable, while FunctionGemma is miles away from being usable.
It is strange that they also benchmarked an M2 Ultra, but they are specifically not releasing a Mac runtime today. Maybe that is still under development.
reply