I really like Muse Glimmer, and on low reasoning level it's both helpful and quite quick.
But I won't turn down a model that is faster.
Nemotron Bolt appears to be really quite good but unfortunately it is quite slow on macOS/Metal in llama.cpp at the moment because of lack of support for a tensor type (or something… not sure I understand).
I've been using Gemini Desktop App purely for dictation. It's a miracle! For the first time in my life I'm blown away by the quality of my (heavy accent) speech recognition. Just be sure to disable the "speak to window -> reasoning" option to make it purely dictation and stop from writing whole emails for you.
- 125 tokens per second on an M4 Pro MacBook Pro
- 512k tokens context window
- 40B total, 2.5B active parameters mixture-of-experts
Pretty cool release!
reply