I just switched to Vulkan, and be done with it. :)
As much as I can tell, the ROCm version of llama.cpp would be a bit faster on prompt processing, but about the same on the token generation as Vulkan. Real life benchmarks don't seem to give any "ROCm or nothing" sort of vibes. And the difference between the performance of different models are way bigger than the difference between the llama.cpp versions (and versus different runtimes like the llama.cpp/GGUF and the MLX runtimes on Mac for the same models)...
I've tinkered enough with the serving, that I'd rather do something with them with, say 10% slower speed, than spending hours on seting things up again... YMMV
Hipfire works on my 7900xtx best of all. Biggest surprise - Qwen3.6-27B dense does not grind to a halt with context depths all the way up to 250K! Measured at 1K, 8K, 32K, 64K, 128K, 192K, 250K - Hipfire speed holds close to 40 tok/s. Finished 5 days run of gpu 100% inferencing, Hipfire did not crash even once afaics. I assumed speed dropping like a stone on Qwen models was a feature/bug of the model, and "nothing can be done about it". Hipfire showed me wrong - pleasantly surprised there.
Atm OMP (oh-my-pi) with opencode-go subscription using deepseek-v4-flash-0731 (and mimo-v2.5 as advisor). Used pi before (still good, but too minimalist for all general use; works on Android under Termux btw) and OpenCode (which is ok too). I've used Codex the longest, but the $20 sub doesn't last that long. Before that Claude (and Claude on the web - about Nov'25), but nowadays with the new models not possible to do anything on the cheap sub. Got Zai (GLM-5.2) legacy sub to supplement too for variety. :-) Run local jobs on the 7900xtx (24gb vram - so dense models) batch bulk jobs. Non-dense models like nemotron-cascade-2-30b-a3b MoE run at 100 tok/s, looking forward to try the new lightning-3.5-30b-a3b. And got old MBP M2 Max 96gb uram (unified ram) for bigger sparse MoE models. Testing DS4F there under ds4 server/harness (too slow, 12-15 tok/s even with REAP25 reduced model), and (got agents) porting Ling-3.0-flash atm from llama.cpp -> ds4. Hoping only A5B (5B active, versus 13B of DS4F) will result in better tok/s. TBS. Back in Feb-Apr'26 looked local will be the only way to run llm jobs (plus 300w gpu job works out for office heating when cold), otherwise the ex-quota API payg costs would bankrupt me. But now with the oss weights models and providers like opencode - actually it's not only faster but also cheaper for me to run llm jobs on the api. Engines for local - llama.cpp for all, hipfire (mentioned) for the 7900xtx, and mlx on the apple ASI (that never quite work out for me). And ds4 trying forever when off-line :-) - e.g. on a flight - either ds4-agent, or ds4-server + agent pi. And I use Hermes as a general agent, non-coding agent, for everything that's not strictly coding.
Lemonade-server works pretty well (most of the time). It wraps llama.cpp and other runtimes - it downloads the official binaries as far as I could see, and you can set alternative versions if needed. Works nicely with Strix Halo for a while now.
