I am using qwen3.8-27B-UD-Q3_K_XL on a 5080 16gb card with 16gb of system ram at around 43-45tok/s with 65k context. I am using that model to set up containers on a proxmox server with two b70s and 96gb ram. The Q3 model implemented 10 different chat models, 2 image models, and 7 web based harnesses so I can compare. It can rewrite any part to improve it.
qwen3.8 is more than capable for software dev. The gemma models were terrible and fell apart during compaction. I think as long as the llm can test the results, you don't need anything being offered by a "frontier" model. The cloud AI is going to be used by people unable to run their own and they will eventually get squeezed on price.
The one tip I could give is compact before starting new steps or any action in the plan that is different than what was previously worked on. You want to manage what is in the context and don't want unnecessary work history details filling it up. You can always ask the llm to list the current plan, then compact after and do it more than once until you get the compaction <30%. This will be fixable by the harness that can choose better times to compact.
You don't want to start a new phase and have it compact a few minute after starting. This happening over and over again seems to potentially cause issues for long running sessions. Compaction slop that screws up what is in the context.
I can compare what I use at home vs paid models like astra at work and the difference is mostly meaningless.
qwen3.8 is more than capable for software dev. The gemma models were terrible and fell apart during compaction. I think as long as the llm can test the results, you don't need anything being offered by a "frontier" model. The cloud AI is going to be used by people unable to run their own and they will eventually get squeezed on price.
The one tip I could give is compact before starting new steps or any action in the plan that is different than what was previously worked on. You want to manage what is in the context and don't want unnecessary work history details filling it up. You can always ask the llm to list the current plan, then compact after and do it more than once until you get the compaction <30%. This will be fixable by the harness that can choose better times to compact.
You don't want to start a new phase and have it compact a few minute after starting. This happening over and over again seems to potentially cause issues for long running sessions. Compaction slop that screws up what is in the context.
I can compare what I use at home vs paid models like astra at work and the difference is mostly meaningless.