What are people using for a local offline harness? I've heard that OpenCode has lax approach to security and has some default dial home settings. Aider? Want to use Qwen3.8-27b locally for coding on 24gb VRAM. Best option?
I'm in the same boat. I was the primary advocate for fixing the config security regressions in the OpenCode tool (separate from the console hack days after the fix). My impression is that they are improving quality across the board. I will not being updating to v2 because I do not jive with the new UX. I would recommend checking it out, it's the best open harness I've used so far, though I'm only interesed in TUIs.
I'm currently using the Pi harness as well, it's closer to OpenCode v1 with a few plugins. However, I have zero faith in their security or maturity after several communications with the CEO. Pi is also developed by an Israeli team, if that is something that matters to you, re: Mossad concerns or Palestinians rights.
I'm going to go back to my own custom harness and do a full rewrite, now that I have seen what 6+ other harnesses offer.
How do we provide org / ops determined configuration that users and agents cannot change while still giving them some flexibility to set the values we do not determine?
We may both be partially right, the main dev I interacted with has Tel Aviv on his GitHub (also 'Hanna' being around the project, but apparently that is a surname not a first name, and a mistaken extrapolation on my part)
The lack of professionalism or seriousness was mainly in a private email chain with Armin. He added a single comment to this issue after that chain which was the gist of a couple of his replies, less snarky. Either way, the interaction left me disappointed, not a founder who seems interested in listening or learning imo.
I wrote my own harness designed to feel like a unix util. If you're worried about security, mine is a single file shell script less than 1,000 loc. It only depends on llama-cli and a few posix shell tools. So it has no way to dial anywhere and no security problems hiding in some dependency.
But it's not optimized for coding specifically. It can edit files fine using Aider style edits. I've used it to write to do lists as a test. But I more designed it with the idea to be my personal debugging tool if I ran into any weird things while using Linux on a laptop.
Thanks! I've been fiddling with my own single html file mini-harness myself, not coding ready yet, but been fascinating to better understand how some of these models work.
I heard in a youtube video of a conference talk that many of the more modern models have been designed to output correct JSON. Like the model creators have done something with the next token prediction. When you are asking for JSON, they do some special stuff to basically force the model to output correctly escaped JSON. In the past, getting all output as JSON could lead to malformed JSON and parsing problems. With modern models that basically does not happen any more.
If you are doing a single HTML file harness, you have access to javascript. You might consider getting all model output as JSON. Should be pretty easy for you to parse and then use the output with javascript.
Dealing with JSON in posix sh would have been too many loc for my harness. So I used other techniques. But you don't have to do that.
It just barely fits in my RX 7900 XTX's 24GB of VRAM (TBH i'm not sure if dflash is much better than MTP - which is included as part of the GGUF - and MTP saves ~1GB of VRAM). I'm using Linux with Xorg/Window Maker as my DE so it uses no VRAM at all (Qt-based stuff - and i guess modern Gtk too - tend to use some VRAM for themselves, even opening Kate shows up in amdgpu_top :-P). The above gives me on average ~50 tokens/sec generation (drops to about 35 when the context is close to full) and ~700 tokens/sec prompt processing, which while not particularly speedy is fine for agentic stuff as i just give the AI a task and leave it in the background doing things. To save some VRAM i run the multimodal projector for vision on the CPU (the --no-mmproj-offload) and only one task is ever executed at the same time (--parallel 1). Also the kvcache is quantized to q8_0 - supposedly this makes the model a bit dumber, though the extra VRAM needed for the full f16 kvcache makes it more than ten times slower :-P (and AFAIK dense models like the 27B aren't as affected by it as MoE models are).
I'm using Pi[1] (a slightly older version that i've had remain offline and disabled the telemetry) for now (will replace it at some point since i'm not a fan of the ~500MB dependency tree, the almost-dark patterns for online access/telemetry and the increasingly worrying push for the devs' services, but i do like how it approaches compaction and that for the most part it doesn't need any handholding when some error happens - which is ideal for "let the AI do something and check it some time later").
It is running inside a container so it doesn't get any access to my real stuff (mainly to avoid the LLM making any mess - of which it sometimes does in the container, but well, it is contained :-P). In the container Pi gives the LLM some basic tools - read, write, edit and bash - and the last one is basically used to do most of the work (i've also made a bunch of tools the LLM can use via the CLI).
Yeah though AFAIK it is discontinued now (or at least i can't find it in the local price aggregator). I think R9700 is a decent alternative, a bit slower (256bit bus vs 384bit bus) but has 32GB of RAM, still isn't too overpriced (at least compared to Nvidia alternatives) and has an opensource stack too (which is why i went with AMD in recent years).
It was a good entertaining movie. I found all the online right-wing hysteria about this really silly and stupid. The moral of the story should actually appeal to reasonable moderates everywhere: Zeus's Law, whereby strangers (refugees/immigrants) are welcomed, should not be taken advantage of, ie when the strangers (refugees/immigrants) misbehave and spit in the faces on their hosts, lest they break the culture and tolerance that allowed them to be welcomed in the first place.
> But there were unspoken conditions to our acceptance, and that was the secret we were meant to glean on our own: we had to be grateful. The hate wasn’t about being darker, or from elsewhere. It was about being those things and daring to be unaware of it. As refugees, we owed them our previous identity. We had to lay it at their door like an offering, and gleefully deny it to earn our place in this new country. There would be no straddling. No third culture here.
> That was the key to being embraced by the population of our town, a community that openly took credit for the fact that we were still alive, but wanted to know nothing of our past. Month after month, my mother was asked to give her testimony in churches and women’s groups, at schools and even at dinners. I remember sensing the moment when all conversation would stop and she would be asked to repeat our escape story. The problem, of course, was that they wanted our salvation story as a talisman, no more. No one ever asked what our house in Iran looked like, what fruits we grew in our yard, what books we read, what music we loved and what it felt like now not to understand any of the songs on the radio. No one asked if we missed our cousins or grandparents or best friends. No one asked what we did in summers or if we had any photos of the Caspian Sea. “Men treat women horribly there, don’t they?” the women would ask. Somehow it didn’t feel OK to tell them about my funny dad with his pockets full of sour cherries, or my grandpa who removed his false teeth when he told ghost stories.
