That sounds really quite interesting- one question pops up, this sounds like it could produce a lot of non-cache input tokens. How much impact does this have, and are you mitigating it?
Interested in adding support to my own 3code- https://3code.capocasa.dev- which aims to reduce cost by using more compact system prompts and a clever compaction variant.
I did some token tests and both pi and my own 3code used 4x fewer tokens than Claude Code, while opencode used 2x fewer tokens than Claude Code. You can tell Anthropic just doesn't care about using tokens and the tradeoff is maximal performance. That's understandable, but it might not be the tradeoff you need, hence, alternative clients.
Nice! I've seen it around on the Nim forums. Is it terminal only or do you plan to make use of Nim's JavaScript backend at some point to make a web UI too?
Yeah, that's fair. I was thinking only locally. However, I would hope they are dropping both stages somewhere relatively responsible. I guess I don't know where that is for polar orbits.
I saw that very much developing 3code. Even slight system prompt variations can have huge differences in results, so I have one system prompt per model family.
Interested in adding support to my own 3code- https://3code.capocasa.dev- which aims to reduce cost by using more compact system prompts and a clever compaction variant.