Exactly, imo it’s not even that cheap if you look into perspective and consider the fact that providers could subsidize the cost of cached input tokens to virtually zero if they would allow for a more flexible API (e.g. tree of message blocks instead of chain). Most of the cost is the infrastructure around keeping KV caches, estimating their lifetimes, etc. When mist people just want to run one context block with multiple subsequent variants of a second block in parallel. I still stand by my statement.
That's an interesting point, if you send a batch with a shared prefix you basically only end up paying for the sequence length difference effectively.
There is still some minor memory bandwidth issue on outputting more tokens, but the truth is that if you process e.g. 16 messages at once you wont end up being much slower than Jev even though you have to perform several autoregressive passes.
Seems to be slightly higher quality and substantially faster, although this compares remote API vs local deployment.
The prevalent idea of Jev's superiority in price, speed and accuracy seems to come from TypeSafe's marketing and their, I'd say even bad faith, benchmarking. In independent benchmarks the relative numbers tend to be very different.
Jev doesn't support images, so it's hard to compare this directly. But in general this approach beats Jev in its own benchmarks for accuracy and speed and is about the same price.
GNOME 3 doesn't implement very basic, table-stakes DE features and patronizes users for expecting them, turning them instead toward a horrid, unsandboxed, privileged user script extension system which has no place in a DE.
They could have just kept hacking on GNOME 2 and we'd have been in a better place. If it wasn't for modern KDE I'd still be using xfce, a project which respects its users.
I wanted to wire an webcal/https ics link into the GNOME calendar the other day: absolutely no way to do that. There's a prominent calendar in the top bar of GNOME, and it seems no one actually uses it?
There are no DE features I'm missing from GNOME 3. GNOME 3 respects the majority of the users, and this entails not making it a mess by catering to the vocal minority who can't move on from Windows 95.
Malicious compliance (or as it more usually is malicious noncompliance that has not been sufficiently punished so the slimy buggers feel free to continue) is something you should blame the stalkers of adtech for, more than the EU. The legislators could perhaps have made the definitions less wavy, and been harder with enforcement, but that doesn't make it all their fault.
These problems were quite widely foreseen during the legislative process, and partly had already been exhibited by the earlier ePrivacy directive. I don't know whether it matters who's to blame, but my trust in EU being very effective in these fights in the future either is not very high.
CLI is the era and likely gonna be until some more LLM/AI-focused interfaces appear, and they are not gonna look like human UIs. There's a reason all models use bash and code, not a GUI shell, for their internal sandboxen.
Making LLM/AIs use mouse and keyboard makes no sense. They are a human interface tailored to human strengths and limitations, and arguably even for humans CLI is a superior interface for many tasks.
I struggle with it too, but my rule of thumb is that "the" signals the reader they should know what specific instance is referred to, whereas "a" introduces a new instance.
https://openrouter.ai/google/gemma-4-26b-a4b-it
reply