So can you not do structured output and get the log probs across all your allowed output classes? Then use that relative measure of how confident it is?
It's because they statically linked it, and probably rely on a trillion pointless crates (as is standard in Rust development). A good statically linked harness can easily be ~5MB MAX
I remember looking into Bitcoin in 2012, and thinking:
"This will never work as a currency, because of the max number of bitcoins it will inherently be deflationary, and the value will go up, which means that there will be a disincentive to spend it, because the value will just go up..."
I could have used BTC to buy into the ethereum ICO and I made up every excuse under the sun that a young kid who wrote shit python code could never compete with the gold rock.
Since then I've heavily revised my thinking. Primarily that I don't dismiss things as easily anymore. I try to focus on coming up with a reason to like it instead of disliking it.
Only state I keep is filesystem and last message (but even that is persisted in the filesystem). Each agent gets its own btrfs volume, when it’s done the “next” agent get the previous agent work mounted in its own filesystem ( and told about it ). Agent is also able to “promote” files if it wants and they are mounted in a more prominent place.
I don’t know yet if it’s a good solution. But only thinking in terms of files / filesystem sure make things easier.
Also makes branching “easier”: no handling of merging or conflicts, the receiving agent just gets N file systems and decides how to handle things.
So this means that every node in the graph has to be a fully fledged agent runtime with the smarts to handle an arbitrary filesystem.
I guess there is some sort of tool call that the agent has to tell the runtime 'I am done, and I succeeded go to the success node in the parent state machine'
---
I think I have been trying to approach a slightly different problem where you have a mostly deterministic control flow of steps/ nodes in the graph, but individual nodes can be pretty stupid/ small models, so having a more constrained universe should be more reliable. (My intuition)
My example is a citation checker for an essay -
1. generate a list of claims in a paragraph,
For each claim:
Read the attached citation, validate that it backs up that claim.
The output is a nested json structure with a list of claims with a explanation+binary classification of whether each one is supported.
This is not really agentic, and the nodes would be claim identification > validate claims (in parallel) > deterministically merge the validated claims into a unified module.
It should be very token efficient, and since it is so constrained I think I can get away with tiny models, and small context.
At least that is my hope... But I think I need typed IO/state so that the llm that writes the graph can deterministically say that the individual nodes complete the problem wholly.
This matches how i think about realtime voice: keep call control deterministic,
and give model steps clear inputs and outputs.
How would you handle retries when a step fails ?
For context, I read Claude output every single day, I know what it is reliable with and what it is not. Or at least I have a feel for how much I can trust it.
It may not be clear to a non-english first language person, but when I read Claudes documentation, my brain immediately picks up claude-speak. Therefore, I expect the code to be generally correct, maybe, depending on how specific I was during my prompting. In no way shape or form do I really trust it, at least until i dig into the code and validate my mental model. And query the review for edge cases etc...
By writing your documentation with Claude, my brain immediately associates the quality of the project with the quality of unreviewed Claude output.
To a degree this is unfair, as it is like judging the quality of someone's work based on their accent.
But Claudes accent, has a high correlation with Claude, so unlike with people, where an accent has no bearing on technical ability, Claude being Claude does.
I would much rather read a typo ridden sentence than claudism, even just as a forward, explaining what you did vs the ai, and telling the users how much we should trust it.
Also, If you really insist on using AI to write, have another model rewrite docs/comments into regular English (opus 4.6 for example is much better than 4.7, 4.8, or 5.
It is open source so you can do whatever you like, but people (especially native English speakers) will discount your work, because Claude, especially opus 5 writes very very badly.
The people who say they would rather see a typo infested mess of a readme are serious. Or even just write in your native language and then have Claude translate.
Both of those are better indications of proof of effort than a Claude readme.
> By writing your documentation with Claude, my brain immediately associates the quality of the project with the quality of unreviewed Claude output.
I'm sorry, but I agree with the author: if a certain writing style makes you associate the work with low-quality, then that's your problem. The author shouldn't have to rewrite the readme just to avoid triggering your automatic unfounded associations. If you look at the substance of the work, including the test cases, then this is clearly not easy work that can be vibe coded in a single pass.
It's just like emdash. Everybody digs on how it's a signifier of LLM text, but I've used emdash for years because it's gramatically correct. I shouldn't have to stop using emdash just to avoid kneejerk reactions.
LLMs often use emdashes in a distinct incorrect way. It's not just the existence of any emdash, although considering you were in a very small minority of older users, it now warrants increased scrutiny, unfortunately for you.
if 99 poor effort/quality projects have a readme that reads in a particular style, then you expect the 100th project with a readme in the same style to also be of poor effort/quality
statistically, it only makes sense for your expectations to immediately be low when you encounter this writing style because there are just so much slop out there
the author is free to keep that writing style but they should be aware that this will—at least on the surface level—make their project look exactly like the metric ton of slop we see posted everyday everywhere
That's like saying a large percentage of X-colored people are criminals, therefore when we see an X-colored person it's reasonable to expect him to be criminal. As a society we have decided that heuristics like this, and prejudicing individuals based on statistics of the group, are not acceptable.
Also, what do we call it when people don't look beyond the writing style, such as at the code, or at the rigorousness of the tests, or at other substances beyond writing style? Slop assessment?
I think the question is how do you keep track of the ideas that the agent is pursuing - like I was working on this for implementing a fft, and doing the optimizations, but I held its hand and was like - hey let's go back and retry this older thing you discounted because of a 3 % slowdown.
Those are all RPC mechanisms, not an ABI. They exist at a much higher level. You could have an RPC protocol be your only interface to the OS, and that would be a valid design, but would have a performance cost (see the classic Tanenbaum–Torvalds debate)
I'm not sure about the others, but COM is an ABI. There's a bunch of stuff surrounding it that is RPC-like but the core specification is just binary layouts and calling conventions. It's arguably more cross-language than the C ABI since it lets you generate type-safe bindings for any language, unlike in the latter where you need to parse header files.
> You could have an RPC protocol be your only interface to the OS, and that would be a valid design, but would have a performance cost
Maybe it would, but I doubt anybody would notice. The whole "everything is a file" concept on unix is basically just this (also X11/Wayland).
The subset of the list that supports in-process calls and which allows for zero-copy (and, realistically, probably mutually mutable) data sharing of its types is shorter still. I think the useful definition of "ABI" includes more than just API discoverability (aka what D-Bus provides if you take away the RPC system).