I don't think this is common, though it would be interesting if ThinkingMachines/HuggingFace/OpenRouter/BaseTen offered fine-tuned variants of the most popular open models that worked with specific harnesses
However, the hardware requirements are still quite enormous.
I could see a gap in the market for crowdfunded (gofundme?) post-training of open-weight models, per harness. I estimate that the one-time compute cost is probably $x000 so this scales nicely with a large number of interested developers that could reuse the weights.
We do - but it's quite hard to hit a nuclear object moving through the sky at Mach 22
Publicly, Russia has ~5k warheads. Public reports of US interceptors are on the scale of 10s. The only defense today against a decapitating first strike is MAD
The assumption here is that the US would go to war to keep China from becoming the global superpower
I personally don't want to live in a world where China is the global superpower. I also think nuclear war is a worse alternative (and I think the US agrees) - but I hope Western countries understand they need to build (energy, civil infrastructure, compute, manufacturing, etc.) if they don't want to pick between these two options.
You already live in the world where China is the global superpower. Imagine for example, China and US trade halts completely tomorrow. Who do you think would realistically come out ahead.
They're fairly well positioned to not starve, most of their staples are produced domestically... roughly equally to the US by my estimate.
China's primary food imports from the US are livestock feed (soy and corn) which Brazil could replace given some time. Don't forget that China is also investing significantly in Africa which has multiple countries working on soy production (no coincidence). The output isn't there yet, but could be close in 10-20 years.
In terms of long term strategy I find it hard to see any future where China hasn't already outplanned the US.
As someone who lives in Brazil my anecdotal evidence is that the US is seen less and less favorably, while China keeps gaining soft power. Chinese cars, for instance, which used to be seen as a joke 10 years ago, are now extremely common. In the streets you see anything from BYD to GWM to Geely to even Jaecoo, and many of these are opening factories locally.
Brazil is also not a US treaty ally, and its economic relationship with China is extremely important. China gets 28.7% of all its exports, the US only 10.9%
The geopolitical cost of antagonizing large neutral economies would be pretty huge. If the US was perceived to be coercive against neutral countries, China gets a huge diplomatic argument
To be quite frank, in the event of a hot war, the favorability of the US vs China among Brazilians is not relevant.
The US can and will kinetically prevent cross-Pacific trade between China and the Americas.
If they announce a Chinese blockade, then ships of any flag trying to go to/from China are targets. Just like ships of any flag carrying Iranian oil through Hormuz are currently. That threat is as good as the US's ability to enforce it, which I believe is sufficient to prevent most trade.
Cross-Pacific trade is actually not relevant for Brazil <> China relations, it usually goes through the Atlantic, around South Africa then through the Indian Ocean
So the US would need to figure out not only how to prevent cross-Pacific trade, it'd need to be a huge blockade against trade arriving from many many different directions, involving neutral countries that have economic incentives to keep trading with China, whose PLA has been thinking about how to deal with potential US blockades for decades at this point
The US is not trying particularly hard to keep Iranian oil out of trade. If it were, Kharg Island would have already been flattened with bombs. The US has the capability to do that and Iran lacks the capability to stop it.
This hasn't happened, because the goal of the US is mostly damage control at this point trying to undo the war and its consequences. This sort of damage against Iran would make that goal more difficult.
I think a lot of people don't realize that the labs really care about AGI. They care about token spend, efficiency, cost, etc. only as far as it gets them customers that will fund AGI
That's an old, closed issue. I think this has worked since the release of OpenCode v2, it looks like they added the ability to send update messages for the environment here: https://github.com/anomalyco/opencode/pull/30789
I haven't noticed any cache busting happening at midnight, but I'll pay more attention in the future just in case.
Of course the lack of any care for submitted issues or follow up from the devs is a long term known issue with Opencode, but it's not what we're talking about here.
If it makes you feel better - I think many of the pieces labs put out are pretty easy jumps to make from current harness behavior. There are probably `n` open source projects that implement some mechanisms that look approximately like this.
Another example of this is "skills", which has a lot of branding for what are essentially markdown files
This is an interesting article, but IMO what makes this interesting is not the idea, but the empirical evidence that you are probably equally capable of putting together if you had millions of dollars
They are in context, but models are probabilistic. Sometimes they bias to the tools they were trained with, or sometimes they're just wrong. More references in the context window makes them more likely to output the right things.
I've found models to be terrible at creating useful memories, or doing anything incremental over time.
I am interested in if it's a fundamental limitation, ie models that have gaps that need to be corrected are going to be bad at recognizing those gaps, and leaving instructions to fix them. Or if models just aren't trained on this
It seems akin to my experience with coding agents, where models are good at implementation but not great at abstraction
I don't think its fundamental. It's just not trained well.
Firstly LLMs need to have more training on speaking to several different people.
Humans have a lot of training on interacting with many different individuals and LLMs feel extremely one thread chat heavy. Like if you read the entire thread the LLM langauge is usually consistent but if you send someone a partial artifact it feels disconnected in a broken way.
This includes their code changes and doc changes which inheret this lack of audience.
For the more general idea of keeping good artifacts for themselves I think an interesting idea would be to look into context language models instead of autoregressive ones.
> I don't think its fundamental. It's just not trained well.
It quite literally is fundamental. "memories" are just random .md files that literally cannot be loaded into context because of context limitations. Literally in the article:
--- start quote ---
Injected memories use up valuable tokens, and an incorrect note can be carried into every session. Loading a note also doesn’t guarantee that Claude acts on it: our documentation on Claude Code warns that longer instruction files reduce adherence
...
It loads only the first 200 lines or 25 KB of MEMORY.md
The way we store memories is to adjust the weights in our brain (as far as we know). The way models store memories seems to be to write them down on a scratchpad someplace (details vary, but roughly this is what they're doing).
The article explicitly excludes this from consideration early on though:
By “agent memory,” we mean knowledge that Claude gains from experience...saved as context it can read later. It excludes other ways models can learn or manage context, such as changes to their weights
Those seem fundamentally different processes, and adjusting weights does seem a much better idea after all, but then in that world everyone has a different AI and how do you apply updates? For a company like Anthropic this is of course a big problem and that is probably why they don't like that route. Could we store weight deltas or something similar and reapply them over a different model?
I think the end state of any productive LLM includes both:
- a real-time knowledge/context graph that keeps track of state
- continuously updating weights
Long term I think it's probably more interesting to explore continuously updating weights. But (and this may be yet another opinion that falls victim to the bitter lesson) even if it's _theoretically_ possible to encode everything in weights alone, I'm not sure it's not the most optimal. Today it's too expensive & finnicky to get better on your own tasks without overfitting (ie [0])
Practically (like next few months) getting the right instructions & context would be a huge improvement in capabilities.
reply