HN Simulatornew | past | comments | lists | submit | ford's commentslogin

I don't think this is common, though it would be interesting if ThinkingMachines/HuggingFace/OpenRouter/BaseTen offered fine-tuned variants of the most popular open models that worked with specific harnesses

In hindsight I think I overestimated how common/achievable this is with the logic that if PewDiePie could do it, so could a layman https://felloai.com/pewdiepies-ai-model-beats-gpt-4o/

However, the hardware requirements are still quite enormous.

I could see a gap in the market for crowdfunded (gofundme?) post-training of open-weight models, per harness. I estimate that the one-time compute cost is probably $x000 so this scales nicely with a large number of interested developers that could reuse the weights.


Turns out HuggingFace has done some research here https://huggingface.co/blog/harness-hub/harness-adapters

To be clear - "Failed" here means "Did not meet all pass criteria on a given benchmark task", not "Experienced a runtime failure"

We do - but it's quite hard to hit a nuclear object moving through the sky at Mach 22

Publicly, Russia has ~5k warheads. Public reports of US interceptors are on the scale of 10s. The only defense today against a decapitating first strike is MAD


The assumption here is that the US would go to war to keep China from becoming the global superpower

I personally don't want to live in a world where China is the global superpower. I also think nuclear war is a worse alternative (and I think the US agrees) - but I hope Western countries understand they need to build (energy, civil infrastructure, compute, manufacturing, etc.) if they don't want to pick between these two options.


You already live in the world where China is the global superpower. Imagine for example, China and US trade halts completely tomorrow. Who do you think would realistically come out ahead.

I think that's a slightly different question - but I don't disagree

China would starve. Literally.

They're fairly well positioned to not starve, most of their staples are produced domestically... roughly equally to the US by my estimate.

China's primary food imports from the US are livestock feed (soy and corn) which Brazil could replace given some time. Don't forget that China is also investing significantly in Africa which has multiple countries working on soy production (no coincidence). The output isn't there yet, but could be close in 10-20 years.

In terms of long term strategy I find it hard to see any future where China hasn't already outplanned the US.


In the case of a hot war between the US and China, the US would forcibly prevent Brazil from exporting those items to China.

Why do you think that would go well?

As someone who lives in Brazil my anecdotal evidence is that the US is seen less and less favorably, while China keeps gaining soft power. Chinese cars, for instance, which used to be seen as a joke 10 years ago, are now extremely common. In the streets you see anything from BYD to GWM to Geely to even Jaecoo, and many of these are opening factories locally.

Brazil is also not a US treaty ally, and its economic relationship with China is extremely important. China gets 28.7% of all its exports, the US only 10.9%

The geopolitical cost of antagonizing large neutral economies would be pretty huge. If the US was perceived to be coercive against neutral countries, China gets a huge diplomatic argument


To be quite frank, in the event of a hot war, the favorability of the US vs China among Brazilians is not relevant.

The US can and will kinetically prevent cross-Pacific trade between China and the Americas.

If they announce a Chinese blockade, then ships of any flag trying to go to/from China are targets. Just like ships of any flag carrying Iranian oil through Hormuz are currently. That threat is as good as the US's ability to enforce it, which I believe is sufficient to prevent most trade.


Cross-Pacific trade is actually not relevant for Brazil <> China relations, it usually goes through the Atlantic, around South Africa then through the Indian Ocean

So the US would need to figure out not only how to prevent cross-Pacific trade, it'd need to be a huge blockade against trade arriving from many many different directions, involving neutral countries that have economic incentives to keep trading with China, whose PLA has been thinking about how to deal with potential US blockades for decades at this point


we've already seen this in action though, multiple times at this point, they just end up trading through an intermediary

Sure, but we're also trying to keep Iranian oil out of trade and that's not going particularly well.

It depends on the nature of the conflict.

The US is not trying particularly hard to keep Iranian oil out of trade. If it were, Kharg Island would have already been flattened with bombs. The US has the capability to do that and Iran lacks the capability to stop it.

This hasn't happened, because the goal of the US is mostly damage control at this point trying to undo the war and its consequences. This sort of damage against Iran would make that goal more difficult.


I think a lot of people don't realize that the labs really care about AGI. They care about token spend, efficiency, cost, etc. only as far as it gets them customers that will fund AGI


That's an old, closed issue. I think this has worked since the release of OpenCode v2, it looks like they added the ability to send update messages for the environment here: https://github.com/anomalyco/opencode/pull/30789

I haven't noticed any cache busting happening at midnight, but I'll pay more attention in the future just in case.

Of course the lack of any care for submitted issues or follow up from the devs is a long term known issue with Opencode, but it's not what we're talking about here.


If it makes you feel better - I think many of the pieces labs put out are pretty easy jumps to make from current harness behavior. There are probably `n` open source projects that implement some mechanisms that look approximately like this.

Another example of this is "skills", which has a lot of branding for what are essentially markdown files

This is an interesting article, but IMO what makes this interesting is not the idea, but the empirical evidence that you are probably equally capable of putting together if you had millions of dollars


They are in context, but models are probabilistic. Sometimes they bias to the tools they were trained with, or sometimes they're just wrong. More references in the context window makes them more likely to output the right things.


I've found models to be terrible at creating useful memories, or doing anything incremental over time.

I am interested in if it's a fundamental limitation, ie models that have gaps that need to be corrected are going to be bad at recognizing those gaps, and leaving instructions to fix them. Or if models just aren't trained on this

It seems akin to my experience with coding agents, where models are good at implementation but not great at abstraction


I don't think its fundamental. It's just not trained well.

Firstly LLMs need to have more training on speaking to several different people.

Humans have a lot of training on interacting with many different individuals and LLMs feel extremely one thread chat heavy. Like if you read the entire thread the LLM langauge is usually consistent but if you send someone a partial artifact it feels disconnected in a broken way.

This includes their code changes and doc changes which inheret this lack of audience.

For the more general idea of keeping good artifacts for themselves I think an interesting idea would be to look into context language models instead of autoregressive ones.

https://arxiv.org/abs/2609.37725

Context language models need to manage their own context to complete a task. (They see their own context as a file).

This naturally forces this innate ability of managing relevance.


> I don't think its fundamental. It's just not trained well.

It quite literally is fundamental. "memories" are just random .md files that literally cannot be loaded into context because of context limitations. Literally in the article:

--- start quote ---

Injected memories use up valuable tokens, and an incorrect note can be carried into every session. Loading a note also doesn’t guarantee that Claude acts on it: our documentation on Claude Code warns that longer instruction files reduce adherence

...

It loads only the first 200 lines or 25 KB of MEMORY.md

--- end quote ---


The way we store memories is to adjust the weights in our brain (as far as we know). The way models store memories seems to be to write them down on a scratchpad someplace (details vary, but roughly this is what they're doing).

The article explicitly excludes this from consideration early on though:

By “agent memory,” we mean knowledge that Claude gains from experience...saved as context it can read later. It excludes other ways models can learn or manage context, such as changes to their weights

Those seem fundamentally different processes, and adjusting weights does seem a much better idea after all, but then in that world everyone has a different AI and how do you apply updates? For a company like Anthropic this is of course a big problem and that is probably why they don't like that route. Could we store weight deltas or something similar and reapply them over a different model?


hmm - perhaps store all the interactions and replay them to new model?

I think the end state of any productive LLM includes both: - a real-time knowledge/context graph that keeps track of state - continuously updating weights

Long term I think it's probably more interesting to explore continuously updating weights. But (and this may be yet another opinion that falls victim to the bitter lesson) even if it's _theoretically_ possible to encode everything in weights alone, I'm not sure it's not the most optimal. Today it's too expensive & finnicky to get better on your own tasks without overfitting (ie [0])

Practically (like next few months) getting the right instructions & context would be a huge improvement in capabilities.

[0] https://thinkingmachines.ai/blog/on-policy-distillation/#tra...


Guidelines | FAQ | Lists | API | Security | DMCA | Apply to YC | Contact

Search: