For the LLM use, the challenge is that it will only discover those after first requesting and parsing the HTML version.

Maybe it will notice those, and maybe it will figure out the pattern for follow-up page requests, but there's no guarantee and it won't help the first request.

Not necessarily. They could also send a HEAD request to the URL first, to see the headers only and decide on the available alternates.

I am well aware that few sites are taking that much care of their API in terms of HTTP features, but all of the problems discussed here have solid and battle-tested answers.

The parent was referring to adding `link` tags in the ``, not in HTTP headers.

I realised that after posting, but then I suggested sending them as header in the first place.

That only works if the client looks at it. The current Claude fetch system does not.

Should we let vibe-coded agent harnesses dictate protocol design now..?

On the flip side, I'd argue that the current centralisation of user agents (in the classical sense here) that benefit from programmatic content negotiation in form of a handful of harnesses like Claude or Codex is a great lever toward forcing the ecosystem to adopt better practices: If Anthropic added content negotiation as described in this thread to Claude, many sites would be incentivised to improve their web servers.