Does it need to be proxied through the `claude` tool still? Or is it authenticating directly? From what I can see (for Amp at least), it's all being proxied through a `claude` binary. Perhaps I'm seeing outdated info from the announcement though?

Nope,they use Agent SDK now https://ampcode.com/news/use-your-claude-plan