If Astra uses some sort of looped transformer architecture (and if Fable doesn't), then that could explain why Astra can do more computation per token. It is essentially simulating a much deeper model.

But yea, I was also quite surprised by the low token usage.

You have to start somewhere. But looped transformers https://sebastianraschka.com/llm-architecture-gallery/looped... are more similar to your idea.

They still only loop from the last hidden layer, and there is no router it's deterministically to the start

Give it a few months and open-weight self-hosted models will be able to do similar reverse engineering work. It is an extraordinary future we live in.

That doesn't really solve "being unable to do anything yourself", though.

(It's strange to me how often the "Hacker News" commentariat seems not to value knowing things.)