GPUs and inference ASICS also have large amounts of high bandwidth memory, plus lots of high speed storage cache, and dedicated very high bandwidth scale-out and scale-up networks. Because they are also often bound by I/O bandwidth.

If your problem is grepping crazy amounts of data, the infrastructure for LLMs isn't a bad place to look for an example.

> plus lots of high speed storage cache

I wouldn't be surprised if that's what Apple is focused on for their next generation platforms – I wonder if more layers of caching between their SSDs and unified memory are on the cards.

Bots have been unable to beat even pretty good amateur humans despite those advantages so it's still pretty interesting.

bot is at the top of the kr ladder right now (excluding drop hackers) https://www.youtube.com/watch?v=1vsTqNwHquE

The Astra one looks pretty good except it's standing on the wrong side of the table

Which to me puts it firmly in the slop category. Why do people think this is good?