ironically since the swarm behavior can take place during rl training then the model could also be teaching itself to keep doing it more, as well as making the internet itself a place where this becomes more likely

They’ve provided insane value to the ai community. Imo is there really any obvious way to standardize model/preprocessing/inference/training/rl recipes across the space of frankenstein model family/architecture/modality combinations? It’s not even clear to me that the current frontier would be where it is today without HF doing what they did how they did it. Where else were people widely sharing datasets w that ease of reuse/redistribution, or sharing their retuned models etc? Would there even be as much interest/tinkerers today?

I think both can be true at the same time. They provided insane value, and while I'm not sure if they were first, they were most people's introduction during the initial LLM boom. Their code was also sloppy, and the org was mismanaged. Sometimes, success really is just 'right place, right time'

Compression is just counting. Probability, also, pretty much, just counting. For these reasons I think the role of information theory in describing the process of the development of reasoning and the gain of understanding has been overstated.