> if LLMs are used as compressors, how well is that expected to work
Quite well. This project[1], by Fabrice Bellard of ffmpeg fame, is quite old in AI years and uses an ancient LLM, but still beats xz by a solid margin.
A challenge as I understand it is reproducibility.
Normal LLM runtimes aren't typically fully reproducible even with same random seeds for distribution sampling, due to floating-point numbers, batching and such.
Though averaging over many runs could alleviate that I suppose.
While it would measure some aspects of intelligence, I'd argue it fails to capture other, more creative aspects.
Quite well. This project[1], by Fabrice Bellard of ffmpeg fame, is quite old in AI years and uses an ancient LLM, but still beats xz by a solid margin.
[1]: https://bellard.org/ts_zip/