I think they dumb down their public models to be only slightly better than the competition. And the real competition is China, so the current state of the Chinese models would define the baseline.
I think one evidence is that the US has more than 5x the compute of China. With that difference in training speed, it should be impossible for Chinese models to close the gap that easily. It's also very unlikely that they sell the same public models to their private customers (military etc). We also know they talk about "unpublished internal models" for things like the last HuggingFace hacking incident. So it's not a bad theory.
> I think one evidence is that the US has more than 5x the compute of China. With that difference in training speed, it should be impossible
How could we really know how much "compute China has" in reality? Is it possible that whatever estimates people has come up with for both China and the US might not be 100% accurate?
I'm not an expert but I think this sort of thing is relatively traceable for two reasons. One, datacenters are difficult to conceal. Two, the supply chains for many of the relevant materials are difficult to conceal. Some of those supply chains still require western components, I believe, so if you know how much of X component was sent to china, you know how much compute they have.
In China and in the US most owners of computing power have to quickly gain from it, as obsolescence hits hard. In China a consensual will emitted by powerful companies may convince the central power to subsidize efforts towards int'l market domination: R&D, including dataset building, learning... Maybe even also low prices obtained by selling at a price inferior to the costs...
I think one evidence is that the US has more than 5x the compute of China. With that difference in training speed, it should be impossible for Chinese models to close the gap that easily. It's also very unlikely that they sell the same public models to their private customers (military etc). We also know they talk about "unpublished internal models" for things like the last HuggingFace hacking incident. So it's not a bad theory.
https://epoch.ai/publications/trends-in-ai-supercomputers