This argument seems somewhat strange to me, and at odds with the recent model releases.
It feels like claiming that because a current generation laptop CPU is slower than a current generation HEDT CPU, we should ignore that the laptop CPU is faster and more efficient than the previous generation HEDT CPU.
Of course a huge system could do more, but these systems are constantly evolving and just in this month we've seen comparatively smaller models approach capabilities that were frontier AI less than a year ago. What you're saying would be true when we eventually find the limits of how much models can compress knowledge. Until then, all I think you're saying is that companies that are willing to burn more money can afford to ignore efficiency.
It feels like claiming that because a current generation laptop CPU is slower than a current generation HEDT CPU, we should ignore that the laptop CPU is faster and more efficient than the previous generation HEDT CPU.
Of course a huge system could do more, but these systems are constantly evolving and just in this month we've seen comparatively smaller models approach capabilities that were frontier AI less than a year ago. What you're saying would be true when we eventually find the limits of how much models can compress knowledge. Until then, all I think you're saying is that companies that are willing to burn more money can afford to ignore efficiency.