I don’t know much about gaming GPU workloads and how they differ from LLM inference workloads, but I’m definitely rich enough to pay for an LLM subscription and definitely too poor to afford the GPUs it would take to get anywhere near that level of inference.
That makes sense. But is the price difference between doing it on home hardware versus cloud hardware less significant than it is for LLM inference? And if so, is it mostly because LLMs need much more VRAM (or more exotic memory architectures)?
