Yes, they have been near 100% utilization on vast.ai at $1 to $1.25 depending on demand for the past two months. They're in two separate workstations, which I purchased for around $12k each before GPU and RAM prices went nuts this year.
Now, I have absolutely no clue how long this situation is going to last! But the economics don't really work out for local models while it does.
You can buy 2 of them from Central Computer any day of the week for $30K US, and that's not the lowest price I've heard lately.
So if you can spend $30K and immediately start mining $1500/month out of thin air, that's a pretty nice investment even if the electricity costs $200/month. Two years later the cards will have paid for themselves entirely and (I suspect) will still be pretty useful.
It does argue in favor of just paying OpenAI or Anthropic for tokens, though.
You do need fairly high end workstations to put them in, adequate cooling, power and very solid business internet as well. I've put more work into keeping them online than I had expected. But overall the economics are fantastic for it right now, the big question is how long it will continue.
How important is it that you maintain a good availability record? I have a few RTX 6000s in a server that I use occasionally for NDA'ed contract work and for experimenting with open models, and now I feel stupid for just letting it idle at ~1 kWh when it could be earning revenue. I didn't realize that anyone would care to rent prosumer-grade GPUs.
At the same time, when I need to use the hardware for something, whoever is renting it from me at the moment is going to get unceremoniously booted, and I imagine they are not going to be happy about that. I assume that vast.ai's providers get uptime ratings that drive their work allocation, right?
Yeah - that is the issue. Doing it that way is going to nuke your reliability rating and you will have to cut prices drastically to get rented.
What I do is… rent on the same platform when I actually need to use a card. A benefit is that if I need a burst of more power, that’s not an issue since it’s available from other hosts. But obviously there’s an inflection point of first party utilization where it makes more sense to own.
