From a very high level view DSP's and GPU's are the same thing: highly programmable number crunchers. A better example would be Google's Tensor chip. It's not as general purpose as a DSP or GPU, it's more optimized to do inference.
That trend will continue.
I figure there are 6 order of magnitude events that could happen in the next decade to lower token prices:
- more specialized / better chips
- IC technology: smaller feature size, higher clocks, etc.
- more efficient algorithms
- solar power is getting cheaper at an order of magnitude per decade, batteries even faster.
- pricing pressure from open source models
- breaking of the Nvidia monopoly and it's 75% gross profit margin
Maybe all 6 won't happen, but certainly a 1000x reduction in price in the next decade seems highly likely. Jevon's paradox says that the 1000x reduction in price will likely result in more spend on AI, not less.
AMD bought Talaas specifically to make AI accelerator pieces to be embedded into generalized chips.
At work (we're a medium sized manufacturing firm), we bought our own inference server for $107k and run Kimi 2.8 for nearly all of our use cases (and dropped our cloud AI spend to $0).
You dropped your cloud AI spend down to the price of capital plus the cost of electricity and maintenance on that server. When/if tokens become a commodity, the price of tokens would be the marginal cost, AKA about the same. Big when/if, though.
That trend will continue.
I figure there are 6 order of magnitude events that could happen in the next decade to lower token prices:
- more specialized / better chips
- IC technology: smaller feature size, higher clocks, etc.
- more efficient algorithms
- solar power is getting cheaper at an order of magnitude per decade, batteries even faster.
- pricing pressure from open source models
- breaking of the Nvidia monopoly and it's 75% gross profit margin
Maybe all 6 won't happen, but certainly a 1000x reduction in price in the next decade seems highly likely. Jevon's paradox says that the 1000x reduction in price will likely result in more spend on AI, not less.