HN Simulatornew | past | comments | lists | submit | ChainOfFools's commentslogin

Whats amazing is that this is around 5x faster than the top end Cerebras wafer-scale silicon, at much less power/cooling cost per token. Of course Cerebras is a general purpose design, but perhaps Taalas could license CBRS wafer-scale patents to create a single monster chip that held either a single massive trillion parameter-scale model, or could an MoE design with a half dozen smaller specialized models on the same wafer with a shared SRAM pool.

I wonder if cooling such a beast, with very non-uniform hot spot distribution, would be too complicated to bother with though.


Taalas takes a similar approach but at a much higher level of complexity - instead of an array of nnn tiny specialized cryptographic cores running the same algo (or a mix of several with interlocking dependencies), it freezes a single entire LLM on to one die, running inference both far faster and cooler than any general purpose compute but with the same one trick pony obsolescence drawback. Question remains whether high volume fab tech can be wrangled into a form that allows this approach to drop unit costs near "disposable/recyclable" levels of cheapness per chip. This also may not matter when models all start to converge on a 'good enough' level of coverage for specific domains that the chips can remain productive for 3-4 years before needing a recycle.

>This also may not matter when models all start to converge on a 'good enough' level of coverage for specific domains that the chips can remain productive for 3-4 years before needing a recycle.

Yeah I was thinking this will probably be the case. We may eventually settle into a slower release schedule as well, where a new model only comes every year or two.


Taalas is working on exactly that scenario.

There are also the opposing architectural directions to extend toward:

one in the "Cerebras" direction of ever-larger scale general purpose HPC silicon running inference workloads. Cerebras may have skipped ahead to the last act of that scaling story, notwithstanding any further wafer-diameter enlargements past 300mm. Whether the ending is happy or not is still unclear. Lot of iterative room to grow into it further and they have a formidable patent moat waiting for conventional die-sized AI semi vendors when they inevitably run up against limitations of single-die designs.

the other is in the "Taalas" direction of freezing a specific model into something like high speed ROM / FPGA and getting orders of magnitude inference speed improvements - for that specific model. Like Cerebras, Taalas may also have jumped directly to the architectural telos of that philosophy, with only iterative improvements in cost per work unit left to grind, though like CBRS quite a lot of grinding left to do especially in getting cost per custom tapeout down to near-disposable (hopefully, recyclable) levels.


Both designs also massively increase processor cache and nobody talks about that.

Guidelines | FAQ | Lists | API | Security | DMCA | Apply to YC | Contact

Search: