Opened the RTL, looked at the floating point math, learned that apparently you don't need correct floating point operations for LLMs, closed the page.

Well, you don't. That's why 1.58-bit (ternary) quants are often used. But if that's what they're going for, no need to dress it up in a floating-point facade.

Indeed there are some bugs/"non standard behavior" regarding very small or very big floating point values. All of those where proven harmless for LLM inference. Thanks for pointing out the bug. If you caught something outside of that, please point that out so I can fix it ;)

> All of those where proven harmless for LLM inference.

Could you share link(s) to those proof(s)?

Yup, the tests already compare to a CPU hugginsface implementation, but I'm working on making running and verifying the results of those tests easier and more available. Also, working on fixing those bugs and make the code more reliable now that the underling hardware has been saturated (memory bound)

Finally, thanks for stating the obvious! I was starting to wonder if I was not going insane.

A team of senior elite chip designers builds something they already know how to build and everyone goes "wow AI be so powerful".

I wonder if they will be inviting him to the next RISC-V design committee meeting.

For a friendly meeting, like Julius Caesar had on March 15, 44 BC.

"This time its different".

> slighly crazy

What a lovely euphemism.

Signed: someone slightly crazy.

Spot on, while reading this post I couldn't help be wonder "but what about the prototypes"?

No AI was used either for the project itself or for the post. I encourage you to look at the code and the git history.

No AI was used either for the project itself or for the post.

Sounds like they could have used you on that team. Bad luck, but it at least you can walk away proud knowing it was not an issue with your technical skills. :)

You can join multiple Tiny Tapeout tiles (grid squares) together to form bigger projects, which is what I did here. It's actually one of the bigger 4 grid projects.

Also how much you can fit really depends on the node the shuttle is targeting. So I was in luck that this was on Global Foundries 180nm as you can cram in a lot for flip-flops onto a single tile compared with Skywater 130nm. On the other hand the gates are much slower than on Skywater 130nm.

Hopefully both programs will be around for a while, so if you are still interested when you have more time you can join the shuttle then.

You can check out the tiles for this project and their exact location on the TT chip[0].

[0] https://tinytapeout.com/chips/ttgf26b/tt_um_coffeepot

Nice, thanks for posting the link. For the people unfamiliar with the Tiny Tapeout program: everyone that gets a copy of the Tiny Tapeout chip will be getting a copy of all the ASICs on the shuttle.

True, I went over the EDA stack a bit in a previous article here: https://talesonthewire.com/projects/blake2s_hashing_accelera...

This used the librelane classic flow, so it's a lot of verilator, yosys, openroad, opensta, magic and klayout. :)