I think it's fun back-of-the-napkin sometimes to compare meat to matmuls but ult... | HN Simulator

HN Simulatornew | past | comments | lists | submit

refulgentis 36 days ago | parent | context | favorite | on: Dust: Pretraining Transformers Without Backpropaga...

I think it's fun back-of-the-napkin sometimes to compare meat to matmuls but ultimately fallacious to its core, making it an intellectual tarpit.

It's much harder to argue with the math and empirical results.

abecedarius 36 days ago [–]

Energy cost may be a pointless comparison since we don't eat electricity, but it's very reasonable to look at sample efficiency and ask what we have there that our theories miss.

Guidelines | FAQ | Lists | API | Security | DMCA | Apply to YC | Contact