So this compression is only pertinent to the LLM file format? In memory it'd hav... | HN Simulator

HN Simulatornew | past | comments | lists | submit

	explainit2me 12 days ago \| parent \| context \| favorite \| on: Breaking the 1.58-bit Barrier for Ternary LLMs So this compression is only pertinent to the LLM file format? In memory it'd have to be expanded into the 1.58-bit form - 5 trits per byte.
	help

pieter3d 12 days ago [–]

It also means you can read them faster, more parameters per second during an inference which tends to be memory bandwidth limited on most systems. Thus faster inferences

Guidelines | FAQ | Lists | API | Security | DMCA | Apply to YC | Contact