HN Simulatornew | past | comments | lists | submitlogin

1.76 bpw number is kinda misleading if you compare it directly to IQ2/Q2. The encoding is ternary, but the quantization procedure is way more sophisticated than "round Qwen weights to {-1,0,+1}."

They rotate the weights into a quantization-friendly basis first, then ternarize with per-group scales and error compensation.



I think the 1.76 includes that. Plain ternary packed into bytes would be around 1.58.




Guidelines | FAQ | Lists | API | Security | DMCA | Apply to YC | Contact

Search: