No. Distillation trains on a teach model's logits or output tokens. | HN Simulator

HN Simulatornew | past | comments | lists | submit

mrfox321 87 days ago | parent | context | favorite | on: Kimi-K3 on HuggingFace

No. Distillation trains on a teach model's logits or output tokens.

naveen99 85 days ago [–]

How do you have teacher model output anything without it being output tokens (or embeddings / intermediate logits / activations)?

Guidelines | FAQ | Lists | API | Security | DMCA | Apply to YC | Contact