I agree that if you have the weights you can use/train a model with the same architecture, and that you won't get the exact weights on your own due to randomness. But isn't data an extremely important part of your ability to effectively train/finetune? It might be much harder to get close to the level of the open weight model if you don't have the data that made it, which is why I think the open weights vs open source distinction is useful.

You fine-tune the open-weight model without access to the original weights.

If you want to train it from scratch you need data, yes. But presumably if you are doing that there is a reason you want to do it.

You lose nothing without access to the original data - you can do every single modification without it.

That is unlike open source where you (mostly) need to source code to modify it beyond what the original designed originally thought.

"Kimi is largely a byproduct of distillation" and "Kimi is introducing new and novel approaches" are not mutually exclusive, and I'm not sure it's clear from the paper how much of the improvement comes from the new approaches. So I wouldn't take the new approaches to be much evidence about whether the distillation attacks occurred.

> distillation attacks

Like the distillation “attacks” Anthropic conducted against literally millions and millions of different creators?

Or against all those old books that they ingested and then literally destroyed

Reading Claude's outputs is an 'attack'

Even Anthropic hasn't claimed "Kimi is largely a byproduct of distillation"

There's been some distillation.

Just like how Elon said in court they distill to make Grok.

I thought there is discussion circulating around whether the main reason Kimi is impressive is due to distillation. Though possibly if this occurred, this was just one component of their training pipeline and not a majority

Hmm, why do you think this is true? One reason I'm skeptical of this is because RL envs are often purchased (and are not publicly available), and this might be a sizable component of why models are getting better.