I also suspect the questions asked matter a lot, and the system prompts matter a lot, because "the map is built from nothing but the words they choose" - so this is more a measure of linguistic style than anything.

If you use the Claude Code harness on two models you will probably get very similarly styled output. I would not be surprised if K3 "stole" a lot of the harness that was leaked.

Notice how the only rows that even come close to being gray is GPT-5.4+Mini, diverging even from other GPT models. Is this because it has a wholly different training set, or (more likely IMO), did it just have a system prompt that leads to a different style?

I think it needs some more explanation - Fable is 0.42 from itself apparently so Kimi K3 is basically indistinguishable?

It seems to boil down to China bad, US good, but with tech people speculating so it's definitely valid

I assume Kimi does provide logits. You can do cross entropy both ways. With slightly different results.

Edit: No that doesn't seem to be what's happening here. I think it's some kind of word frequency analysis.

On a more serious note:

If I am working on something not sensitive and afterwards upload my chat log to a public dataset, can open models use that data for training?

Keep in mind, I paid for access to model and half of the work is mine (the prompts). But can the model provider forbid me from publishing my result as open data?

My guess is they used the other Anthropic models extensively for synthetic data generation. The top most similar models for K3 are (lower means more similar)

Fable 5 -> 0.42

Opus 4.8 -> 0.45

Sonnet 5 -> 0.45

Opus 4.7 -> 0.46

Grok 4.3 -> 0.52

There's an obvious jump at Grok 4.3, and it would not surprise me that the similarity there is because Grok used Anthropic models for training too (you can get the similarity list for Grok and it does look like the top most similar models for Grok are either Anthropic models or some Chinese models).

The damning evidence that K3 used Anthropic models for training is that K3 is more similar to those models than it is to K2.6. If you look at the Anthropic, OpenAI or Google models, they are most similar with their own other models. Not so with K3, where K2.6 is less similar than 15 other models.

Now, why is Fable 5 the most similar to K3 and not Opus 4.8. I think it's quite likely that K3 did some fine tuning at the end, when Fable 5 became available. They probably had all the infrastructure in place, and Mythos had been announced for months, so they were probably waiting for the second the newest Anthropic model was released to start using it for synthetic data generation.

Can you elaborate?

This looks very similar to the claim that distilling a model from Anthropic is the same thing as Anthropic distilling the model from information on the internet.

Which is very flawed, since distillation requires the thing to exist in the thing it’s distilled from. And no LLM model existed in the information Anthropic used to train the model. Instead the model was built using information and utilizing new technology including hardware, software, transformer architecture, etc.

I get what you're saying and two wrongs do not make a right but the irony, and why people are even talking about this, is that the thing being distilled clearly, and knowingly, violated copyright & terms across the entire internet.

If it does then their original training did as well.

It's important to note that even if these open models are distilled, they are showing genuine improvements in their architecture, which enables inference costs to be several factors below what equivalent closed models have.

The interesting question is: will Anthropic release a Fable like model with an architecture similar to Kimi, and get the inference cost gains? They should surely beat Kimi because they can internally distill as much as they want.

Distillation is absolutely not the reason they're good. It's not necessarily even done on a more capable model. It can even be done on itself and still bring improvement, or on a weaker model as well (see GLM and Gemini, which is definitely true because it repeats Deepmind's injections).

Yes. I bet Moonshot paid for API access as opposed to pirating like Anthropic did.

What of the costs for creating the data that was used to train the model being distilled?

Could you give an example of the value that only training a model can create but none of the rights-holder could? I feel like if you got a direct, instant communication channel to any of the rights-holder that created the content in the training set of those models, you'd get more value than what the LLM could ever give you on any specific subject.

the outputs of the model have no property protections, and training a model on the outputs of another model does the exact same thing - its expensive and creates new value over what was in the input - a set of documents.

I'm going to hope this was sarcasm and if it is, it's great.

An incredibly ironic comment.

There is a lot more one has to be good at to make a fable level model. Distillation won't get you there, it will help refine some at the end

A lot of those arguments could be said about writing a book, or a decent forum guide.