World Models

28 points by ankit219 113 days ago | 4 comments

in-silico 109 days ago |

> Current LLMs struggle here because they’re trained on imitation. They learn what people said about competitive dynamics, not how competition unfolds. They can recite game theory but can’t simulate a price war.

I don't think this is true. LLM training data almost certainly contains accounts of competitions and other events unfolding. In fact, depending on how the data was filtered, there might be more data of competitions and price wars unfolding than data about game theory.

I would challenge the author to actually task a frontier LLM with simulating a price war and see if it fails, rather than assuming it would. In fact, that would be my feedback on many of these "LLMs can't do X" articles, because often the best models can in fact do X.

ankit219 108 days ago | |

you are comparing post hoc narratives in the training data to real time learning from causal dynamics. The objectives are different. They may look the same in scenarios where its heavily and accurately documented, but most narratives suffer from survivorship bias and reasoning post facto, eulogising the given outcomes.

tehsauce 109 days ago |

“A transformer predicts the next token”

Nope. A transformer is much more general than that. A GPT predicts the next token.

0xuK9UX1110 109 days ago |

Cloudflare, Palo Alto Networks, or keepalived DNS term the alternative charset.