But this is certainly one area we plan to improve going forward.

- Renderers output pixels or views of a world, either the real world or a generated world

- Simulators predict how the world responds to actions

- Planners predict actions for an agent to make to affect some change in the world

Under this taxonomy, Atlas is somewhere between renderer and simulator. It predicts high-quality views of a consistent 3D world (renderer) but it can also predict explicit 3D to aid in simulation. Atlas doesn't currently predict actions to directly drive a robot, so it isn't natively a planner; but its simulation abilities can help train another model to be planner.

Source: https://www.worldlabs.ai/blog/taxonomy-of-world-models

Best definition I've heard is: AI systems that can build an internal map of their surroundings to anticipate what happens next and make decisions based on their predictions about the consequences the different actions they can take would have.

There's a bunch of different approaches people are trying:

- World labs (linked in this post) is going down the route of neural 3D representation work (NeRFs, 3D Gaussian Splatting)

- Yann LeCun is pretty famously betting on JEPA architectures (check out the excellent Welch Labs videos for more)

- Google is betting on generative video

- Karl Friston was pursuing 'active interference,' which is just traditional RL techniques with different reward functions

However essentially a world model is something that has the understanding of 3d world and can generate novel view point given either text or image input.

The use I have seen is for robotics. You feed in the current view and describe the action you want it to do, and then it plans the arm movements. (really useful for softbody manipulation.

There are other meanings. but essentially a world model is able to reason in 3d, rather than text.

If you only want to reconstruct the content in the input images without any imagination, Atlas can do sparse 3D reconstruction where it predicts a depth map for each input pixel, without predicting anything occluded or not visible in the input views. This could be used to implement your "fog of war" view.

But sometimes you want the model to stitch together views and imagine a coherent completion; this is a powerful tool for building and authoring generative worlds. For example the 1 minute video we show in the post was built by positioning seven unrelated input images in space and prompting the model to interpolate between them along a hand-specified camera path.