Training data for
world models.
Studio renders a scene you specify, then renders it again with one thing moved: the camera, the layout, the light or how full the room is. Every frame comes with the geometry behind it.
The same room, four times,
one thing moved.
Layout, then clutter, then light. Everything you do not name is held still, so a change in the score has one cause.
The room holds still.
The camera does the work.
Four camera moves through one gym, each from the same first frame. Nothing in the room moves. Pose is written on every frame, so a predicted view can be scored against the view that was drawn.
A world model only learns the variation it was shown.
Studio renders the same room under conditions it would never otherwise see, and reads the geometry off the scene as it draws each frame.
Three jobs a world model team runs.
Every one of these reads its layers off the same frames. The output formats and the full run settings are on the Studio page.
Video prediction pretraining
Indoor sequences on a known camera path, where the geometry behind every frame is recorded at render time.
Beauty · Camera pose
Geometry-scored evaluation
Give the model the opening frames and score its continuation against the rendered one, which carries depth, normals and masks.
Depth · Normals · Segmentation
Counterfactual pairs
The same room twice with one thing changed. That pairing is how you tell a learned factor from a correlated one.
Permutations · The axes, recordedScore your model on a room
it has never seen.
Studio is open to a small group of researchers and design partners.
World models and rendered data.
What data do you need to train a world model?
Large amounts of video, and the same place under many conditions. The second is what real footage rarely gives you: a scene varied along known axes, with the geometry behind every frame available for scoring.
How do you evaluate a world model?
Give the model the opening frames of a sequence and compare its continuation against a rendered reference carrying depth, surface normals and instance masks. Scoring geometry rather than pixels tells you whether the model kept the room, not whether it matched the lighting.
Why is real video not enough for world model training?
A recording gives you one version of a place: the clutter that happened to be there, the light that happened to fall, the path the camera happened to take. A model cannot learn which of those it is allowed to change if it has only ever seen one of them.
What is a world model, exactly?
A world model predicts what a scene will look like next, given what it has seen and what happens in it. The useful test is not whether the next frame is pretty but whether the geometry it implies is still the same room.
What are counterfactual pairs and why do they matter?
The same room rendered twice with exactly one thing changed. Pairs like that are how you separate a factor your model has actually learned from one that merely correlated with it in the training set. They are almost impossible to collect by recording and take one run to render.
Can I control the camera path?
Yes, and it is one of the axes you can hold still while others move. Camera pose is written for every frame, so a run is reproducible and two runs down the same path are directly comparable.
What resolution and frame rate do runs come at?
1080p at 24 frames per second by default. Resolution, samples, stride and the number of runs are all set on the run, which shows how long it will take before anything renders.
Can I use this to evaluate a model I already have?
Yes. A set of runs your model has never seen, carrying the geometry behind every frame, is a held-out benchmark you own. Nothing has to be trained on it for it to be useful.