Glossary.
01Passes and labels
Beauty pass
The final colour image of a rendered frame, with lighting, materials and camera effects applied. Other passes store additional per-pixel quantities for the same frame.
Depth pass
A per-pixel image of the distance from the camera to the nearest visible surface, usually measured along the viewing axis. In rendering it is written with the colour image; for real images it comes from a depth sensor or a depth estimation model.
Surface normals
A per-pixel map of the direction each visible surface faces, stored as a unit vector and usually encoded as RGB, in camera or world coordinates.
Instance segmentation
Labelling each pixel with the individual object it belongs to, so that two objects of the same class receive different IDs.
Semantic segmentation
Labelling each pixel with a class, such as chair or floor, without separating objects of the same class.
Bounding box
The smallest axis-aligned rectangle enclosing an object in an image, usually stored with a class label. A 3D bounding box encloses the object in scene coordinates.
Camera pose
The position and orientation of a camera in a scene, written as a rotation and a translation. Together with the camera intrinsics, it maps scene points to pixels.
Metric depth
Depth in physical units such as metres, as opposed to relative depth, which only preserves the order of distances within an image.
02Data generation
Synthetic data
Training data produced by rendering a 3D scene or running a simulation, rather than captured from the real world.
Ground truth
The reference values a model is trained or evaluated against, such as labels, depth or poses. For real data they come from annotation or measurement; for synthetic data they are exported from the scene.
Domain randomisation
Varying nuisance factors such as lighting, textures, object placement and camera position across training data, so that a model learns features that hold across them. Introduced for sim-to-real transfer in robotics by Tobin et al. (2017).
Counterfactual pair
Two samples that are identical except for one controlled change, used to test whether a model's output depends on that factor.
03Models
Physical AI
AI systems that perceive, predict or act in the physical world, including robot policies, world models and the perception models they rely on.
Robot policy
A function that maps a robot's observations to actions, learned by imitation from demonstrations, by reinforcement learning, or both.
VLA
Vision-language-action model: a model that maps camera images and a language instruction to robot actions, typically built on a pretrained vision-language model.
World model
A model that learns how an environment evolves, predicting future observations or states from past ones and, often, from actions. Used for planning, simulation and video prediction.
Novel view synthesis
Generating an image of a scene from a viewpoint absent from the input images. Methods such as NeRF and Gaussian splatting are evaluated on held-out views with known camera poses.
Embodiment
The physical body an agent controls, including its sensors, kinematics and dimensions. Data and policies are often specific to one embodiment.
04Simulation
Sim-ready asset
A 3D asset that carries what a physics simulator needs besides appearance: collision geometry, mass and inertia, physical materials and, for objects with moving parts, joints.
Sim-to-real gap
The drop in performance when a model or policy trained in simulation is deployed in the real world. Causes include differences in appearance, physics, sensor behaviour and the situations encountered.
USD
Universal Scene Description, an open framework for describing and exchanging 3D scenes, originally developed at Pixar and used by NVIDIA Omniverse and Isaac Sim, among others.
05In Studio
Axis
A property of a Studio scene that can change between runs, such as layout, lighting, clutter or camera path. Anything not declared as an axis is held fixed.
Pass
One of the layers Studio writes for every frame: the beauty image, depth, surface normals, instance segmentation or 2D boxes, with the camera pose alongside.
Permutation
One run of a Studio scene with a declared axis changed and every other setting held fixed.
Run manifest
The file written with every Studio run, recording the seed, the camera and the axes that were turned, with their values.
See these on your own scene.
Every frame Studio renders carries the depth, normals, instance IDs and camera pose described on this page.