Design Book › Isaac Sim and Isaac Lab

Isaac Sim and Isaac Lab

What a designer must do

  1. Decide whether you need Isaac Sim at all. Isaac Lab 3.0.0 installs Newton by default and runs without it [1].
  2. Pin one pair: Isaac Lab release/3.0.0 with Isaac Sim 6.2 [2]. Read docs at that pin [3].
  3. Check the host against the hardware table before you install [4], [5], [6].
  4. Count an install as failed until the checker prints PASSED and the warmup log is clean ([7] (note), [8] (note)).
  5. Set the physics engine explicitly and print the active one. The defaults mislead ([9] (note)).
  6. Run the bring-up checklist at the end of this chapter, in order.
  7. Read gains, armature and poses back from the running engine. Do not trust what you authored ([10], [11], [12] (note)).

Scope: the Isaac Sim and Isaac Lab guidelines, and the simulator classes. The Isaac Sim guidelines cover Isaac Sim 6.2.0, read from the repo at tag v6.2.0 [13]. The Isaac Lab guidelines cover Isaac Lab 3.0.0 at tag v3.0.0 [1] and Newton 1.7.0 at tag v1.7.0 [14], plus SO-101 rows from LeRobot v0.7.0 [15] and MuJoCo Playground v1.0.0 [16]. MuJoCo recipes are in Simulation setup. Shipped values are in RL configs that ship.

Two kinds of source

The Isaac Sim docs host is held. Its terms bar scrapers, so no guideline comes from it. The Isaac Sim guidelines come from the repo instead, and that repo holds two kinds of text.

Two kinds of source
Text in the repoSourcesCountHow to read them
README and extension docs[13] to [17], and all guidelines on actuators, the simulation app, gains, Newton and system identification33Project documents
Agent-written notes in skills/All other Isaac Sim guidelines, listed below109Guidance only

Treat every guideline marked (note) as guidance. These guidelines come from agent-written notes: SKILL.md text in the repo's skills/ folder. The catalog's record of that source calls it text written for coding agents. Some of their install examples still name 6.0.1 ([18] (note)), so do not copy a version string from them. A must on a note guideline is this catalog's rating of the point. It does not make the note a product specification. Check any default against the installed release [19].

Two kinds of source
Note under skills/SourcesCountVersion as the guideline records it
isaac-sim-installation[18] (note) to [20] (note)8Pip and Docker examples still name 6.0.1
urdf-mjcf-to-usd-conversion[21] (note) to [22] (note)18Written for Isaac Sim 6 / Kit 110
physics-simulation[9] (note) to [23] (note), [24] (note), [25] (note)35Written for Isaac Sim 6 / Kit 110
usd-articulation[26] (note) to [27] (note)7Written for Isaac Sim 6 / Kit 110
isaac-camera[28] (note) to [29] (note)5Written for Isaac Sim 6 / Kit 110
isaac-sim-sensor[30] (note) to [31] (note)12Written for Isaac Sim 6 / Kit 110
isaac-sim-headless-deployment[32] (note) to [33] (note)10Written for Isaac Sim 6 / Kit 110
isaac-sim-ros2-bridge[34] (note) to [35] (note)9Written for Isaac Sim 6.0, carried at tag v6.2.0
isaac-sim-ros-workspaces[36] (note), [37] (note)2Build matrix at tag v6.2.0
isaac-sim-troubleshooting[38] (note) to [39] (note)3Written for Isaac Sim 6.0 / Kit 110

One note shows why. The camera note says focal length is in tenths of a scene unit, then gives a 25 mm lens on a metre stage as 0.025 ([28] (note)). In tenths of a metre, 25 mm is 0.25. The statement and its example disagree. Render a known target before you trust either ([40] (note)).

What each one is for

From the simulator classes. Each row names its class id.

What each one is for
ToolClassWhat it isUse it forLimits it statesSources
Isaac Sim 6.2.0sim-isaac-simOmniverse-based robot simulator. The full-featured oneImporting URDF, MJCF and CAD robots, RTX and physics sensors, ROS 2 work, synthetic data, digital twins. Choose it when you need RTX sensors or the ROS bridgeNeeds an NVIDIA RTX GPU. Kit and assets sit outside the Apache-2.0 source licence. Cold start stated as about 30 to 60 s[4], [41] (note), [42] (note)
Isaac Lab 3.0.0sim-isaac-labGPU-accelerated learning framework on several physics and render backendsReinforcement and imitation learning. Ships tasks and PPO configs, including an SO-101 keyboard typing taskMulti-GPU is Linux only. Solver settings do not carry over between backends. The SO-101 typing project needs its own patched runtime[1], [43], [44], [45]
Newton 1.7.0sim-newtonGPU physics engine on NVIDIA Warp, with MuJoCo Warp as its primary backend. The physics layer, alone or under Isaac Lab or Isaac SimGPU physics, differentiable simulation, OpenUSDmacOS has no GPU acceleration. Camera images need a renderer on top. Deprecates CUDA 12 support[46], [47], [48] (note)
MuJoCo Playground 1.0.0sim-mujoco-playgroundPyTorch suite of robot environments, RL tasks and hardware workflows on reaf-sim and MuJoCoThe full SO-101 path: teleoperation, system identification, large-batch training, zero-shot deployment. Ships fitted SO-101 armature, stiffness, damping and frictionTested on 3 robot setups, and a visual gap may remain under darker shadows. Manual SO-101 calibration can be off by 1 to 2 cm[49], [50], [51], [52], [53]

The summaries in the catalog's release records use older wording. They say Isaac Lab runs on Isaac Sim and that Playground v1.0.0 is built on MuJoCo MJX. The Isaac Lab install guideline and the v1.0.0 README cited by sim-mujoco-playground say otherwise [1]. Trust the pinned repo rows.

Which solver for which job

From class sim-isaac-lab-solvers and the Isaac Lab solver-choice guideline [54]. A setting means something only in the solver that uses it, and Isaac Lab does not translate values [44].

Which solver for which job
SolverEngineForLimitSource
PhysX, TGS or PGSPhysXRigid robots and mesh deformablesDeformables are experimental, need CUDA and replicate_physics=False. Isaac Lab turns CCD off with GPU dynamics[55], [56]
MJWarpNewtonRigid articulated robots. The primary validated Newton pathSize nconmax and njmax yourself[54], [57]
KaminoNewtonRigid mechanisms, including loopsNone stated in the database[54]
VBDNewtonCloth, soft bodies and cablesIts joint support does not replace a general robot articulation[54]
MPMNewtonFluids, granular media and snowNo joint dynamics[54]

Isaac Sim's engine default

Read these three guidelines together. Two defaults point in opposite directions.

Isaac Sim's engine default
SettingDefaultEffectSource
default_enginephysxNames PhysX in the simulation manager[58]
Newton auto_switch_on_startuptrueSwitches to Newton when the extension starts[59]
Standard isaacsim.exp.full.kitEnables the Newton extensionNewton is active at startup, despite the first row[9] (note)

So set the engine at launch and print get_active_physics_engine. Armature, joint friction and joint velocity limit are authored per backend, and each backend reads only its own schema [60]. So a value set for PhysX may do nothing on a Newton run.

The note on engine choice points to Newton for thousands of environments ([61] (note)). It states PhysX as good below 4096 environments and Newton as designed for more than 10K. These are statements, not benchmarks, so measure on your GPU ([62] (note)). Newton is differentiable through Warp and PhysX is not ([48] (note)).

Install routes

Isaac Sim 6.2.0

Isaac Sim 6.2.0
RoutePlatformsNeedsSources
Standalone binaryLinux x86_64, Windows x86_64, and Linux aarch64 only on supported DGX SparkHost on the list[63] (note)
Python packageNo platform list in the databasePython 3.12 with venv and ensurepip. python3.12-venv on Debian and Ubuntu. An exact version pin with --extra-index-url[64] (note), [20] (note)
DockerNative Linux only. Not supported on Windows or WSLDocker, NVIDIA Container Toolkit, a passing GPU check[65] (note)
Source buildWindows 11, Ubuntu 22.04 and 24.04GCC and G++ 11 on Linux, as 12 and later are not supported. Outbound HTTPS to the Omniverse extension host, checked from the build process. Install the built wheels in a Python 3.12 venv[13], [66], [67], [17]
Every routeAs above50 GB free, NVIDIA RTX GPU, compatible driver. A person accepts the Omniverse EULA. The Compatibility Checker prints PASSED. The warmup log is clean[18] (note), [41] (note), [7] (note), [8] (note)

When you use --extra-index-url, pin isaacsim[all,extscache]==<version> to the release you chose, to reduce dependency-confusion risk ([20] (note)). The note's own pip and Docker examples name 6.0.1.

Isaac Lab 3.0.0

Isaac Lab 3.0.0
RouteGives youNeedsSources
Default, kit-lessNewton, with no Isaac Sim. Tasks with a Newton or OV presetPython 3.12[1], [68], [69]
OV PhysX and OV RTX extrasListed in class sim-isaac-labNo guideline gives their requirements[69]
isaacsim extraFull PhysX and RTXOn x86_64: at least 32 GB RAM, 16 GB VRAM, a supported RTX GPU. GLIBC 2.35 or newer on Linux. Isaac Sim newer than 5.1, paired with 6.2[70], [71], [72], [2]
WheelsMajor releases only. Use a source checkout for a patchWheel version names a major release[73]
WindowsWindows 11 on x86_64Long-path support on before you clone[5], [74]
DGX SparkLinux aarch64CUDA 13 or newer with its PyTorch build. SkillGen, XR teleoperation, livestream, Hub Workstation Cache, Cosmos Transfer1 and RLinf are not supported or validated[75], [76]
Multi-GPULinux and NVIDIA NCCL. Windows is not supportedOne GPU run first. --num_envs is per GPU, so four GPUs at 4096 collect from 16,384 environments[43], [77], [78]

The v3.0.0 README warns that this active-development branch may have breaking changes, errors and regressions [79]. Record the commit in every reproducible run.

Newton 1.7.0

Python 3.10 or later, with 3.11 or later recommended [14]. Newton 1.7 deprecates CUDA 12 but keeps it through a CUDA 12 Warp wheel across the 1.7 line. On an R545-R579 driver or an older supported GPU, install newton with warp-lang==1.17.0 [47]. Isaac Lab 3.0.0 uses Warp 1.17, and its wheels carry the CUDA parts Warp needs [80].

MuJoCo Playground 1.0.0

The database holds no install guideline for it. Pin the release in the project lock file, as the Playground library guideline asks [53]. The captured main-branch README describes a PyTorch suite built on reaf-sim.

Hardware

As the guidelines state it. "No guideline" means the database holds no statement, not that there is no limit.

Hardware
ItemIsaac Sim 6.2.0Isaac Lab 3.0.0, defaultIsaac Lab with isaacsim extraNewton 1.7.0
OSStandalone: Linux x86_64, Windows x86_64, DGX Spark aarch64 ([63] (note)). Source build: Windows 11, Ubuntu 22.04 and 24.04 [13]Ubuntu 22.04 or later on x86_64, Linux aarch64 including DGX Spark, Windows 11 on x86_64 [5]Same, with GLIBC 2.35 or newer on Linux [71]Linux x86-64 and aarch64, Windows x86-64, macOS on CPU only [46]
Local GPURTX 4080 minimum, RTX 5080 recommended [4]Turing or newer, compute capability 7.5 or higher, for GPU training on the default CUDA 13 stack [81]. Headless Newton needs no RTX hardware [82]A supported RTX GPU [70]Compute capability 7.5 or higher with the Warp 1.18 wheel [83]
Datacenter GPUA40 minimum, L40S recommended [84]No guidelineNo guidelineNo guideline
Driver"Compatible". No number in the database ([18] (note))580.65.06 or newer on Linux, 580.88 or newer on Windows, set by the default PyTorch build [6]No separate guideline for the extraR580 or newer with the Warp 1.18 wheel [83]
VRAMNo guidelineNo guidelineAt least 16 GB [70]No guideline
RAMNo guidelineNo guidelineAt least 32 GB [70]No guideline
DiskAt least 50 GB free ([18] (note))No guidelineNo guidelineNo guideline
Python3.12 ([17], [64] (note))3.12 [68]3.12 [68]3.10 or later [14]

The SO-101 typing project sets a stricter floor. It runs on Linux x86-64 only, with Windows and ARM not validated [85]. Allow at least 70 GB of disk, and its image is about 32.6 GB [86]. It uses a patched Isaac Lab 3.0.0 runtime in its own Docker image [45]. Hardware inference runs on CPU with LeRobot and needs neither Isaac Sim nor a GPU [87].

macOS. The database has no statement on whether Isaac Sim runs on macOS. No guideline says it does, and no guideline says it does not. The source-build and standalone platform lists name no macOS entry ([13], [63] (note)), and the install guideline asks for an NVIDIA RTX GPU ([18] (note)), but a list that leaves a platform out is not a statement about it. The requirements page sits on the docs host, which is held because its terms bar scrapers. Read that page yourself before you plan on a Mac. The same gap holds for Isaac Lab: its platform guideline lists the supported platforms and does not mention macOS [5].

Your machines

Your machines
MachineWhat the guidelines sayWhat they do not say
RTX 5090 PCOne project measured 2 h 38 min 21 s of training and evaluation on an RTX 5090 [88]The database has no part row for it. No guideline places it against the RTX 4080 floor [4] or gives its VRAM against the 16 GB floor [70]. Read nvidia-smi
The PC, if it runs WindowsIsaac Sim standalone and Isaac Lab list Windows ([63] (note), [5]). Docker and multi-GPU do not ([65] (note), [43]). The SO-101 typing project is not validated there [85]The database does not record which OS the RTX 5090 run used [88]
MacBook Air M5Newton runs on CPU only [46]. mjlab is for evaluation only [89]. A camera is an OpenCV index that can change after a replug [90]The database has no part row for it. It has no statement on Isaac Sim or Isaac Lab on macOS

Headless and standalone

Headless and standalone
JobDoSource
Headless, still renders imagesStart isaac-sim.sh with the --no-window flag[32] (note)
Headless, physics onlyAdd the Kit setting --/app/renderer/enabled=false to the --no-window start, so the renderer stays off[32] (note)
Standalone scriptRun it with python.sh. Set headless in the SimulationApp config. It needs no window flag[91] (note)
Any scriptCreate SimulationApp before other Omniverse imports[92]
Physics-only CIUse isaacsim.exp.base.python.physics.headless.kit. It turns off rendering, window, streaming and UI extensions[93] (note)
Advance physics with renderingCall SimulationContext.step. app.update does not sync physics[94] (note)
Pick render GPUsactive_cuda_gpus or active_gpu, never both. The first follows CUDA_VISIBLE_DEVICES order, the second nvidia-smi order[95]
Batch driverUse the simulation manager and experimental utilities. World and core.utils.stage are deprecated on Kit 110[96] (note)
Custom preparationRegister on SIMULATION_STARTED, not SIMULATION_SETUP[97]
Lifecycle owned elsewhereDisable default callbacks when another manager owns the simulation lifecycle[98]
Headless viewport, complex stageSome GPU configurations need DISPLAY=:0, or the viewport never initializes[33] (note)
RTX sensors, headlessSet drawPoints false[99] (note)
Isaac Lab, physics onlyHeadless Newton needs no RTX hardware[82]

These limits are stated by the notes. Measure them on your host.

Headless and standalone
ItemStated valueSource
GPU memoryKeep under 90 percent of VRAM[100] (note)
Stage memoryClear the stage between episodes. Kit leaks GPU memory otherwise[101] (note)
Cold startAbout 30 to 60 s. Keep a persistent process for interactive batch work[42] (note)
Physics-only throughput10 to 50 times real time, by scene complexity[102] (note)
GB10 JetsonAbout 35K prims with 16 GB of shared memory[103] (note)
Startup hang on high core countsLimit Kit threads. The worked value is 16[38] (note)
Shutdown deadlockGuard SimulationApp.close with a SIGALRM watchdog on Linux, at 120 s for full shutdown and 30 s for plugin unload[104] (note), [39] (note)

Importers

URDF and MJCF into USD

All rows here come from notes.

URDF and MJCF into USD
SettingDefault or behaviour as statedDoSource
InputThe URDF core does not read XACROExpand it first, or import from a running node that publishes robot_description[21] (note)
package:// pathsUnresolved without a mappingPass ros_package_paths or --ros-package[105] (note)
collision_typeConvex HullName it in the config. Also Convex Decomposition, Bounding Sphere, Bounding Cube[106] (note)
allow_self_collisionFalseKeep it off while you train: turning it on slows training[107] (note)
link_densityNoneA kg/m^3 fallback when a URDF has no mass. Give every link a mass from the CAD or a scale[108] (note)
merge_fixed_jointsFalseName it in the config. Enabling it collapses fixed joints. A robot that flies apart with this false and stiff drives needs a merge or lower gains[109] (note), [110] (note)
import_inertia_tensorWrong masses can result when false with bad geometrySet true for a URDF robot[111] (note), [112] (note)
override_joint_stiffness, dampingNoneRevolute: Nm/rad and Nm s/rad. Prismatic: N/m and N s/m[113] (note)
fix_base-True for an arm on a table or wall. False for wheeled, legged and aerial robots[114] (note)
robot_typeMust be a value returned by get_allowed_tokensNine tokens. Arms: Manipulator, Mobile Manipulators, End Effector. Legged: Humanoid, Quadruped. Wheeled: Wheeled, Holonomic. Also Aerial and Default[115] (note)
make_instanceableOut of memory when false during trainingSet true for RL. Without it each of 4096 environments holds a full mesh in GPU memory[116] (note), [22] (note)
Asset transformerRuns after import by default and restructures the USDIf an old test asset is running and the transformer output is ignored, remove the file at usd_path and import again with run_asset_transformer=True[117] (note)
Drive, position teleoperationPreset: position drive, stiffness 800, damping 40A start for a step test, not an SO-101 value. See chapter 26[118] (note)
Drive, torque policyPreset: drive type none, stiffness 0, damping 0Required for a torque policy. An active drive fights the agent[119] (note), [120] (note)
MJCF actuatorsGain and bias types left at the MJCF default are the stated cause of wrong behaviourSet the four override fields: gain type and bias type (override_gain_type, override_bias_type), and their parameters (override_gain_prm, override_bias_prm)[121] (note)
MJCF commandsThe legacy commands MJCFCreateAsset and MJCFCreateImportConfig are deprecatedCall MJCFImporter instead[122] (note)

Isaac Lab URDF converter

Isaac Lab URDF converter
PointGuideline saysSource
DriveForce drive type and position target type for most cases. Target type none disables the drive and sets gains to 0.0[123]
GainsZero-gain USD drives select effort mode in ImplicitActuatorCfg. A gain set to None keeps the USD value. Pick each on purpose[124]
RerunsA changed config writes a new numbered folder. Delete stale folders or wipe usd_dir if you do not want them to accumulate[125]

After import

After import
CheckGuideline saysSource
Articulation rootMulti-arm robot: exactly one, on the chassis and nowhere else[26] (note)
Joint graphMulti-arm robot: every joint body is reachable from that root through one connected joint hierarchy[126] (note)
Attached armsUse a FixedJoint. Remove the root from the arm USD first[127] (note), [128] (note)
SchemaRoot carries IsaacRobotAPI with links and joints filled. Grasp frames carry IsaacSiteAPI[129] (note), [130] (note)
ProofVisual presence does not prove mechanical correctness. Drive every joint[27] (note)

The SO-101

The SO-101
PointGuideline saysSource
Kinematics5 actuated arm joints, so no arbitrary 6-DOF pose. Soft-weight orientation in task-space control[131]
USD variantsphysx for PhysX, the SysID physics variant for Newton MJWarp. Select the one that matches the backend[132]
Asset gainsThe default asset takes gains from the USD, with stiffness and damping None[133]
Model sourceThe models table holds URDF and MJCF rows for the SO-ARM101 and no USD row. Isaac Lab ships its own asset[133]
Import testNo guideline reports a tested URDF-to-USD import of this armNone

Traps: units and frames

All from notes. Convert at every crossing, then test with a known value.

Traps: units and frames
ItemAs statedSource
DriveAPI gains, angularStiffness in Nm/rad, damping in Nm s/rad, target position in degrees[134] (note)
physics:angularVelocityDegrees per second[135] (note)
physics:velocityPhysX ignores it at run time. Set velocity through the runtime API after play[136] (note)
QuaternionsUSD and Isaac: w, x, y, z. scipy: x, y, z, w[137] (note)
Poses during simulationXformCache returns the authored initial pose. Use the simulated-state APIs[12] (note)
Cube.sizeSize 1.0 has half extents of 0.5[138] (note)
Camera axesLooks down minus Z, plus Y up, plus X right, whatever the stage up axis[40] (note)
CameraSensor resolutionHeight, then width[139] (note)

Starting values the notes state

All from notes. Each is a place to start a test, not a result. Where a project document differs, see the next table.

Starting values the notes state
ItemValue as statedSource
Physics rate, general rigid bodies60 to 120 Hz[140] (note)
Physics rate, stacking and contact-rich240 Hz[141] (note)
Physics rate, high-velocity impacts120 Hz with 2 to 4 substeps, with CCD[142] (note)
Physics rate, spinning bodies, gyros, stiff contact chainsAbout 480 Hz[143] (note)
Physics rate, any sceneMore than 4 times the highest frequency in the system[144] (note)
Solver iterations, complex joints and articulations64 position, 16 velocity[145] (note)
Solver iterations, stacking32 position, 8 velocity[146] (note)
Contact, rubber on concreteStatic 1.0, dynamic 0.8, restitution 0.3[147] (note)
Contact, generic metalStatic 0.4, dynamic 0.3, restitution 0.2[148] (note)
Contact, rubber on rubberStatic 0.8, dynamic 0.7, restitution 0.5[149] (note)
Contact, plastic diceStatic 0.4, dynamic 0.3, restitution 0.3[150] (note)
Fast spinnersCompound colliders are stated to tunnel above about 50 rad/s after 5 to 6 s[151] (note)

Replace each contact value with a measured friction test for the real pair.

Where a note and a project document differ

Where a note and a project document differ
PointNote saysOther source saysDo
Iterations, articulated64 position, 16 velocity ([145] (note))Contact-rich PhysX baseline: 8 to 64 position, 1 to 4 velocity. Raise position first for penetration, joint drift or grasp instability [152], [153]Start from the Isaac Lab baseline. Stop at the plateau
Contact-rich rateStart at 240 Hz ([141] (note))Shipped tasks use 100 Hz and 120 Hz [154], [155]Sweep the timestep [156]
CCD for fast impactsPair 120 Hz and 2 to 4 substeps with CCD ([142] (note))CCD is CPU only. With GPU dynamics Isaac Lab forces it off [157], [55]Do not rely on CCD in a GPU task
StabilizationOn by default. Turn off for free-spinning bodies ([158] (note))Keep off when contact forces feed an observation or evaluation [159]Both apply. Set it on purpose

Actuators and gains

Actuators and gains
PointGuideline saysSource
ChoiceThe simplest model that meets the requirement[160]
Implicit PDRuns in the solver. Can still go unstable with contacts, mimic joints, limits, large target changes or poor convergence[161]
DCMotorsaturation_effort is stall torque. actuator_velocity_limit is no-load speed. Fill both at the joint side[162]
DelayedPDActuatorDelay sampled uniformly from min_delay to max_delay at reset. Set from measured latency[163]
RemotizedPDActuatorFor linkages whose lever arm changes through their range[164]
Learned modelsNeed a TorchScript checkpoint in Isaac Lab, a pt2 archive in Newton[165], [166]
Newton explicit effortCan overshoot at stiff gains and large timesteps. Use implicit mode there[167]
GroupsDisjoint joint sets by regular expression[168]
CommandsJoint-side values[169]
Newton actuators in USDSame control law in Isaac Sim and Isaac Lab. Experimental API[170]
Actuator pipelineEach Newton actuator: delay, then controller, then clamping, on one DOF[171]
USD gains under an actuatorZeroed automatically on actuated joints. Others keep theirs[172]
FeedforwardApplies only to joints with an explicit actuator[173]
Which gains are liveA joint may hold PhysX, Newton actuator and MuJoCo gains. Under the Newton MuJoCo solver, the MuJoCo gains are the ones that apply[10], [174]
ArmatureUnauthored is not zero in Newton. Read back what the engine uses[11]
Editing while playingOnly PhysX rereads joint parameters. A Newton edit takes effect at the next play[175]

ROS 2, if you use it

ROS 2, if you use it
PointGuideline saysSource
ActivationPublishers, subscribers and services run only while the timeline plays[34] (note)
Node namesUse isaacsim.ros2.bridge. omni.isaac.ros2_bridge will not load on Kit 110[176] (note)
EnvironmentChange LD_LIBRARY_PATH before launch only. Bundled libraries may lack custom messages[177] (note), [178] (note)
TimeSet use_sim_time to true in the Nav2 parameters, and publish simulation time on /clock[179] (note)
Multi-robotEach robot needs its own namespace. Keep prim paths fixed at run time[180] (note), [181] (note)
CamerasA publisher needs a render product on the camera prim[182] (note)
Missing topicsWith the graph playing, suspect a ROS_DOMAIN_ID mismatch[35] (note)
WorkspacesJazzy on Ubuntu 24.04, Humble on Ubuntu 22.04, Docker for crossed pairs. Windows 11: Jazzy through Pixi only[36] (note), [37] (note)

Bring-up checklist

Bring a robot into Isaac in this order. Guidelines marked (note) are guidance. Their checks are still cheap to run.

Host and install

Host and install
StepDoPass whenSource
1Choose the stackThe install names the backend the task needs[1], [69]
2Pin the pairisaacsim matches the compatibility table. Run metadata records the Isaac Lab commit[2], [79]
3Check the hostOS, GPU, driver, disk and Python are on the hardware table[4], [18] (note), [5], [6]
4Have a person accept the EULAAccepted before install[41] (note)
5Install with an exact versionThe install line pins it. Nothing is copied from a 6.0.1 example[20] (note)
6Run the Compatibility CheckerThe log holds the exact PASSED line[7] (note)
7Run warmupThe log is clean, with a completion marker and no assertions or aborts. Exit code alone does not count[8] (note)

Import

Import
StepDoPass whenSource
8Check the source modelMesh scale and total mass pass. Run the description checklist[183]
9Expand XACROImporter input ends in .urdf and holds no xacro namespace[21] (note)
10Map every package:// pathNo URL lacks a mapping[105] (note)
11Give every link a mass. Import its inertia tensorImported mass of each link matches the URDF[108] (note), [111] (note), [112] (note)
12Set fix_base and the robot typefix_base matches the type. The token is in the allowed list[114] (note), [115] (note)
13Set collider, self collision and fixed-joint merge on purposeThe config names them. A training asset has self collision off, or a written reason[106] (note), [107] (note), [109] (note)
14Choose the drive. Convert gain unitsTorque policy: drive none, gains 0. Otherwise gains and targets are in the stated units[113] (note), [119] (note), [134] (note), [120] (note), [124]
15For RL, set make_instanceable trueThe config holds it. GPU memory is checked during training[116] (note), [22] (note)
16Re-import from a clean folderusd_path or usd_dir is deleted first[117] (note), [125]
17MJCF only: use MJCFImporter, set actuator overridesStep response of each actuator is correct after import[121] (note), [122] (note)

Stage and articulation

Stage and articulation
StepDoPass whenSource
18Count articulation rootsMulti-arm robot: exactly 1[26] (note)
19Walk the joint graphMulti-arm robot: every body is reached from the root[126] (note)
20Attach sub-assembliesArm follows when the chassis moves. Attached parts carry no root[127] (note), [128] (note)
21Fill the robot schemaROBOT_LINKS and ROBOT_JOINTS are not empty. No IsaacReferencePointAPI remains[129] (note), [130] (note)
22Check collision primsRigidBodyAPI and CollisionAPI sit on one prim. No scale op on a collider. Each dynamic prim has RigidBodyAPI[184] (note), [185] (note), [186] (note)
23Check for kinematic bodiesThey set the kinematic-enabled attribute[187] (note)

Engine

Engine
StepDoPass whenSource
24Set the engine at launchget_active_physics_engine prints the one you chose[9] (note), [58], [59]
25Order the startupSimulationApp comes first. Torch comes after Newton settles[92], [188] (note)
26Enable solver outputs earlyconfigure_newton runs before physics starts. Joint wrench consumers run on XPBD or MuJoCo Warp[189], [190]
27Check the sceneThe stage has a PhysicsScene, the timeline is playing, and the collision APIs you expect are applied[23] (note)
28Set solver and ratePhysX: TGS, or a recorded test that favours PGS. Rate is above 4 times the highest modelled frequency[191] (note), [192], [144] (note)
29Hold the home pose for 5 s with gravity onNo explosion[110] (note)

Gains and motion

Gains and motion
StepDoPass whenSource
30Find which gains are liveresolve_joint_gains is called for each tuned joint[10], [174]
31Compare joint parameters across backendsdiverging_joint_params is empty, or each difference is explained[60]
32Read armature from the engineget_dof_engine_armature for each joint[11]
33Replay after a Newton editStop and replay after each joint parameter edit[175]
34Sweep the timestepDiscretizationSweepTest on each actuated joint, reported at the dt you plan to ship[156]
35Drive every jointEach linked body moves[27] (note)
36Check the loopIt calls SimulationContext.step, has no XformCache call, and converts quaternions[94] (note), [12] (note), [137] (note)
37Check for non-finite stateFinite state and observations are asserted each step in debug runs. On PhysX backends, a newly imported robot without tuned parameters can send NaNs into training[193]

Sensors, if used

Sensors, if used
StepDoPass whenSource
38Author camerasA known target lands at the expected image position. Resolution is height, width[28] (note), [40] (note), [139] (note), [194] (note)
39Model lens distortion when the camera is calibratedThe prim holds the schema that matches the calibration[29] (note)
40Read RTX lidar, radar and acoustic output with a WriterGMO frames received match frames produced[30] (note)
41Create lidar with a variant. Enable motion BVH for radarLidar.create names the variant. enable_motion_bvh is true in radar scenes[195] (note), [196] (note)
42Read physics sensors after playPlay is called before get_data. The timeline is not paused[24] (note), [197] (note)
43Set each sensor tick ratetick_rate equals the real sensor output rate[198] (note)

Isaac Lab task

Isaac Lab task
StepDoPass whenSource
44Define actuator groupsNo joint matches two groups[168]
45Fill motor limits and commands at the joint sideValues match the datasheet after gearing[162], [169]
46Select the USD physics variantIt matches the active backend[132]
47Size MJWarp capacityWorst-case contacts and rows over resets, with headroom. njmax is not derived from nconmax[57]
48Keep debug_mode onIt goes off only after the full reset, command and randomization range is validated[199]
49Print the policy perioddt times decimation matches the hardware loop. Substeps do not change it[200], [201]
50Tune with one reproductionLog holds seed, state, action sequence and one change per run[202]
51Train on one GPU before severalA one-GPU run completes first[77]

Where the database is thin

Sources

Each guideline in this chapter with its source, the date it was read and the exact words of the source. Level: must means breaking it causes failure or harm, should is good practice, guide is a rule of thumb.

  1. Isaac Lab 3.0.0 installs Newton by default and its standard install runs without Isaac Sim; add Isaac Sim only when the workflow needs it. Level: should. raw.githubusercontent.com 2026-11-09“**Isaac Lab installs Newton by default.** The standard installation runs without Isaac Sim;”
  2. Isaac Lab release/3.0.0 pairs with Isaac Sim 6.2, and workflows that use Isaac Sim need a compatible version; check the table before mixing. Level: must. raw.githubusercontent.com 2026-11-09“| `release/3.0.0` branch | Isaac Sim 6.2 |”
  3. Have agents read docs at the pinned simulator version, because the Isaac Lab 3.0.0 project notes that compatibility with later releases has not been validated. Level: must. isaac-sim.github.io 2026-11-08“or later versions has not been validated.”
  4. The README states an RTX 4080 as the minimum local workstation GPU and an RTX 5080 as recommended. Level: must. raw.githubusercontent.com 2026-11-09
  5. Supported platforms are Ubuntu 22.04 or later on x86_64, Linux aarch64 including DGX Spark, and Windows 11 on x86_64. Level: must. raw.githubusercontent.com 2026-11-09“- Ubuntu 22.04+ on x86_64, Linux aarch64 (including DGX Spark), or Windows 11 on x86_64.”
  6. The NVIDIA driver must be 580.65.06 or newer on Linux and 580.88 or newer on Windows; the PyTorch build in the default environment sets this floor. Level: must. raw.githubusercontent.com 2026-11-09“- ``580.65.06`` or newer on Linux; ``580.88`` or newer on Windows.”
  7. Treat the install as failed unless the Compatibility Checker prints the PASSED result. Level: must. raw.githubusercontent.com 2026-11-09
  8. Never accept warmup exit code alone; require a clean log and completion marker, failing on assertions or aborts. Level: must. raw.githubusercontent.com 2026-11-09
  9. Set the physics engine explicitly at launch; an app that enables the Newton extension, as the standard full kit does, starts on Newton even though the simulation manager default names PhysX. Level: must. raw.githubusercontent.com 2026-11-09
  10. A joint may hold gains in three places: PhysX drive, Newton actuator and MuJoCo parameters. Check which one the active backend reads before tuning. Level: should. raw.githubusercontent.com 2026-11-09
  11. An unauthored armature is not zero in Newton: the engine simulates its own default. Read back what the running engine uses. Level: should. raw.githubusercontent.com 2026-11-09
  12. During simulation read poses from the simulated-state APIs, not from XformCache, which returns the authored initial pose. Level: must. raw.githubusercontent.com 2026-11-09
  13. Source builds of the tag target Windows 11 or Ubuntu 22.04 and 24.04. Level: must. raw.githubusercontent.com 2026-11-09
  14. Newton needs Python 3.10 or later, with 3.11 or later recommended. Level: must. raw.githubusercontent.com 2026-11-09“- 3.10 - 3.11+ recommended”
  15. The SO-101 follower uses 6 STS3215 motors with 1/345 gearing. Level: guide. raw.githubusercontent.com 2026-11-09“The follower arm uses 6x STS3215 motors with 1/345 gearing.”
  16. Identify SO-101 arms by USB serial number; it stays stable across reboots and replugs. Level: should. raw.githubusercontent.com 2026-11-09“**Arms** — identified by USB serial number, which is stable across reboots and replugs.”
  17. This guideline is held back: its source does not allow us to publish it yet.
  18. Have at least 50 GB of free storage and an NVIDIA RTX GPU with a compatible driver before installing. 50 GB Level: must. raw.githubusercontent.com 2026-11-09
  19. Have agents verify default values and API names against the code of the installed release, as Newton's tuning guide tells humans and agents to do. Level: must. newton-physics.github.io 2026-11-08“the code is the source of truth.”
  20. When using --extra-index-url, pin the exact Isaac Sim version to reduce dependency-confusion risk. Level: must. raw.githubusercontent.com 2026-11-09
  21. The URDF importer core does not read XACRO; expand it first or import from a running node that publishes robot_description. Level: must. raw.githubusercontent.com 2026-11-09
  22. Out-of-memory during training with non-instanced assets is fixed by turning on instancing. Level: must. raw.githubusercontent.com 2026-11-09
  23. Before blaming a controller, confirm the stage has a PhysicsScene, an active timeline and the expected collision APIs. Level: should. raw.githubusercontent.com 2026-11-09
  24. Experimental physics sensors need the timeline running before the first data read; do not use the legacy World initialize flow. Level: must. raw.githubusercontent.com 2026-11-09
  25. New code uses the isaacsim.sensors.experimental.physics namespace; the older path still loads but is deprecated. Level: should. raw.githubusercontent.com 2026-11-09
  26. A multi-arm robot must have exactly one ArticulationRootAPI, on the chassis and nowhere else. Level: must. raw.githubusercontent.com 2026-11-09
  27. Visual presence does not prove mechanical correctness; a part that does not articulate must not be counted as attached. Level: must. raw.githubusercontent.com 2026-11-09
  28. USD camera focal length and aperture are stated in tenths of a scene unit. The source's metre-stage example (25 mm is 0.025) disagrees with that unit, which gives 0.25, so check a known lens in your own stage. Level: should. raw.githubusercontent.com 2026-11-09Review note: The source note contradicts itself: its unit table and its metre-stage example give values ten times apart.
  29. Model real lens distortion with the OpenCV pinhole, OpenCV fisheye or LUT schemas when a calibrated camera has one. Level: should. raw.githubusercontent.com 2026-11-09
  30. Read RTX lidar, radar and acoustic output with a Writer; polling get_data drops or duplicates frames under multitick. Level: must. raw.githubusercontent.com 2026-11-09
  31. DOPE, YCB Video and PyTorch writers are deprecated; use Basic, Kitti, Cosmos, Pose or DataVisualization writers. Level: should. raw.githubusercontent.com 2026-11-09
  32. Use --no-window for headless runs that still render images, and add --/app/renderer/enabled=false for physics-only runs. Level: should. raw.githubusercontent.com 2026-11-09
  33. On some GPU configurations a headless viewport with a complex stage needs DISPLAY=:0, or the viewport never initializes. Level: should. raw.githubusercontent.com 2026-11-09
  34. ROS 2 publishers, subscribers and services are active only while the timeline plays; no topics in a stopped stage is expected. Level: must. raw.githubusercontent.com 2026-11-09
  35. A missing-topics fault with the graph playing points to a ROS_DOMAIN_ID mismatch; set it the same everywhere. Level: should. raw.githubusercontent.com 2026-11-09
  36. Native ROS workspace builds need a matching distro: Jazzy on Ubuntu 24.04, Humble on Ubuntu 22.04. Use Docker for the crossed pairs. Level: must. raw.githubusercontent.com 2026-11-09
  37. On Windows 11 x64 the supported ROS workspace path is Jazzy through Pixi; Windows Humble and WSL2 are not supported by this tool. Level: must. raw.githubusercontent.com 2026-11-09
  38. On high core count CPUs, limit the Kit thread counts to clear a startup hang; the worked value is 16. 16 threads Level: should. raw.githubusercontent.com 2026-11-09
  39. Use the stated timeouts for the watchdog: 120 s for full shutdown and 30 s for plugin unload. Level: guide. raw.githubusercontent.com 2026-11-09
  40. A USD camera looks down minus Z with plus Y up and plus X right, whatever the stage up axis is; convert when loading OpenCV or ROS extrinsics. Level: should. raw.githubusercontent.com 2026-11-09
  41. This guideline is held back: its source does not allow us to publish it yet.
  42. Expect about 30 to 60 s for a cold start; keep a persistent process for interactive batch work. Level: guide. raw.githubusercontent.com 2026-11-09
  43. Multi-GPU and multi-node training need Linux and NVIDIA NCCL; Windows is not supported. Level: must. raw.githubusercontent.com 2026-11-09“Multi-GPU and multi-node training requires Linux and NVIDIA NCCL. Windows is not supported.”
  44. A solver setting has meaning only in the solver that uses it; Isaac Lab selects a solver but does not translate values between solvers. Level: should. raw.githubusercontent.com 2026-11-09“A configuration value has meaning only within the solver that consumes it.”
  45. The SO-101 typing project uses a patched Isaac Lab 3.0.0 runtime in its own Docker image; compatibility with plain 3.0 GA has not been validated. Level: must. raw.githubusercontent.com 2026-11-09“Compatibility with unmodified 3.0 GA or later versions has not been validated.”
  46. Newton runs on Linux x86-64 and aarch64, Windows x86-64, and macOS on CPU only. Level: must. raw.githubusercontent.com 2026-11-09“- Linux (x86-64, aarch64), Windows (x86-64), or macOS (CPU only) - macOS has no GPU acceleration”
  47. Newton 1.7 deprecates CUDA 12 GPU support but keeps it through a CUDA 12 Warp wheel across the 1.7 line; on an R545-R579 driver or older supported GPU, install newton with warp-lang==1.17.0. Level: should. raw.githubusercontent.com 2026-11-09“Newton 1.7 deprecates CUDA 12 GPU support, while retaining it with a Warp”
  48. Newton is differentiable through Warp and PhysX is not; pick Newton when the method needs gradients through the physics. Level: should. raw.githubusercontent.com 2026-11-09
  49. The Playground v1.0.0 SO-101 armature values identified from real teleop data are 0.1088 shoulder pan, 0.1472 shoulder lift, 0.1224 elbow, 0.1133 wrist flex, 0.1146 wrist roll and 0.1985 gripper. Level: guide. raw.githubusercontent.com 2026-11-09“# real teleop data. ARMATURE = { "shoulder_pan": 0.1088, "shoulder_lift": 0.1472, "elbow_flex": 0.1224, "wrist_flex": 0.1133, "wrist_roll": 0.1146, "gripper": 0.1985,”
  50. Fit 24 parameters for the SO-101: armature, friction loss, stiffness and damping on each of the 6 joints, from teleoperation logs by rollout tracking error. 24 parameters Level: should. raw.githubusercontent.com 2026-11-09“The optimizer fits a 24-parameter model (`armature`, `frictionloss`, `stiffness`, `damping` for all 6 joints) from teleoperation MCAP logs by minimizing rollout tracking error in MuJoCo physics.”
  51. Manual SO-101 middle-pose calibration can be off by 1 to 2 cm or more in end-effector height; recalibrate if the arm reaches too low or hovers too high. Level: should. raw.githubusercontent.com 2026-11-09“Manual middle-pose calibration on the physical SO101 arm can easily be off by 1–2+ cm in end-effector height.”Review note: Source only says calibration can easily be off by 1-2+ cm; it does not require recalibration, so severity must is too strong.
  52. The SO-101 pick-cube policy was tested on 3 robot setups; a visual gap can remain with darker shadows, so light the scene to avoid them. Level: should. raw.githubusercontent.com 2026-11-09“We tested on 3 different robot setups under various lighting conditions, but there may still be visual domain gap with darker shadows.”
  53. The captured main-branch MuJoCo Playground README describes a PyTorch suite built on reaf-sim. Level: guide. raw.githubusercontent.com 2026-11-08“written in [PyTorch](https://pytorch.org/) and built on [`reaf-sim`]”
  54. Use MJWarp for rigid articulated robots, Kamino for rigid mechanisms with loops, VBD for cloth, soft bodies and cables, and MPM for fluids and granular media. Level: guide. raw.githubusercontent.com 2026-11-09“- Rigid robots and mesh deformables - Rigid articulated robots - Rigid mechanisms, including loops - Cloth, soft bodies, and cables - Fluids, granular media, and snow”
  55. Isaac Lab turns PhysX continuous collision detection off when GPU dynamics is on. Level: guide. raw.githubusercontent.com 2026-11-09“but Isaac Lab disables that option when GPU dynamics is enabled.”
  56. PhysX deformables use an experimental API, need CUDA and need replicate_physics false. Level: must. raw.githubusercontent.com 2026-11-09“PhysX surface and volume deformables use an experimental asset API, require CUDA, and need ``scene.replicate_physics=False``.”
  57. Size nconmax for contact points per environment and njmax for constraint rows; do not derive njmax from nconmax, because joint limits, drives and mimic constraints use rows too. Level: must. raw.githubusercontent.com 2026-11-09“Do not derive ``njmax`` directly from ``nconmax``.”
  58. The simulation manager setting default_engine is physx unless changed, and accepts physx or newton. Level: guide. raw.githubusercontent.com 2026-11-09
  59. Newton auto-switch on startup defaults to true and automatically selects Newton when the extension starts. Level: guide. raw.githubusercontent.com 2026-11-09
  60. Armature, joint friction and joint velocity limit are authored per backend, and each backend reads only its own schema; report any difference between backends. Level: should. raw.githubusercontent.com 2026-11-09
  61. For RL with thousands of environments the note points to Newton, with the Featherstone or MuJoCo solver. Level: guide. raw.githubusercontent.com 2026-11-09
  62. PhysX is stated as good below 4096 environments; Newton is stated as designed for more than 10K. These are maker statements, not benchmarks. Level: guide. raw.githubusercontent.com 2026-11-09
  63. The standalone binary is offered on Linux x86_64, Windows x86_64 and Linux aarch64 only on supported DGX Spark systems. Level: must. raw.githubusercontent.com 2026-11-09
  64. The Python package install needs Python 3.12 with venv and ensurepip; on Debian and Ubuntu install python3.12-venv. Level: must. raw.githubusercontent.com 2026-11-09
  65. Docker installs need Linux, Docker, the NVIDIA Container Toolkit and a passing GPU check; Docker is not supported on Windows or WSL. Level: must. raw.githubusercontent.com 2026-11-09
  66. On Linux, build from source with GCC and G++ 11; version 12 and later are not supported, and Ubuntu 24.04 needs 11 installed. Level: must. raw.githubusercontent.com 2026-11-09
  67. A source build needs outbound HTTPS to the Omniverse extension host; check reachability from the build process itself, not from a browser. Level: must. raw.githubusercontent.com 2026-11-09
  68. Use Python 3.12. Level: must. raw.githubusercontent.com 2026-11-09“- Python 3.12. The automatic ``uv`` setup manages the environment.”
  69. The kit-less install runs on Python 3.12 and does not install Isaac Sim, so a task that needs PhysX with Isaac Sim needs the isaacsim extra. Level: should. raw.githubusercontent.com 2026-11-09“Kit-less installation uses a Python 3.12 environment and does not install Isaac Sim.”
  70. Adding the isaacsim extra on x86_64 calls for at least 32 GB RAM, 16 GB VRAM and a supported RTX GPU; training and rendering may need more. Level: should. raw.githubusercontent.com 2026-11-09“For x86_64 workstations, plan for at least 32 GB RAM, 16 GB VRAM, and a supported RTX GPU; training and rendering may need more.”Review note: Source says 'plan for at least' 32 GB RAM and 16 GB VRAM, which is guidance, not a requirement.
  71. Isaac Sim wheels on Linux need GLIBC 2.35 or newer. Level: must. raw.githubusercontent.com 2026-11-09“Isaac Sim's pip packages require GLIBC 2.35 or newer on Linux.”
  72. Isaac Sim 5.1 and older are not supported by this Isaac Lab release. Level: must. raw.githubusercontent.com 2026-11-09“Isaac Sim 5.1 and older are unsupported.”
  73. Isaac Lab wheels ship for major releases, not for each patch release; use a source checkout for a patch. Level: guide. raw.githubusercontent.com 2026-11-09“Isaac Lab wheels are published for major releases, not every patch release.”
  74. On Windows, enable long-path support before cloning or installing. Level: must. raw.githubusercontent.com 2026-11-09“Enable Windows long-path support before cloning. In an elevated PowerShell window, run:”
  75. On DGX Spark, SkillGen, XR teleoperation, livestream, Hub Workstation Cache, Cosmos Transfer1 and RLinf are not supported or validated. Level: must. raw.githubusercontent.com 2026-11-09“SkillGen, XR teleoperation, livestream, Hub Workstation Cache, Cosmos Transfer1, and RLinf are not currently supported or validated on DGX Spark.”
  76. On DGX Spark, CUDA 13 or newer and the matching PyTorch build are required. Level: must. raw.githubusercontent.com 2026-11-09“DGX Spark requires CUDA 13 or newer and the corresponding PyTorch build.”
  77. Confirm the task trains on one GPU before using several, so task faults and launch faults stay separate. Level: should. raw.githubusercontent.com 2026-11-09“First, verify that the task trains on one GPU. This separates task or configuration problems from distributed-launch problems:”
  78. The --num_envs value is per GPU: four GPUs at 4096 collect from 16,384 environments. Level: should. raw.githubusercontent.com 2026-11-09“``--num_envs`` is the number of environments **on each GPU**. With four GPUs and ``--num_envs 4096``, the job collects experience from 16,384 environments in total.”
  79. The v3.0.0 README warns that this active-development branch may have breaking changes, errors and regressions. Level: should. raw.githubusercontent.com 2026-11-09“Note that this branch is currently under active development and may experience breaking changes or error messages.”
  80. This release uses Warp 1.17, and its wheels carry the CUDA parts Warp needs, so no separate CUDA Toolkit is needed for standard wheel use. Level: guide. raw.githubusercontent.com 2026-11-09“This release uses Warp 1.17. Its prebuilt wheels include the CUDA components needed by Warp,”
  81. GPU training on the default CUDA 13 stack needs a Turing or newer NVIDIA GPU, compute capability 7.5 or higher. Level: must. raw.githubusercontent.com 2026-11-09“The default CUDA 13 stack requires Turing or newer (compute capability 7.5+).”
  82. Headless Newton simulation does not need RTX hardware; add renderer requirements only when you need camera images or a view. Level: guide. raw.githubusercontent.com 2026-11-09“**Rendering:** Headless Newton simulation does not require RTX hardware.”
  83. With the Warp 1.18 wheel Newton needs compute capability 7.5 or higher and driver R580 or newer. Level: must. raw.githubusercontent.com 2026-11-09“- 1.18: compute capability 7.5+ (Turing) - 1.18: R580 or newer”
  84. The README states an A40 as the minimum datacenter GPU and an L40S as recommended. Level: must. raw.githubusercontent.com 2026-11-09
  85. The documented SO-101 typing workflow supports Linux x86-64 only; Windows and ARM are not validated. Level: must. raw.githubusercontent.com 2026-11-09“The documented workflow supports **Linux x86-64 only**. Windows and ARM have not been validated for this project.”
  86. Allow at least 70 GB of disk for the SO-101 typing project; its built image is about 32.6 GB. Level: must. raw.githubusercontent.com 2026-11-09“Allow **at least 70 GB of available disk space** for Docker layers, caches and runs; the built image is approximately 32.6 GB.”
  87. Hardware inference for the typing policy runs on CPU with LeRobot and needs neither Isaac Sim nor a GPU. Level: guide. raw.githubusercontent.com 2026-11-09“Hardware inference does not require Isaac Sim or a GPU.”
  88. The measured training and evaluation wall time was 2 h 38 min 21 s on an RTX 5090; use it as one planning point. Level: guide. raw.githubusercontent.com 2026-11-09“Its measured training/evaluation wall time was **2 h 38 min 21 s** on an RTX 5090.”
  89. Plan on an NVIDIA GPU for mjlab training; macOS works for evaluation only. Level: must. raw.githubusercontent.com 2026-11-08“mjlab requires an NVIDIA GPU for training. macOS is supported for evaluation only.”
  90. On macOS a camera is an OpenCV index that can change after a replug; on Linux use the stable /dev/v4l/by-id path. Level: should. raw.githubusercontent.com 2026-11-09“On Linux the camera value is a stable `/dev/v4l/by-id/...` path. On macOS it is an OpenCV index, which can reshuffle if you replug devices”
  91. A python.sh standalone script sets headless through the SimulationApp config and needs no window flag. Level: should. raw.githubusercontent.com 2026-11-09
  92. Create SimulationApp before importing other Omniverse modules; Omniverse Toolkit must be running first. Level: must. raw.githubusercontent.com 2026-11-09
  93. For physics-only CI use the dedicated headless kit file, which turns off rendering, window, streaming and UI extensions. Level: should. raw.githubusercontent.com 2026-11-09
  94. Advance physics with rendering through SimulationContext.step; app.update does not sync physics. Level: must. raw.githubusercontent.com 2026-11-09
  95. Pick renderer GPUs with active_cuda_gpus or active_gpu, never both; the first follows CUDA_VISIBLE_DEVICES order and the second follows nvidia-smi order. Level: must. raw.githubusercontent.com 2026-11-09
  96. Batch drivers use the current simulation manager and experimental utilities; World and core.utils.stage are deprecated on Kit 110. Level: should. raw.githubusercontent.com 2026-11-09
  97. Run custom preparation on the SIMULATION_STARTED event, not SIMULATION_SETUP, which core extensions use to prepare physics tensors. Level: should. raw.githubusercontent.com 2026-11-09
  98. Disable default callbacks when another manager owns the simulation lifecycle. Level: should. raw.githubusercontent.com 2026-11-09
  99. Set drawPoints false in headless runs for speed; high LOD gives more points and runs slower. Level: should. raw.githubusercontent.com 2026-11-09
  100. Keep GPU memory use under 90 percent of VRAM; CUDA out-of-memory is a named known issue. 90 % VRAM Level: should. raw.githubusercontent.com 2026-11-09
  101. Clear the stage between episodes; Kit leaks GPU memory when stages are not cleared properly. Level: should. raw.githubusercontent.com 2026-11-09
  102. Physics-only headless throughput is stated as 10 to 50 times real time depending on scene complexity. Level: guide. raw.githubusercontent.com 2026-11-09
  103. On GB10 Jetson hardware expect a limit of about 35K prims with 16 GB of shared memory. Level: guide. raw.githubusercontent.com 2026-11-09
  104. Guard SimulationApp.close with a SIGALRM watchdog on Linux; a threading timer cannot fire while the deadlocked C call holds the GIL. Level: should. raw.githubusercontent.com 2026-11-09
  105. Map every package:// path with ros_package_paths or --ros-package; an unmapped path leaves the URL unresolved. Level: must. raw.githubusercontent.com 2026-11-09
  106. The URDF importer's default collision_type is Convex Hull. Level: guide. raw.githubusercontent.com 2026-11-09
  107. Keep self-collision off for training; it defaults to off and turning it on slows training. Level: should. raw.githubusercontent.com 2026-11-09
  108. link_density is a kg/m^3 fallback when a URDF has no mass. Level: guide. raw.githubusercontent.com 2026-11-09
  109. merge_fixed_joints defaults to false; enabling it collapses fixed joints. Level: guide. raw.githubusercontent.com 2026-11-09
  110. A robot that flies apart with fixed joints left unmerged and stiff drives is fixed by merging them or lowering gains. Level: should. raw.githubusercontent.com 2026-11-09
  111. Wrong masses can result from import_inertia_tensor=false with bad geometry; set import_inertia_tensor=true. Level: should. raw.githubusercontent.com 2026-11-09
  112. For a URDF robot, import the URDF inertia tensor instead of computing inertia from the geometry. Level: should. raw.githubusercontent.com 2026-11-09
  113. Joint stiffness and damping overrides are in Nm/rad and Nm s/rad on revolute joints and N/m and N s/m on prismatic joints. Level: should. raw.githubusercontent.com 2026-11-09
  114. Set fix_base true for an arm mounted to a table or wall, and false for wheeled, legged and aerial robots. Level: should. raw.githubusercontent.com 2026-11-09
  115. The robot-type token must be one of the values returned by get_allowed_tokens. Level: must. raw.githubusercontent.com 2026-11-09
  116. Set make_instanceable true for RL; without it each of 4096 environments holds a full mesh in GPU memory. 4096 envs Level: must. raw.githubusercontent.com 2026-11-09
  117. The asset transformer runs after import by default and restructures the USD output; if its output is ignored because an old test asset is running, delete usd_path and re-import with run_asset_transformer=True. Level: should. raw.githubusercontent.com 2026-11-09
  118. The stated preset for position-controlled teleoperation is a position drive with stiffness 800 and damping 40. Level: guide. raw.githubusercontent.com 2026-11-09
  119. The stated preset for an RL agent that commands torques is drive type none with stiffness and damping 0. Level: guide. raw.githubusercontent.com 2026-11-09
  120. A joint drive left active fights a torque-commanding RL agent; for torque policies set the imported drive type to none with zero stiffness and damping. Level: must. raw.githubusercontent.com 2026-11-09
  121. Set the MuJoCo actuator gain and bias overrides on import when actuators misbehave; values left at the MJCF default are the stated cause. Level: should. raw.githubusercontent.com 2026-11-09
  122. Use MJCFImporter directly; the MJCFCreateAsset and MJCFCreateImportConfig commands are deprecated. Level: should. raw.githubusercontent.com 2026-11-09
  123. The URDF converter recommends the force drive type and position target type for most cases; target type none disables the drive and sets gains to 0.0. Level: guide. raw.githubusercontent.com 2026-11-09“Setting this to ``"none"`` will disable the drive and set the joint gains to 0.0.”
  124. Zero-gain USD drives select effort mode in ImplicitActuatorCfg; a gain set to None keeps the USD value, so pick each on purpose. Level: guide. raw.githubusercontent.com 2026-11-09“A gain set to ``None`` keeps”
  125. Rerunning the URDF converter with a changed config writes a new numbered folder; delete stale folders or wipe usd_dir if you do not want them to accumulate. Level: should. raw.githubusercontent.com 2026-11-09“Delete stale subdirectories manually (or wipe ``usd_dir``) if you do not”
  126. For a multi-arm robot, every joint body must be reachable from the single articulation root through one connected joint hierarchy. Level: must. raw.githubusercontent.com 2026-11-09
  127. Attach an arm to a chassis with a FixedJoint; arms that render but float have none. Level: must. raw.githubusercontent.com 2026-11-09
  128. Remove the articulation root from an arm USD before attaching it to another articulation. Level: must. raw.githubusercontent.com 2026-11-09
  129. Grasp frames carry IsaacSiteAPI, not the deprecated reference point API. Level: should. raw.githubusercontent.com 2026-11-09
  130. The robot root carries IsaacRobotAPI with a valid robot type, and its link and joint relations are filled. Level: should. raw.githubusercontent.com 2026-11-09
  131. The SO-101 has 5 actuated arm joints and cannot reach an arbitrary 6-DOF pose; soft-weight orientation in task-space control. Level: should. raw.githubusercontent.com 2026-11-09“only 5 actuated DOF, it cannot achieve an arbitrary 6-DOF end-effector pose; task-space controllers should command the full pose but soft-weight the orientation rows”
  132. The SO-101 USD has separate physics variants: physx for PhysX and the SysID physics variant for Newton MJWarp. Select the variant that matches the backend. Level: guide. raw.githubusercontent.com 2026-11-09“Multi-backend tasks select the ``physx`` physics variant for PhysX and the SysID ``physics`` variant for Newton MJWarp.”
  133. The default SO-101 asset in Isaac Lab v3.0.0 takes actuator gains from the USD, with stiffness and damping set to None in the implicit actuator group. Level: guide. raw.githubusercontent.com 2026-11-09“stiffness=None,”
  134. For an angular DriveAPI, stiffness is Nm/rad, damping is Nm·s/rad and target position is in degrees. Level: should. raw.githubusercontent.com 2026-11-09
  135. The USD attribute for angular velocity is in degrees per second, not radians per second. Level: must. raw.githubusercontent.com 2026-11-09
  136. PhysX ignores physics:velocity attributes in USD at run time; set velocities through the runtime API after the timeline starts. Level: must. raw.githubusercontent.com 2026-11-09
  137. Isaac and USD quaternions are ordered w, x, y, z and scipy expects x, y, z, w; convert at every crossing. Level: must. raw.githubusercontent.com 2026-11-09
  138. A Cube of size 1.0 has half extents of 0.5; use size 2.0 when the scale op should equal the half extent. Level: should. raw.githubusercontent.com 2026-11-09
  139. CameraSensor resolution is given as height then width. Level: should. raw.githubusercontent.com 2026-11-09
  140. For general rigid-body scenes without tight contact, a physics rate of 60 to 120 Hz is the stated default band. Level: guide. raw.githubusercontent.com 2026-11-09
  141. For stacking and other contact-rich scenes, start at a 240 Hz physics rate. 240 Hz Level: guide. raw.githubusercontent.com 2026-11-09
  142. For high-velocity impacts, run 120 Hz with 2 to 4 substeps and turn on continuous collision detection. 120 Hz Level: guide. raw.githubusercontent.com 2026-11-09
  143. Spinning bodies, gyros and stiff contact chains such as escapements call for about 480 Hz. 480 Hz Level: guide. raw.githubusercontent.com 2026-11-09
  144. The physics rate must be more than 4 times the highest frequency in the system, counting vibration, spin and contact stiffness modes. 4 times highest frequency Level: should. raw.githubusercontent.com 2026-11-09
  145. Give complex joints and articulations 64 position and 16 velocity solver iterations as the starting point. Level: guide. raw.githubusercontent.com 2026-11-09
  146. Give stacking bodies 32 position and 8 velocity solver iterations as the starting point. Level: guide. raw.githubusercontent.com 2026-11-09
  147. Reference contact values for rubber on concrete are static 1.0, dynamic 0.8 and restitution 0.3. Level: guide. raw.githubusercontent.com 2026-11-09
  148. Reference contact values for generic metal are static 0.4, dynamic 0.3 and restitution 0.2. Level: guide. raw.githubusercontent.com 2026-11-09
  149. Reference contact values for rubber on rubber are static 0.8, dynamic 0.7 and restitution 0.5. Level: guide. raw.githubusercontent.com 2026-11-09
  150. Reference contact values for plastic dice are static 0.4, dynamic 0.3 and restitution 0.3. Level: guide. raw.githubusercontent.com 2026-11-09
  151. Compound colliders are stated to tunnel above about 50 rad/s after 5 to 6 s; use simpler convex hulls or a higher physics rate for fast spinners. 50 rad/s Level: should. raw.githubusercontent.com 2026-11-09
  152. The contact-rich PhysX baseline uses 8 to 64 position and 1 to 4 velocity iterations. Level: guide. raw.githubusercontent.com 2026-11-09“min_position_iteration_count=8, max_position_iteration_count=64, min_velocity_iteration_count=1, max_velocity_iteration_count=4,”
  153. Raise position iterations first for penetration, joint drift or grasp instability; raise velocity iterations only for unstable restitution or a velocity-level convergence issue that position iterations do not correct; stop when behaviour plateaus. Level: should. raw.githubusercontent.com 2026-11-09“Increase position iterations first when the fixed reproduction shows penetration, joint constraint drift, or grasp instability.”
  154. The SO-101 keyboard typing task uses decimation = 4, which gives 100 Hz simulation and 25 Hz control. 4 Level: guide. raw.githubusercontent.com 2026-11-09“self.decimation = 4 # 100 Hz sim -> 25 Hz control”
  155. The lift task base config uses sim dt = 1/120. Level: guide. raw.githubusercontent.com 2026-11-09“self.sim.dt = 1 / 120”
  156. Run a timestep sweep to find the coarsest physics step still within tolerance, and report the result at the timestep you plan to ship. Level: should. raw.githubusercontent.com 2026-11-09
  157. CCD is for CPU dynamics only; with GPU dynamics Isaac Lab forces it off and warns. Level: must. raw.githubusercontent.com 2026-11-09“CCD is not supported with GPU dynamics; Isaac Lab forces it off and emits a warning”
  158. Turn scene stabilization off when correct behaviour depends on conserved angular velocity, such as flywheels, gyros and pendulums; it removes angular momentum. Level: must. raw.githubusercontent.com 2026-11-09
  159. Keep PhysX stabilization off when contact sensor forces feed an observation or evaluation; it can make reported force magnitudes wrong. Level: must. raw.githubusercontent.com 2026-11-09“this extra pass can make reported contact-sensor force magnitudes inaccurate, so keep it disabled when those forces are part of the observation or evaluation path.”
  160. Choose the simplest actuator model that meets the requirement. Level: should. raw.githubusercontent.com 2026-11-09“Choose the simplest model that meets your requirements.”
  161. Implicit actuators run PD in the solver and can still go unstable with contacts, mimic joints, joint limits, large target changes or poor convergence. Level: should. raw.githubusercontent.com 2026-11-09“Contacts, mimic joints, joint limits, large target changes, and insufficient solver convergence can still make them unstable.”
  162. In the DCMotor model saturation_effort is the stall torque and actuator_velocity_limit is the no-load speed; fill both from the motor data at the joint side. Level: should. raw.githubusercontent.com 2026-11-09“``saturation_effort`` is the stall torque, and ``actuator_velocity_limit`` is the no-load speed.”
  163. DelayedPDActuator samples its delay uniformly from min_delay to max_delay at reset; set these from measured command latency. Level: should. raw.githubusercontent.com 2026-11-09“The delay is sampled uniformly from ``[min_delay, max_delay]`` at reset.”
  164. Use RemotizedPDActuator for linkages whose effective lever arm changes through their range; its table stores joint angle, transmission ratio and maximum torque. Level: should. raw.githubusercontent.com 2026-11-09“Use it for linkages whose effective lever arm changes through their range of motion.”
  165. Learned actuator models need a TorchScript checkpoint and clip output with the DC motor envelope. Level: must. raw.githubusercontent.com 2026-11-09“They require a TorchScript checkpoint.”
  166. Newton neural drive checkpoints must be pt2 archives saved with torch.export.save. Level: must. raw.githubusercontent.com 2026-11-09“Torch checkpoints must be pt2 archives saved with ``torch.export.save``.”
  167. Newton actuators default to an explicit effort held over the step, which can overshoot or go unstable at stiff gains and large timesteps; use the implicit mode there. Level: should. raw.githubusercontent.com 2026-11-09“At stiff gains and large timesteps this can overshoot or go unstable.”
  168. Actuator groups must select disjoint joint sets by regular expression. Level: must. raw.githubusercontent.com 2026-11-09“Each group selects a disjoint set of joints by regular expression and chooses a model:”
  169. Actuator position and velocity commands are joint-side values. Level: guide. raw.githubusercontent.com 2026-11-09“and velocity commands are expressed in joint-side coordinates”
  170. Author Newton actuators in USD when the robot must carry the same control law into Isaac Sim and Isaac Lab; the API is experimental and may change. Level: should. raw.githubusercontent.com 2026-11-09
  171. Each Newton actuator runs a delay, controller, then clamping pipeline on one DOF. Level: guide. raw.githubusercontent.com 2026-11-09
  172. USD drive gains on actuated joints are zeroed automatically so the implicit drive does not fight the actuator; joints without an actuator keep their drive gains. Level: must. raw.githubusercontent.com 2026-11-09
  173. Feedforward effort applies only to joints with an explicit actuator. Level: must. raw.githubusercontent.com 2026-11-09
  174. This guideline is held back: its source does not allow us to publish it yet.
  175. Only PhysX rereads joint parameters while playing; an edit under Newton takes effect at the next play. Level: must. raw.githubusercontent.com 2026-11-09
  176. Use the isaacsim.ros2.bridge node names; the old omni.isaac.ros2_bridge namespace will not load on Kit 110. Level: must. raw.githubusercontent.com 2026-11-09
  177. Change LD_LIBRARY_PATH before launch only; fix the parent shell and relaunch instead of patching a running process. Level: must. raw.githubusercontent.com 2026-11-09
  178. Bundled ROS 2 libraries may lack custom message types; source a system ROS 2 install in the parent shell before launch when you need them. Level: should. raw.githubusercontent.com 2026-11-09
  179. Publish the sim clock on /clock and set use_sim_time true in Nav2 params. Level: must. raw.githubusercontent.com 2026-11-09
  180. Each robot in a multi-robot scene needs its own ROS 2 namespace. Level: must. raw.githubusercontent.com 2026-11-09
  181. Do not change robot prim paths at run time; the TF tree can become inconsistent. Level: should. raw.githubusercontent.com 2026-11-09
  182. Camera publishers need a render product attached to the camera prim. Level: must. raw.githubusercontent.com 2026-11-09
  183. Before any tuning, check mesh scale and total mass to catch a millimetre-for-metre mistake. Level: must. mujoco.readthedocs.io 2026-11-08
  184. Apply RigidBodyAPI and CollisionAPI to the same prim; splitting them between a parent and a child gives intermittent collision failures. Level: must. raw.githubusercontent.com 2026-11-09
  185. Do not scale a Cube prim that has CollisionAPI directly; place it with a parent Xform and scale a child mesh, or objects fall through the wrong bounds. Level: must. raw.githubusercontent.com 2026-11-09
  186. CollisionAPI alone makes a static collider; add RigidBodyAPI to make the body dynamic. Level: should. raw.githubusercontent.com 2026-11-09
  187. Make a kinematic body with the kinematic-enabled attribute, not by toggling RigidBodyAPI. Level: should. raw.githubusercontent.com 2026-11-09
  188. Do not import torch before Newton physics has settled; the CUDA context conflict hangs Kit. Level: must. raw.githubusercontent.com 2026-11-09
  189. Enable Newton solver outputs such as joint wrenches before physics initialization, because they change state allocation and per-step work. Level: must. raw.githubusercontent.com 2026-11-09
  190. Joint wrench output is available only with XPBD and MuJoCo Warp; VBD and native MuJoCo CPU warn at initialization. Level: must. raw.githubusercontent.com 2026-11-09
  191. Use the TGS solver rather than PGS for articulated robots in PhysX scenes. Level: should. raw.githubusercontent.com 2026-11-09
  192. Keep TGS, solver_type 1, as the first choice for articulated robots and contact-rich manipulation. Level: should. raw.githubusercontent.com 2026-11-09“Keep ``solver_type=1`` (TGS) as the first candidate for articulated robots and contact-rich manipulation.”
  193. On Isaac Lab PhysX backends, a newly imported robot without tuned parameters can go unstable, often sending NaNs into RL training; consult the stability guide. Level: should. isaac-sim.github.io 2026-11-08“this will often result in NaNs propagating into the learning pipeline”
  194. RtxCamera with CameraSensor is the only supported camera authoring path; there is no supported non-RTX route. Level: should. raw.githubusercontent.com 2026-11-09
  195. Pass a variant when creating a lidar whose asset exposes several sensor variants. Level: must. raw.githubusercontent.com 2026-11-09
  196. Radar needs motion BVH on, through the SimulationApp config or the renderer setting. Level: must. raw.githubusercontent.com 2026-11-09
  197. This guideline is held back: its source does not allow us to publish it yet.
  198. The worked IMU example uses a 200 Hz tick rate. 200 Hz Level: guide. raw.githubusercontent.com 2026-11-09
  199. Keep debug_mode on until the solver budget is validated over the full reset, command and randomization range. Level: should. raw.githubusercontent.com 2026-11-09“Turn off ``debug_mode`` only after the solver budget has been validated for the full reset, command, and randomization distribution.”
  200. Compute the policy period as SimulationCfg.dt times env.decimation; substeps do not change it. Level: must. isaac-sim.github.io 2026-11-08“Substeps do not change the policy period”
  201. Each MJWarp substep uses dt divided by num_substeps, and substeps do not change the policy period. Level: guide. raw.githubusercontent.com 2026-11-09“Each solver substep uses ``SimulationCfg.dt / NewtonCfg.num_substeps``.”
  202. Tune with one fixed reproduction: same initial state, seed and action sequence, changing one setting at a time. Level: should. raw.githubusercontent.com 2026-11-09“reproduce one issue with a fixed initial state, seed, and action sequence.”