Design Book › RL training

RL training

What a designer must do

  1. Define actions only in terms the real controller accepts [1].
  2. Give the policy only observations available on the real robot [2].
  3. Print the policy period and compare it with the hardware loop [3].
  4. Separate time-outs from failures, and bootstrap the critic at time-outs [4].
  5. Search for bugs before you tune hyperparameters [5].
  6. Derive each randomization range from real uncertainty [6].
  7. End any curriculum at the final deployment distribution [7].
  8. Keep one deterministic evaluation apart from randomized testing [8].
  9. Ship the checkpoint with its full params directory [9].

Scope: the guidelines on RL tasks and rewards, and on domain randomization. A policy can learn to exploit a solver or a model error [10], [11]. So design the task against the real robot's interface from the start [1].

Library choice

Library choice
LevelLibraryWhat the guideline saysDoc as the guideline states itSource
guideRSL-RLStart here for a new robotics project with no special needIsaac Lab 3.0.0 docs, Reinforcement Learning[12]
guideRSL-RLCore has PPO and student-teacher distillation. Other methods sit on its extras branchRSL-RL README, main branch, captured 2026-11-08[13]
mustmjlabTraining needs an NVIDIA GPU. macOS works for evaluation onlymjlab README, main branch, MuJoCo Warp 3.16.0 badge, captured 2026-11-08[14]
guideMuJoCo PlaygroundThe captured main-branch README describes a PyTorch suite built on reaf-simMuJoCo Playground README, main branch, captured 2026-11-08[15]

Two version traps show up here. The catalog's release records list MuJoCo Warp v4.0.0 on 2026-11-06, while the mjlab badge read 3.16.0 [14]. They also describe MuJoCo Playground v1.0.0 (2026-11-06) as built on MuJoCo MJX, which differs from the main branch README [15] and from the v1.0.0 README, which names PyTorch and reaf-sim (chapter 25). Pin versions and read the docs at that pin [16].

Your machines. Under the mjlab GPU guideline [14], the MacBook Air M5 can evaluate mjlab policies and the RTX 5090 PC is the training candidate. The guideline does not say whether Windows is supported for mjlab. The database has no part row for either machine. Isaac Lab lists Windows 11 on x86_64 [17], but multi-GPU training needs Linux [18] and the SO-101 typing workflow is Linux x86-64 only [19]. See chapter 25.

Task and episode

Freeze the episode definition early. Sim-to-sim transfer needs it unchanged [20].

Task and episode
ItemRequirementSource
ResetReject robot-object and robot-support penetration before the first physics step[21]
Reset under randomized geometryRe-run the reset check on every geometry variant[22]
Scene, SO-101 typingKeep the keyboard fixed. The policy uses a fixed A–Z key map without camera input and cannot adjust to movement[23]
TerminationTime-outs and failures are different flags[4]
Non-finite stateOn Isaac Lab PhysX backends, a newly imported robot without tuned parameters can go unstable, often sending NaNs into training. Consult the stability guide[24]

Actions and action scaling

Actions and action scaling
LevelRequirementCheckSource
shouldActions match what the real controller can executereview: each term maps to a real interface[1]
guideLegged RL: consider desired joint positions fed to a low-level PD loop. Rudin et al. used this form for ANYmalreview: term and PD gains documented[25]
guideAction representation is a design choice to test. Local feedback in higher-level actions can strongly affect learning and robustnesssim: same task with two action forms[26]
shouldTune scale per task, smaller for contact-rich worksim: sweep scale, log success and smoothness[27]
shouldSet scale with control rate in view. The gear task notes that a higher rate allows larger stepsreview: scale and rate noted together[28]
guide0.025 is the action scale of the UR10e and Rizon 4s gear task, not a general valuereview: a copied scale gets its own sweep[29]

Apply the same policy-output post-processing during real-world inference as in Isaac Lab [30]. During transfer, keep targets short of the hard joint stops [31].

Observations

Observations
LevelRequirementCheckSource
mustOnly observations available on the real robotreview: each maps to a sensor or estimator[2]
shouldSimulation-only dynamics parameters go to the critic onlyreview: actor group has no privileged term[32]
shouldModel sensor noise and delay where they mattermeasure: compare hardware and simulated noise[33]
mustAdding noise never changes tensor shape or orderlint: names, shape and order match in training and deployment[34]
guideConsider a history of observations and actions when hidden dynamics must be inferredsim: with and without history[35]

Rates

Rates
LevelRequirementCheckSource
mustPolicy period is SimulationCfg.dt times env.decimation. Substeps do not change itlint: print it, compare with the hardware loop[3]
guide50 Hz is one published legged reference, not a universal ratemeasure: real policy loop rate and jitter[36]
guideConfirm the compute target holds the rate. Mini Cheetah held 50 Hz on a Jetson TX2 NXmeasure: inference loop time on the target[37]

Do not change decimation to hide an unstable physics step [38].

Rewards

Rewards
LevelRequirementCheckSource
mustPPO: treat time-outs differently from failures. Bootstrap the critic target at time-outsreview: code separates the two flags[4]
shouldCheck that your library handles this. The Rudin paper found it often ignoredreview: read the bootstrap code[39]
shouldLegged sim-to-real: add auxiliary rewards. Walk These Ways states that task reward alone has not been shown to transfersim: gait metrics with and without[40]
shouldPenalize torque, joint acceleration, target change and unwanted collisions for smoother legged motionsim: log each term against task reward[41]
guideConsider an action-rate penalty to promote smooth motion, as used in the gear assembly tasksim: log the term and sign changes[42]
guideInsertion: pair a distance reward with a dense exponential term near alignmentsim: final error with and without[43]
guideThe Rizon 4s gear task linearly ramps its grasp reward weight from zero so the policy first learns coarse behaviour. The config shows 250,000 steps, while the prose says 512ksim: plot weight against steps[44]

Batch and curriculum

Batch and curriculum
LevelRequirementCheckSource
shouldDo not drop PPO steps per environment too low. ANYmal training struggled below 25 steps, or 0.5 s of simulated timesim: sweep against reward[45]
shouldChanging the environment count changes samples per update, so it is a training changereview: run config records both[46]
shouldConsider a curriculum that adapts difficulty to policy performancesim: log level against reward[47]
guideDo not rely on a hand-written curriculum when feasibility is unknownreview: a measurement backs it[48]
mustThe curriculum ends at the final deployment distribution. Evaluate theresim: final stage, full gravity, full ranges[7]
shouldEasier physics such as low gravity is a start only. Zero gravity is not the deployment conditionsim: final evaluation uses full gravity[49]

Domain randomization: method

Domain randomization: method
LevelRequirementCheckSource
shouldRanges span plausible modeling uncertainty, not arbitrary widthmeasure: each range cites a measurement or tolerance[6]
shouldIf transfer works only with extreme ranges, go back to the nominal model and what the policy exploitsreview: range growth triggers a model review[11]
shouldCover real variation without blocking learning. Wider ranges make training hardersim: success against range width[50]
guidePick terms by asking what actually changes on the real robot and around itreview: each term names its cause[51]
shouldCombine identification with randomization. Hwangbo notes each alone has a costreview: the plan names both[52]
shouldWith ADR, start from calibration values measured on the robotmeasure: record them first[53]
guideConsider ADR when manual ranges are hard to choosesim: log range boundaries[54]

Domain randomization: keep it physical

Domain randomization: keep it physical
LevelRequirementCheckSource
mustInertia stays positive and consistent. Decide whether a mass change recomputes inertiasim: assert for every sampled body[55]
mustArmature ranges stay positive and within what hardware supports. Linked mechanisms vary togetherlint: range minimum above zero[56]
mustEach geometry variant gets its own reset collision checksim: on each sampled size[22]
mustPhysX friction randomization does not carry over. Newton events use one coefficientreview: defined per backend[57]

Domain randomization: which terms

Domain randomization: which terms
LevelTermSource contextCheckSource
guideLink mass, joint damping, controller gains, action timestep, observation noisePeng, 95 parameters, arm pushingreview: compare your list[58]
guidePhysical terms fixed per episode; action timestep and observation noise vary per stepPengreview: event mode matches physics[59]
shouldTime between actions, as a simple latency modelPengmeasure: real latency sets the range[60]
guideMemory-based policy when dynamics are unknownPengsim: recurrent against feedforward[61]
shouldGround friction, observation noise, pushesANYmalsim: push recovery at friction extremes[62]
shouldObservation noise based on data measured on the robotANYmalmeasure: idle sensor spread[63]
guideCamera pose and FOV, lighting, object size, object friction, gripper closing speedSquint, SO-101sim: disable each term once, test real success[64]
shouldActuator gain and joint friction where the real robot shows stiction. Needed on UR10e, not on Rizon 4sIsaac Lab gear taskmeasure: step response, steady-state error[65]
guideGear tasks: policies trained without proprioceptive noise transferred well. The authors say UR10e joint feedback is accurate enough that modeling the noise does not helpIsaac Lab gear taskmeasure: encoder against external reference[66]
shouldPose-observation noise calibrated to the real perception pipelineIsaac Lab gear taskmeasure: perceived against true pose[67]
shouldVision: test wide rendering variation rather than photorealismTobin, object localizationmeasure: error on real images[68]
guideTreat "reality is another sample" as a hypothesis for one task classTobinmeasure: run the real task[69]

Published values: do not copy

Each number belongs to one robot and one source [29], [70]. Use it to sanity-check a design, not to fill a config. Whole shipped configs, with PPO settings and reward weights, are in RL configs that ship.

Published values: do not copy
SystemItemValueSource
UR10e, Rizon 4sAction scale0.025[29]
SO-101, SquintDeployment action scale0.15[71]
ANYmalMinimum steps per environment25 steps, 0.5 s[45]
Rizon 4sGrasp reward ramp250,000 steps in the config, 512k in the prose[44]
Arm pushingRandomized parameters95[58]
Mini CheetahGround friction0.05 to 4.00[72]
Mini CheetahPayload-1.0 to 3.0 kg[70]
Mini CheetahMotor strength90 to 110 percent[73]
ANYmalFriction0.5 to 1.25[74]
SO-101, SquintJoint position noiseSigma 5. The sentence states no unit[75]
Object localizationDetector accuracy1.5 cm[68]

Debug and evaluate

Debug and evaluate
LevelRequirementCheckSource
shouldBroken RL code often fails silently. Search for bugs before tuningreview: test against a simple solved task[5]
shouldDebug on a simple environment where learning should be quicksim: benchmark task first[76]
shouldLog many statistics each iterationsim: reward terms, losses, action statistics[77]
shouldOne evaluation with no randomization and fixed seedssim[8]
shouldEvaluate between stages. A published update count is not a guaranteed budgetsim: store results per stage[78]
shouldKeep the checkpoint with its complete params/ directoryreview: release bundle holds both[9]

On the SO-101

This table holds published settings from one paper, Squint, in ManiSkill3 [79], plus the Isaac Lab typing project [80]. Shipped configs from Isaac Lab v3.0.0, NVIDIA's typing project and MuJoCo Playground v1.0.0 are compared in RL configs that ship.

On the SO-101
ItemPublished settingSource
Training control10 Hz, PD joint delta position[79]
DeploymentAction scale 0.15, 30 Hz[71]
Randomized termsSquint's vision terms, in the term table above[64]
Isaac Lab runtimeTyping project's pinned, patched 3.0.0-based runtime[80]
CalibrationYour own. Released offsets do not calibrate your robot[81]

Before you randomize, measure the servo: backlash, step response and encoder accuracy. The backlash, stiction and joint-noise guidelines ask for these checks [82], [65], [66]. One bench test of the STS3215-C018 found about 1 deg of backlash against a datasheet value of at most 0.5 deg [82].

Where the database is thin

Sources

Each guideline in this chapter with its source, the date it was read and the exact words of the source. Level: must means breaking it causes failure or harm, should is good practice, guide is a rule of thumb.

  1. Define policy actions only in terms that the real controller accepts. Level: should. isaac-sim.github.io 2026-11-08“Your action space should match what the real robot controller can execute.”
  2. Give the policy only observations that are available on the real robot. Level: must. isaac-sim.github.io 2026-11-08“The observation space should only include information available from real sensors”
  3. Compute the policy period as SimulationCfg.dt times env.decimation; substeps do not change it. Level: must. isaac-sim.github.io 2026-11-08“Substeps do not change the policy period”
  4. In PPO, treat time-outs differently from failures, and bootstrap the critic target at time-outs. Level: must. arxiv.org 2026-11-08
  5. When RL fails, search for bugs before tuning hyperparameters, as broken RL code often fails silently. Level: should. spinningup.openai.com 2026-11-08
  6. Derive each randomization range from how uncertain the model really is; do not widen ranges without a reason. Level: should. isaac-sim.github.io 2026-11-08“Domain randomization should span plausible modeling uncertainty, not arbitrarily wide values.”
  7. Make any curriculum end at the final deployment distribution, and evaluate there. Level: must. isaac-sim.github.io 2026-11-08“the curriculum must promote to the final deployment distribution.”
  8. Run one evaluation with no randomization and fixed seeds, kept apart from randomized testing, so sampling noise cannot hide differences. Level: should. isaac-sim.github.io 2026-11-08“Keep a separate deterministic nominal evaluation”
  9. Ship a checkpoint together with its full params directory, as playback, evaluation and deployment all need it. Level: should. isaac-sim.github.io 2026-11-08“Keep the checkpoint’s complete params/ directory for playback, evaluation and deployment.”
  10. Raise damping when a position policy alternates saturated commands to exploit one solver's response. Level: should. isaac-sim.github.io 2026-11-08“Increased damping is often necessary to prevent bang-bang control.”
  11. If transfer works only with extreme ranges, go back to the nominal model and to what the policy exploits. Level: should. isaac-sim.github.io 2026-11-08“revisit the nominal model and the feature the policy is exploiting.”
  12. For a new robotics project with no special need, start with RSL-RL under Isaac Lab 3.0.0. Level: guide. isaac-sim.github.io 2026-11-08“start with RSL-RL.”
  13. Use RSL-RL core for PPO and student-teacher distillation; other methods sit on its extras branch. Level: guide. raw.githubusercontent.com 2026-11-08
  14. Plan on an NVIDIA GPU for mjlab training; macOS works for evaluation only. Level: must. raw.githubusercontent.com 2026-11-08“mjlab requires an NVIDIA GPU for training. macOS is supported for evaluation only.”
  15. The captured main-branch MuJoCo Playground README describes a PyTorch suite built on reaf-sim. Level: guide. raw.githubusercontent.com 2026-11-08“written in [PyTorch](https://pytorch.org/) and built on [`reaf-sim`]”
  16. Have agents read docs at the pinned simulator version, because the Isaac Lab 3.0.0 project notes that compatibility with later releases has not been validated. Level: must. isaac-sim.github.io 2026-11-08“or later versions has not been validated.”
  17. Supported platforms are Ubuntu 22.04 or later on x86_64, Linux aarch64 including DGX Spark, and Windows 11 on x86_64. Level: must. raw.githubusercontent.com 2026-11-09“- Ubuntu 22.04+ on x86_64, Linux aarch64 (including DGX Spark), or Windows 11 on x86_64.”
  18. Multi-GPU and multi-node training need Linux and NVIDIA NCCL; Windows is not supported. Level: must. raw.githubusercontent.com 2026-11-09“Multi-GPU and multi-node training requires Linux and NVIDIA NCCL. Windows is not supported.”
  19. The documented SO-101 typing workflow supports Linux x86-64 only; Windows and ARM are not validated. Level: must. raw.githubusercontent.com 2026-11-09“The documented workflow supports **Linux x86-64 only**. Windows and ARM have not been validated for this project.”
  20. Keep the episode definition the same between backends: how resets and commands are sampled, what rewards and terminations mean, episode length and the success test. Level: must. isaac-sim.github.io 2026-11-08“Reset and command distributions, reward and termination meanings, horizon, and success”
  21. Reject resets with robot-object or robot-support penetration before the first physics step. Level: must. isaac-sim.github.io 2026-11-08“Reject robot-object and robot-support penetration, impossible mimic states, and invalid randomized geometry before the first physics step.”
  22. Each sampled geometry variant needs its own check that reset states are free of collisions. Level: must. isaac-sim.github.io 2026-11-08“Re-run collision-valid reset checks for every geometry variant.”
  23. Keep the SO-101 typing keyboard fixed: its policy uses a fixed A–Z key map without camera input and cannot adjust to movement. Level: must. isaac-sim.github.io 2026-11-08“it cannot adjust to a moved keyboard.”
  24. On Isaac Lab PhysX backends, a newly imported robot without tuned parameters can go unstable, often sending NaNs into RL training; consult the stability guide. Level: should. isaac-sim.github.io 2026-11-08“this will often result in NaNs propagating into the learning pipeline”
  25. For legged RL, consider desired joint positions fed to a low-level PD loop; Rudin et al. used this form for ANYmal. Level: guide. arxiv.org 2026-11-08
  26. Treat action representation as a design choice to test, because local feedback in higher-level actions can strongly affect learning and robustness. Level: guide. arxiv.org 2026-11-08
  27. Tune action scale per task: smaller for contact-rich work, with control rate in mind. Level: should. isaac-sim.github.io 2026-11-08“Task precision requirements (smaller for contact-rich tasks)”
  28. Set action scale with the control rate in view; the gear task notes that a higher rate allows larger steps. Level: should. isaac-sim.github.io 2026-11-08“Control frequency (higher frequency allows larger steps)”
  29. Treat 0.025 as the action scale of the UR10e and Rizon 4s gear task, not as a general value. 0.025 action scale Level: guide. isaac-sim.github.io 2026-11-08“self.joint_action_scale = 0.025”
  30. Apply the same policy-output post-processing during real-world inference as in Isaac Lab. Level: must. isaac-sim.github.io 2026-11-08“is also applied during real-world inference”
  31. During policy transfer, stop targets short of the hard limits of each joint. Level: should. isaac-sim.github.io 2026-11-08“keep targets away from hard joint stops”
  32. Pass simulation-only dynamics parameters to the critic only, not to the deployed policy. Level: should. arxiv.org 2026-11-08
  33. Model sensor noise and delay in the observations where they matter for the robot. Level: should. isaac-sim.github.io 2026-11-08“Sensor noise and delays should be modeled appropriately”
  34. Keep observation tensor shape and order fixed when you add noise. Level: must. isaac-sim.github.io 2026-11-08“Match a plausible sensor and never change tensor shape or ordering.”
  35. Consider a history of past observations and actions as policy input when hidden dynamics must be inferred. Level: guide. arxiv.org 2026-11-08
  36. Use 50 Hz as one published legged reference, not a universal rate; Rudin et al. ran policies at this rate. 50 Hz Level: guide. arxiv.org 2026-11-08
  37. Confirm the compute target can hold the policy rate; the Mini Cheetah work held 50 Hz on an onboard Jetson TX2 NX. 50 Hz Level: guide. arxiv.org 2026-11-08
  38. Keep policy decimation out of stability fixes; if the physics step is unstable, fix the physics. Level: must. isaac-sim.github.io 2026-11-08“Do not change policy decimation merely to conceal an unstable physics step.”
  39. Check that your RL library handles time-outs, because the Rudin paper found this often ignored in popular code at the time. Level: should. arxiv.org 2026-11-08
  40. For legged sim-to-real, add auxiliary rewards; the Walk These Ways authors state that task reward alone has not been shown to transfer. Level: should. arxiv.org 2026-11-08
  41. Penalize torque, joint acceleration, target change and unwanted collisions for smoother legged motion. Level: should. arxiv.org 2026-11-08
  42. Consider an action-rate penalty to promote smooth motion, as used in the gear assembly task. Level: guide. isaac-sim.github.io 2026-11-08“Penalizes large changes in actions between timesteps, promoting smooth motions.”
  43. For insertion, pair a distance reward with a dense exponential term that grows sharply near alignment. Level: guide. isaac-sim.github.io 2026-11-08“Provides a dense exponential reward that grows sharply as keypoints align”
  44. The Rizon 4s gear task linearly ramps its grasp reward weight from zero so the policy first learns coarse behavior; the config shows 250,000 steps, while the prose says 512k. 512k environment steps Level: guide. isaac-sim.github.io 2026-11-08“linearly ramps the reward weight from zero over the first 512k environment steps”
  45. Do not drop PPO steps per environment too low; ANYmal training struggled below 25 steps, or 0.5 s of simulated time. 25 steps Level: should. arxiv.org 2026-11-08
  46. Treat a change in parallel environment count as a training change, because it changes samples per update. Level: should. isaac-sim.github.io 2026-11-08“this also changes the number of samples per training update.”
  47. Consider a curriculum that automatically adapts task difficulty to policy performance. Level: should. arxiv.org 2026-11-08
  48. Do not rely on a hand-written curriculum when task feasibility is unknown. Level: guide. arxiv.org 2026-11-08
  49. Use easier physics such as low gravity only as a curriculum start, because zero gravity is not the deployment condition. Level: should. isaac-sim.github.io 2026-11-08“Zero gravity is not the deployment condition.”
  50. Randomize enough to cover real variation without blocking learning, as wider ranges make training harder. Level: should. isaac-sim.github.io 2026-11-08“Increasing randomization ranges makes it harder for the policy to learn”
  51. Pick randomization terms by asking what actually changes on the real robot and around it. Level: guide. docs.nvidia.com 2026-11-08
  52. Combine identification with randomization; Hwangbo notes that accurate modeling is cumbersome alone and robustness alone can cost performance. Level: should. arxiv.org 2026-11-08
  53. For ADR policy training, start with the calibrated physical-robot environment, then expand as performance improves. Level: should. arxiv.org 2026-11-08
  54. Consider ADR when manual fixed ranges are hard to choose, as it expands a distribution over environments where the model performs well. Level: guide. arxiv.org 2026-11-08
  55. Keep inertia positive and physically consistent when you randomize mass, and decide whether mass changes recompute inertia. Level: must. isaac-sim.github.io 2026-11-08“Decide explicitly whether changing mass recomputes inertia.”
  56. Armature randomization must stay positive and within what the hardware supports, and linked mechanisms must vary together. Level: must. isaac-sim.github.io 2026-11-08“Use positive, physically supported ranges and randomize coupled mechanisms coherently.”
  57. Do not assume PhysX friction randomization carries over: Newton events use one friction coefficient, and PhysX values and buckets do not map one-to-one. Level: must. isaac-sim.github.io 2026-11-08“PhysX static/dynamic values and buckets do not map one-to-one.”
  58. For arm dynamics randomization, consider link mass, joint damping, controller gains, action timestep and observation noise, as in Peng's 95-parameter pushing setup. 95 parameters Level: guide. arxiv.org 2026-11-08
  59. Hold physical parameters constant within an episode, and vary action timestep and observation noise at every step, when you follow Peng's method. Level: guide. arxiv.org 2026-11-08
  60. Randomize the time between actions as a simple model of controller latency. Level: should. arxiv.org 2026-11-08
  61. Consider a memory-based policy when the policy must adapt to unknown dynamics. Level: guide. arxiv.org 2026-11-08
  62. Add ground friction randomization, observation noise and pushes for legged sim-to-real, as the ANYmal work did. Level: should. arxiv.org 2026-11-08
  63. Base observation noise on data measured on the robot. Level: should. arxiv.org 2026-11-08
  64. For SO-101 vision tasks, consider camera pose and FOV, lighting, object size, object friction and gripper closing speed, as Squint randomized. Level: guide. arxiv.org 2026-11-08
  65. Add actuator gain and joint friction randomization where the real robot shows stiction; the gear task needed it on UR10e and not on Rizon 4s. Level: should. isaac-sim.github.io 2026-11-08“with negligible stiction and steady-state error.”
  66. For the gear tasks, policies trained without proprioceptive noise transferred well; the authors say UR10e joint feedback is accurate enough that modeling the noise does not help. Level: guide. isaac-sim.github.io 2026-11-08“policies trained without noise on proprioceptive observations (joint positions and velocities) transfer well to real hardware.”
  67. Calibrate pose-observation noise to the real perception pipeline's error. Level: should. isaac-sim.github.io 2026-11-08“Vision-based observations with noise calibrated to FoundationPose + RealSense D435 error”
  68. For vision, test wide rendering variation rather than photorealism; Tobin's detector reached 1.5 cm for object localization with non-realistic textures. 1.5 cm Level: should. arxiv.org 2026-11-08
  69. Treat the claim that enough variation makes reality another sample as a tested hypothesis for one task class, and verify it on yours. Level: guide. arxiv.org 2026-11-08
  70. Treat a -1.0 to 3.0 kg payload range as the Mini Cheetah value only, scaled to no other robot without a reason. -1.0 to 3.0 kg Level: guide. arxiv.org 2026-11-08
  71. On a first SO-101 transfer, consider scaling actions down and raising control rate at deployment; Squint used 0.15 and 30 Hz and reports smoother paths. 0.15 action scale Level: guide. arxiv.org 2026-11-08
  72. The Mini Cheetah paper randomizes ground friction from 0.05 to 4.00. 0.05-4.00 friction coefficient Level: guide. arxiv.org 2026-11-08
  73. Consider randomizing motor strength; the Mini Cheetah work used 90-110 percent. 90-110 % Level: guide. arxiv.org 2026-11-08
  74. Treat a uniform 0.5-1.25 friction range as the ANYmal value from one paper. 0.5-1.25 friction coefficient Level: guide. arxiv.org 2026-11-08
  75. Consider joint-position noise to absorb sim-to-real joint mismatch; Squint uses sigma = 5 without stating its unit in that sentence. 5 unit not stated in sentence Level: guide. arxiv.org 2026-11-08
  76. Debug a new RL implementation on a simple environment where learning should be quick. Level: should. spinningup.openai.com 2026-11-08
  77. Log many learning statistics at each iteration, or you will not see that training is broken. Level: should. spinningup.openai.com 2026-11-08
  78. Evaluate between training stages, and do not treat a published update count as a guaranteed budget. Level: should. isaac-sim.github.io 2026-11-08“this is not a guaranteed minimum.”
  79. Treat 10 Hz delta joint position control as Squint's SO-101 training setting, chosen for faster training. 10 Hz Level: guide. arxiv.org 2026-11-08
  80. Use the SO-101 typing project's pinned Isaac Lab runtime, because unmodified 3.0 GA or later is not validated. Level: must. isaac-sim.github.io 2026-11-08“Compatibility with unmodified 3.0 GA or later versions has not been validated.”
  81. Calibrate each SO-101 before running a policy; another setup's released offsets do not calibrate your robot. Level: must. isaac-sim.github.io 2026-11-08“they do not calibrate another robot.”
  82. A bench test of the STS3215 measured about 1° of encoder backlash against a ≤0.5° datasheet value. Level: guide. github.com 2026-11-08
  83. The measured training and evaluation wall time was 2 h 38 min 21 s on an RTX 5090; use it as one planning point. Level: guide. raw.githubusercontent.com 2026-11-09“Its measured training/evaluation wall time was **2 h 38 min 21 s** on an RTX 5090.”
  84. The measured rate was 36.72 s per 10 updates with 4096 environments. Level: guide. raw.githubusercontent.com 2026-11-09“At the measured 36.72 seconds per 10 updates with 4096 environments”
  85. The SO-101 keyboard typing reward gives typing progress a weight of 2.0. 2.0 Level: guide. raw.githubusercontent.com 2026-11-09“typing_progress = RewTerm(func=mdp.letter_typing_progress, weight=2.0, params={"command_name": "typing"})”
  86. The Playground SO-101 task weights the grasp reward at 1.5 with open 0.80, closed 0.15 and reaching std 0.025. Level: guide. raw.githubusercontent.com 2026-11-09“open_pos=0.80, closed_pos=0.15, reaching_std=0.025, weight=1.5,”