How a Unitree robot learns to walk

Meet the robots

This repository (unitree_rl_gym) supports four robots made by Unitree Robotics: one four-legged robot and three humanoids. Every joint you see below is a DOF (Degree of Freedom) — one motor, one axis it can rotate around.

Go2 is a quadruped ("robot dog"): 4 legs × 3 joints = 12 DOF, all driven by the walking policy. The humanoids are biped robots with a torso, arms, and legs — but legged_gym only trains the legs to walk (H1: 10 leg joints, G1/H1-2: 12 leg joints), even though the full robot has many more DOF (a full G1 URDF variant has up to 29, including arms, waist, and hands).

The motors

Why don't these joints use the stiff, high-ratio gear motors you'd find in a factory robot arm?

Unitree's actuators (like most modern legged robots) are quasi-direct-drive units: an electric motor with a small gear reduction, sitting right at the joint. Because the reduction is small, the joint stays back-drivable — if the foot hits the ground unexpectedly hard, the leg can absorb it by yielding slightly, the way your knee flexes on a curb you didn't see, rather than transmitting the full shock straight through a rigid gearbox.

One tick of a robot's life

Before anything else, here is the loop that runs, over and over, whether the robot is in a simulator or standing in a lab. Every section from here on is really just zooming into one box of this diagram.

The two arrows aren't the same speed. The policy "thinks" relatively slowly — about 50 times a second (every 0.02s). The PD controller reacts much faster — every 0.005s in this repo's simulator. The ratio between them (4, here) is called decimation: the brain gives a target once, and the fast, dumb, reliable PD loop chases that target several times before the brain speaks again.

PD control: the stubborn assistant

The policy never says "push with 12 newton-metres." It says something much simpler: "knee, please be at 40°." Turning that wish into an actual motor torque is the PD controller's entire job — and it's dumb on purpose, which is what makes it trustworthy.

Proportional-Derivative control, in one line:

• Kp (stiffness) is a pull strength: the farther the joint is from its target, the harder it pulls back — like a spring.
• Kd (damping) is a brake: it resists whatever speed the joint already has — like a shock absorber — so the spring doesn't overshoot and oscillate forever.

This is a single free-swinging joint, not a real leg — it's here so you can see what "too soft" and "oscillating" actually look like before reading the numbers below.

The real code — sim and MuJoCo, side by side

Here is the exact same law, twice: once as it runs inside Isaac Gym during training (tensors, thousands of robots at once), and once as it runs in the MuJoCo sim2sim script (plain NumPy, one robot).

actions_scaled = actions * self.cfg.control.action_scale
control_type = self.cfg.control.control_type
if control_type=="P":
    torques = self.p_gains*(actions_scaled + self.default_dof_pos - self.dof_pos) \
              - self.d_gains*self.dof_vel
...
return torch.clip(torques, -self.torque_limits, self.torque_limits)

def pd_control(target_q, q, kp, target_dq, dq, kd):
    """Calculates torques from position commands"""
    return (target_q - q) * kp + (target_dq - dq) * kd

Note the extra ingredient: action_scale and default_dof_pos. The policy doesn't output a raw joint angle — it outputs a small correction, which gets scaled down (action_scale = 0.25 for Go2 and G1) and added to a hand-picked resting pose (default_dof_pos). This keeps the policy's job "small nudges around a sane standing pose" rather than "invent a pose from nothing," which is much easier to learn.

default_joint_angles = {  # target angle [rad] when action = 0.0
    'FL_hip_joint': 0.1,   'FR_hip_joint': -0.1,
    'RL_hip_joint': 0.1,   'RR_hip_joint': -0.1,
    'FL_thigh_joint': 0.8, 'FR_thigh_joint': 0.8,
    'RL_thigh_joint': 1.0, 'RR_thigh_joint': 1.0,
    'FL_calf_joint': -1.5, 'FR_calf_joint': -1.5,
    'RL_calf_joint': -1.5, 'RR_calf_joint': -1.5,
}
control_type = 'P'
stiffness = {'joint': 20.}   # Kp, N·m/rad — same gain for every joint
damping   = {'joint': 0.5}   # Kd, N·m·s/rad
action_scale = 0.25
decimation = 4               # 4 PD steps per policy step

The torque limits in that clip() call aren't chosen by hand either — they're read straight out of the robot's URDF file, so the simulated robot can never be commanded to do something the real motors couldn't survive.

Learning to walk

Nobody hand-writes the walking motion. So where does it come from?

Think of it as a video game played by an athlete who starts out unable to stand, and only gets a score at the end of each attempt. The simulator is the game (you'll meet the whole cast of simulators this fork uses in §11) — physics, gravity, contact with the ground. The policy is the athlete's reflexes: a small neural network that reads the current situation (the observation) and outputs what to do next (the action — those target joint angles from §4).

• Observation — everything the robot can currently sense: orientation, joint angles & speeds, the velocity command it's been asked to follow, and what it did last tick.
• Action — one number per joint: a small offset from the resting pose (fed into the PD controller from §4).
• Reward — a score computed every tick: mostly "did you track the requested walking speed," with penalties for wasting torque, jerky motion, or falling.

Training runs this loop 4096 robots at once, in parallel, for millions of ticks, and gradually adjusts the network so that higher-reward behavior becomes more likely. The algorithm that does the adjusting is called PPO (Proximal Policy Optimization) — the one detail worth knowing is proximal: it always takes small, careful steps, so one unlucky batch of falls can't undo everything learned so far.

Not just weights — a policy with a memory

The network trained for G1 isn't a plain feedforward MLP (Multi-Layer Perceptron — a stack of plain fully-connected layers with no memory of earlier ticks). It's a small LSTM (Long Short-Term Memory — a recurrent network that carries a running "memory" from one tick to the next) with 64 hidden units, followed by one small MLP layer (32 units) that turns that memory into the final action. The LSTM earns its keep because walking is a sequential problem — "am I mid-swing or mid-stance" isn't always recoverable from a single instantaneous snapshot of joint angles alone; a short memory of the last few ticks helps the policy disambiguate what a static observation can't.

The same network, as nodes and connections

The chip diagram above shows the network as a pipeline. Here's the same actor drawn as an actual graph: one circle per neuron, one line per connection (dense layers are sampled — a handful of neurons per layer stand in for hundreds, with a dotted gap marking the rest). Two sibling families sit next to it for contrast: g1_target, which reuses this exact shape, and Go2, the quadruped, which has no memory block at all.

Reading weights directly: the pixel fingerprint

A diagram like the one above shows shape, not what the network actually learned. For that, this fork also renders each layer's weight matrix as a strip of colored pixels — one strip per layer, values mapped to color instead of read one-by-one as numbers. Two color rules, chosen by the value's range: signed weights (roughly [-1,1]) get a two-color gradient that passes through ink at zero; values bounded to [0,1] (like an LSTM gate's sigmoid output) get grayscale instead. The example below uses G1's real layer shapes with synthetic values, just to show the encoding:

The live version of this — real weights, read from whichever policy is currently loaded, updating as you switch policies in the control web — is in the Policy Info Dock's Weight Fingerprint panel (see §12).

class policy:
    rnn_type = 'lstm'
    rnn_hidden_size = 64
    rnn_num_layers = 1
    actor_hidden_dims = [32]      # one small layer after the LSTM
    critic_hidden_dims = [32]
    activation = 'elu'
    init_noise_std = 0.8          # initial exploration noise (std-dev)

class algorithm:                  # PPO, from rsl_rl
    clip_param = 0.2
    entropy_coef = 0.01
    num_learning_epochs = 5
    num_mini_batches = 4
    learning_rate = 1.e-3          # adaptive — adjusted by measured KL divergence
    gamma = 0.99                    # discount factor: how much future reward matters vs. immediate reward
    lam = 0.95                     # GAE lambda
    max_grad_norm = 1.0

class runner:
    num_steps_per_env = 24         # rollout length collected before each update
    max_iterations = 10000

One "rollout, then update" cycle: run 24 ticks across every parallel environment → estimate how much better each action was than the average action in that state, using GAE (Generalized Advantage Estimation — the gamma/lam pair above control it) → update the network for 5 epochs over 4 mini-batches of that same batch of experience → discard it and collect a fresh rollout. The clip_param line is literally what makes PPO proximal ("Proximal Policy Optimization," §5): it caps how far a single update is allowed to move the policy's behavior, measured via KL divergence (Kullback-Leibler divergence, a measure of how much the policy's behavior just shifted), so one batch of unlucky falls can't erase weeks of otherwise-good learning.

Where the randomness actually comes from

init_noise_std and entropy_coef look like they do the same job — both sound like "how much randomness." They don't. Only one of them injects the randomness that makes 4096 identical copies of the same network behave differently; the other one is a brake that keeps that randomness from disappearing too early.

self.std: nn.Parameter = nn.Parameter(init_noise_std * torch.ones(num_actions))
...
def update_distribution(self, observations):
    mean = self.actor(observations)                    # ← comes from the network
    self.distribution = Normal(mean, mean*0. + self.std)  # ← self.std does NOT come from the network

def act(self, observations):
    self.update_distribution(observations)
    return self.distribution.sample()                    # ← the actual dice roll

The network doesn't output a joint angle directly. Every tick, it outputs the center (mean) of a bell curve — a Normal (Gaussian) distribution — and the action that actually gets executed is a number sampled from that bell, not the center itself. The two halves of that bell come from two completely different places:

• The center (mean) is a real forward pass through the LSTM+MLP from earlier in this section, computed from the current observation. It's different for every one of the 4096 parallel robots because each one is in a different pose right now — but by itself, this step has no randomness in it at all.
• The width (self.std) — how wide that bell is, i.e. how much the sampled action is allowed to wander from the center — is not computed from the observation, and doesn't come from the network's forward pass either. It's a separate, free-standing learnable parameter (12 numbers, one per joint), starting at init_noise_std = 0.8, shared identically by all 4096 environments at any given moment of training. It only changes when training updates it — usually shrinking over time, as the policy gets more confident and needs to explore less.
• The dice roll (.sample()) is where the actual randomness enters: PyTorch draws one random number from that bell curve, independently for every environment, every tick. That's the concrete mechanism behind "4096 robots, same weights, different outcomes" — alongside the domain randomization (different friction/mass/pushes per environment) already mentioned above.

The coach who isn't there on game day

During training, a second network — the critic — watches every attempt with a clipboard of information the real robot could never sense directly (true forward velocity, exact ground friction). It only exists to help the athlete's reflexes (the actor) improve faster. Once training ends, the critic is thrown away — only the actor ships. This split is called actor-critic, and the critic's extra information is called privileged observations for exactly that reason.

The full pipeline

Four commands, four stages, each one's output feeding the next:

The hinge between "Play" and everything after it is TorchScript export: the trained network gets frozen into one portable file, loadable with torch.jit.load(...) by the MuJoCo script, the real-robot Python script, and even a C++ program — none of which need Isaac Gym, rsl_rl, or a GPU installed.

Anatomy of a config: Go2

Every robot in this repo is defined almost entirely by one Python config file. Here's Go2's, in full, mapped back to what you've already learned:

class init_state(LeggedRobotCfg.init_state):
    pos = [0.0, 0.0, 0.42]
    default_joint_angles = { ... }     # → §4, the PD "home" pose

class control(LeggedRobotCfg.control):
    control_type = 'P'
    stiffness = {'joint': 20.}          # → §4, Kp
    damping   = {'joint': 0.5}          # → §4, Kd
    action_scale = 0.25
    decimation = 4                      # → §3, brain-speed vs PD-speed

class asset(LeggedRobotCfg.asset):
    file = '{LEGGED_GYM_ROOT_DIR}/resources/robots/go2/urdf/go2.urdf'
    foot_name = "foot"
    penalize_contacts_on = ["thigh", "calf"]
    terminate_after_contacts_on = ["base"]

class rewards(LeggedRobotCfg.rewards):
    class scales:
        torques = -0.0002        # small bribe: don't waste energy
        dof_pos_limits = -10.0   # big bribe: never approach joint limits

That last block — rewards.scales — is worth dwelling on. It's not documentation, it's the actual reward function: legged_robot.py looks for a method named _reward_torques, _reward_dof_pos_limits, and so on, and adds up scale × value for every name it finds. Writing a good reward function is closer to bribing the robot than programming it — reward walking-speed tracking highly, and gently fine everything you don't want (wasted torque, joints near their mechanical limit, feet dragging).

class scales:
    tracking_lin_vel = 1.0    # + reward for matching requested forward/side speed
    tracking_ang_vel = 0.5    # + reward for matching requested turn rate
    lin_vel_z    = -2.0       # fine for bobbing up/down
    torques      = -0.00001   # fine for using more torque than necessary
    dof_acc      = -2.5e-7    # fine for jerky joint accelerations
    feet_air_time = 1.0       # + reward for a natural, not-shuffling stride
    collision    = -1.0       # fine for non-foot body parts touching things
    action_rate  = -0.01      # fine for changing its mind abruptly tick-to-tick

A harder robot: G1

G1 is a humanoid, and its config shows two ideas Go2 didn't need.

Per-joint-group PD gains

Go2 used one Kp/Kd for every joint. A humanoid's hip, knee, and ankle behave differently enough that G1 tunes each group separately:

stiffness = {'hip_yaw': 100, 'hip_roll': 100, 'hip_pitch': 100,
             'knee': 150, 'ankle': 40}     # N·m/rad
damping   = {'hip_yaw': 2,   'hip_roll': 2,   'hip_pitch': 2,
             'knee': 4,   'ankle': 2}       # N·m·s/rad

Notice the knee is both stiffer (150) and more damped (4) than the ankle (40, 2) — it carries more load and needs to resist buckling harder than a lightly-loaded ankle does.

The gait-phase clock

A quadruped can sort of feel out a stable rhythm through raw sensor feedback alone. Bipeds benefit from an explicit sense of rhythm — otherwise the network has no way to know "is now a swing-leg moment or a stance-leg moment" from a single instantaneous snapshot. So G1 (and H1/H1-2) hand the policy an extra pair of numbers every tick: sin(2π·phase) and cos(2π·phase), where phase cycles smoothly from 0 to 1 every 0.8 seconds — like a metronome ticking for a dance, silently counting "left foot's beat... right foot's beat..." so the network can learn a rhythmic gait instead of a jittery one.

period = 0.8
self.phase = (self.episode_length_buf * self.dt) % period / period
self.phase_left  = self.phase
self.phase_right = (self.phase + offset) % 1
sin_phase = torch.sin(2 * np.pi * self.phase)
cos_phase = torch.cos(2 * np.pi * self.phase)
# ... appended onto the observation vector, alongside joint angles etc.

The same file also adds new reward terms not present in the base robot — _reward_contact, _reward_feet_swing_height, _reward_alive — by simply defining new _reward_* methods and referencing them in rewards.scales, the same auto-discovery mechanism from §8.

G1's full reward function

Every row below is a separate _reward_* method, multiplied by its weight and summed every tick (§8):

Two engineering details worth knowing, because they're the kind of thing that's invisible until a policy fails in a specific, confusing way:

• Only positive rewards. After every term above is summed, the total is clipped to never go below zero before it's used to update the network (only_positive_rewards = True). Without this, a reward function this dense — mostly small, ever-present penalties — gives a struggling policy a perverse shortcut: end the episode immediately (fall on purpose) to stop accumulating penalty, rather than push through and actually learn to walk. Clipping the total at zero removes that shortcut's payoff.
• The cautious variant (§11–§12) is this same table, re-weighted — not a different reward function. Switching to it changes four numbers: gentler speed tracking (tracking_lin_vel 1.0→0.5), a new torques penalty not present at all above, and roughly 10× harsher penalties on dof_acc, dof_vel, and action_rate. Same bribe list, different priorities — which is also why it could be produced by resuming training from the stable checkpoint's weights (§11) rather than starting over.

What the "brain" can't see about itself

Look closely at the observation vector and something is deliberately missing: the robot's own linear velocity — how fast the torso is actually moving through the world — is fed only to the critic from §5's "coach" analogy, never to the actor that actually controls the robot (num_privileged_obs = 50 vs. num_observations = 47 — that extra 3-dimensional gap is the linear velocity vector). This isn't an oversight; it's the actor-critic split from §5, made concrete. A simulator can report your exact velocity through the world at every instant for free; a real robot mostly can't, not without noisy, drifting state estimation. So the actor — the network that ships — is trained to walk using only what a real robot could plausibly know about itself: joint state, orientation (via gravity direction), and its own recent action. The critic, which exists only during training and gets thrown away afterward, is allowed to "cheat" with that ground-truth velocity, because a better-informed coach makes the athlete's training itself faster and more stable — without ever letting the athlete lean on that same crutch during the actual performance.

Onto the real robot

The exported policy from §7 gets loaded by deploy/deploy_real/deploy_real.py, which talks to the robot over the network using Unitree's own SDK, unitree_sdk2py. Underneath, it uses DDS (Data Distribution Service) — messages are published on a topic rt/lowcmd (your commands out) and read back from rt/lowstate (the robot's sensors in), over a plain Ethernet connection.

Each command message carries, per joint: q (target angle), qd (target speed), kp, kd, and tau (feed-forward torque) — the exact same PD quantities from §4. The real motor controllers run the PD law themselves, in firmware, at a rate faster than the network can reach them.

Two loose ends solved on real hardware

• Arms & waist: the policy only ever learned to control legs (§1). On the real humanoid, the untrained joints (arms, waist) are simply held at a fixed target angle with their own stiffer PD gains, driven independently of the walking policy.
• IMU placement: H1 and H1-2 mount their IMU on the torso rather than the pelvis, so the deploy script applies a small rotation (using the measured waist-yaw angle) to express orientation in the frame the policy was actually trained on.

Build your own example

Every piece above composes into a recipe. To bring up a new robot or task:

1. Add the robot's geometry: a URDF under resources/robots/<robot>/ (and an MJCF .xml too, if you'll sim2sim it in MuJoCo).
2. Write legged_gym/envs/<robot>/<robot>_config.py: subclass LeggedRobotCfg and LeggedRobotCfgPPO, set asset.file, init_state.default_joint_angles (names must match the URDF), control.stiffness/damping/action_scale, and your rewards.scales bribe list.
3. Optional: subclass LeggedRobot in <robot>_env.py if you need new observations (like G1's gait-phase clock) or new _reward_* methods — they're auto-discovered by name.
4. Register it: one line in legged_gym/envs/__init__.py — task_registry.register("<name>", EnvClass, RobotCfg(), RobotCfgPPO()).
5. Train: python legged_gym/scripts/train.py --task=<name>, then check progress with play.py once you have a checkpoint.
6. Export & validate: play.py writes a TorchScript policy; write a matching YAML under deploy/deploy_mujoco/configs/ (policy path, per-joint kp/kd, default angles) and run it in MuJoCo before ever touching hardware.

Simulators: what runs where, and why

A simulator is the physics engine that steps a robot (or thousands of copies of one) forward in time — gravity, contact, joint torques. That's a completely different piece of software from a viewer, which just draws what's already happening onto a screen (Viser, met at the end of this section) — a training job needs the former, and often runs with none of the latter at all. This fork touches up to four different simulators/viewers depending on where a policy is in its life; here's the whole cast in one place.

Isaac Gym reads URDF files; MuJoCo reads its own format, MJCF (files like scene.xml). Both describe the same physical Go2 or G1 — just translated for two different simulators.

Why two training simulators, not one

unitree_rl_gym upstream trains exclusively on Isaac Gym — Linux and an NVIDIA GPU, no exceptions. This fork (built and tested entirely on an Apple Silicon Mac) added Genesis via LeggedGym-Ex specifically so training works with no GPU at all — but kept Isaac Gym alive for one reason: Kaggle's free-tier cloud GPU.

Selection isn't an environment variable alone — legged_gym/__init__.py hardcodes SIMULATOR=isaacgym under Python ≤3.8 and only reads SIMULATOR=genesis under 3.10+, so it's really which Python interpreter runs the job that decides:

Two clocks, one policy: how Genesis actually steps a tick

§3 introduced decimation — how often the policy "thinks" vs. how often the low-level control loop runs. Here's exactly what that looks like inside GenesisSimulator, this fork's translation layer between a policy's action and real physics:

def step(self, actions):
    for _ in range(self.cfg.control.decimation):   # 4, here
        self.torques = self._compute_torques(actions)  # same PD law as §4
        self._scene.step()                              # Genesis advances physics 0.005s
    self.post_physics_step()                            # read state back, once

Running the PD loop faster than the network needs to "think" is the same reason a thermostat doesn't wait for a committee meeting to turn the furnace on: a dumb, fast, reliable local loop reacting to the last known target is more stable than making the slow network decide every single physics-rate step itself.

One more Genesis-specific tuning knob: substeps = 4, extra contact-solver iterations within each physics step — raised from Genesis's default after early training (with a still-random, untrained policy) produced contact forces violent enough to blow up into NaNs on frequent, hard early falls. More solver iterations per step resolves exactly those violent, high-force contacts more accurately, at some speed cost.

We trained a G1 walking policy from scratch this way — 1800 PPO iterations, 64 parallel environments, all on CPU. Real, if modest — nowhere near a full 10,000-iteration run's polish, but genuinely learned, not scripted. More usefully, we confirmed unitree_rl_gym's own shipped checkpoint (deploy/pre_train/g1/motion.pt) loads and walks correctly in this Genesis fork too — same URDF, joint order, PD gains — staying upright at its target height (~0.78m) for hundreds of steps. That checkpoint became this fork's reference "stable" policy.

Viser: the viewer behind the web control panel

Viser exists in this fork because Genesis's own native viewer window has a rendering bug on this project's Mac/asset combination — --viewer=viser --viser_port=<port> serves the same 3D scene over a WebSocket instead. That's also exactly what a headless Kaggle kernel needs: no display is attached to a cloud GPU kernel at all, so Kaggle jobs always run --headless, with no viewer serving anything.

It's the same Viser instance powering §12's live policy-switching demo: rugiar_driver.py starts Viser on one port (the raw 3D view) and, alongside it on a second port, the unified control web — Pause/Restart, policy switching, E-STOP, velocity commands — that a browser tab talks to over its own WebSocket (/ws, see legged_gym/control/transport.py). Viser only ever matters for watching a policy live on a machine with a browser pointed at it; it plays no role in a training job's speed or correctness on either simulator.

Switching policies live

Having two policies that both control the same G1 raises an obvious question: can you change which one is driving the robot while it's running — in the simulator, and eventually on the real robot — without restarting anything? This section is the short version; the repo's root README has the full design write-up.

Three things go wrong if you just reassign which network gets called each tick:

• The old policy's LSTM hidden state doesn't belong to the new policy — it must be reset, not carried over.
• A sudden change in target angle is a sudden torque spike through the PD controller (§4) — fine in sim, potentially damaging on real hardware.
• "Who decided to switch" (a human, an autonomous rule, eventually an LLM) is a different question from "is this actually a safe instant to switch" — and you don't want that safety judgment re-implemented differently by every possible caller.

The key design choice: instead of a hard cut, PolicySupervisor cross-fades — blends the outgoing and incoming policy's actions linearly over ~15 ticks — so the PD controller sees a gradually-moving target. And the decision "is this instant safe to switch" lives in exactly one place, SafetyGovernor, checking the same upright/fallen signal (projected_gravity) this repo already uses to end a training episode when a robot falls (§8).

def confirm_pending_switch(self) -> bool:
    # Called ONLY by SafetyGovernor, once it judges this instant safe.
    new_policy = self.policies[self.pending_name]
    new_policy.backend.reset()          # the new policy's LSTM starts clean
    self._ramp_from = self.active       # keep the outgoing policy around briefly
    self.active_name = self.pending_name
    self._ramp_remaining = self.ramp_ticks
    return True

def step(self, obs):
    new_action = self.active.backend.step(obs)
    if self._ramp_remaining <= 0:
        return new_action
    old_action = self._ramp_from.backend.step(obs)
    alpha = 1.0 - (self._ramp_remaining / self.ramp_ticks)   # 0 -> 1 across the ramp
    action = (1.0 - alpha) * old_action + alpha * new_action  # cross-fade, not a hard cut
    self._ramp_remaining -= 1
    return action

Try it yourself:

python legged_gym/scripts/rugiar_driver.py \
    --policy stable:/path/to/unitree_rl_gym/deploy/pre_train/g1/motion.pt \
    --policy crouch:logs/g1_crouch/<run_name>/exported/policy_lstm_1.pt \
    --active stable
# open http://localhost:9006 — Restart / Pause / per-policy switch buttons,
# a live "active policy" label, running against Genesis via viser (a web-based
# 3D viewer — Genesis's own native window has a rendering bug on this Mac/asset
# combination, so viser is what actually works here).

Pause & Restart and Stress Stimuli, next to the switching controls above — pause/resume/restart and the sim-only set_random_events RPC from §13's method table, exposed as one panel each. Stress Stimuli is exactly the domain-randomization idea from §5 (random pushes, random commands) made into something an operator can toggle live, on a policy that's already trained, to probe how it holds up outside the exact conditions it saw in training.

What's genuinely unfinished, honestly stated: deploy_real/real_adapter.py (the real-hardware version of this) is ported carefully against unitree_rl_gym's own deploy code — including the physical button-gated safety sequence from the plate above and the observation layout matching G1Robot.compute_observations() exactly — but is still untested on an actual robot: this fork was built with no physical G1 and no unitree_sdk2py installed. The autonomous Selector (today just a tilt-threshold rule; a learned gating network is the active research direction — see RPG and SkillBlender in the README) is left as a clearly-marked next step.

Talking to the robot: the control protocol

Every controller in §12's diagram — the web UI, an autonomous Selector, and (once --real is passed) anyone driving an actual G1 — goes through the exact same door: ControlServer (legged_gym/control/transport.py), a plain WebSocket JSON-RPC-ish endpoint at /ws. There's nothing web-UI-specific about it. Anyone who can open a WebSocket and send JSON — a phone app, a Python script reading a USB gamepad, a microcontroller with WiFi — can drive the robot with it, without touching a line of this repo's code.

Connecting

ws://<host>:<control_port>/ws. If the server was started with --token <secret> (strongly recommended for --real — anything on the robot's WiFi/LAN can otherwise reach it), append ?token=<secret> to the URL or the connection is rejected before it opens. The web UI does exactly this by forwarding its own page's ?token=... query param — share http://<host>:<control_port>/?token=<secret> with anyone who needs either the UI or to build their own client against the same robot.

1. Start the server with a token: python legged_gym/scripts/rugiar_driver.py --policy stable:policies/stable/checkpoint.pt --control_port 9013 --token <a-shared-secret>. Omit --token only for a trusted, localhost-only sim session — never for --real.
2. Connect to ws://<host>:9013/ws?token=<a-shared-secret>. The handshake is rejected before it opens if the token is missing or wrong — check that first if a connection attempt fails silently.
3. Send a first message. Either {"method": "status", "params": {}, "id": 1} to pull one snapshot immediately, or nothing at all — the server starts pushing unprompted status messages at ~10Hz to every open connection the instant it's connected (see "Reading telemetry," below). Most clients then just start calling set_command on their own polling loop.

This is the same three-step handshake examples/joystick_controller.py performs — see "Build your own controller," below, for a working reference implementation of exactly this.

Sending a command

Every outgoing message is a JSON object {"method": "...", "params": {...}, "id": N}, calling one method on ControlService by name. The reply comes back as {"id": N, "result": ...} or {"id": N, "error": "..."}. The method a home-made joystick controller almost always wants:

{"method": "set_command", "params": {"vx": 0.4, "vy": 0.0, "yaw": 0.0}, "id": 1}
// vx, vy in m/s, yaw in rad/s — clamped server-side to the exact envelope
// the active policy was trained across (out-of-range values are silently
// clamped, not rejected). Call this at whatever rate your controller polls
// (e.g. every gamepad tick) — there's no need to throttle it yourself.

The rest of the surface — every method ControlServer will dispatch:

Reading telemetry

You don't have to poll status — once connected, the server pushes {"method": "status", "result": {...}} to every open connection at ~10Hz, unprompted. The shape (trimmed):

{
  "active": "stable", "pending": null, "ramping": false,
  "paused": false, "safety_tripped": false,
  "policies": ["stable", "crouch", ...],
  "backend": "real",                       // "sim" or "real" — see §9
  "capabilities": {"restart": false},      // what THIS backend can't do
  "command": {"vx": 0.4, "vy": 0.0, "yaw": 0.0},
  "telemetry": {
    "projected_gravity": {"value": [0.02, -0.01, -0.999], "unit": "g",
                           "source": "sensor", "label": "Orientation ..."},
    "base_ang_vel": {"value": [...], "unit": "rad/s", "source": "sensor", ...},
    "base_lin_vel": {"value": null, "source": "sim_ground_truth", ...}
    // null on RealAdapter — no IMU measures velocity directly, see §9
  }
}

Every telemetry field is self-describing — source tells you whether it's a real sensor reading (available on real hardware too) or simulator-only ground truth (null on RealAdapter), so a client never has to hardcode which fields are trustworthy off-simulator.

The Live Telemetry panel — every field carries the same SIM/SENSOR tag as the status push's source key above, in plain English: "not measured by any real sensor" next to base_height and base_lin_vel, "real IMU gyroscope reading — available on both sim and real hardware" next to base_ang_vel. What you read in the JSON is exactly what's shown on screen.

Build your own controller

The Command panel pictured above is just one client of the protocol described in this section — a home-made joystick (a USB gamepad, a phone app, a physical box with real potentiometers, a robot-to-robot bridge) is exactly as capable a client, going through the exact same door. Nothing below is web-UI-specific.

The Command panel is literally the on-screen equivalent of the joystick described below — the FWD/BACK slider and yaw dial issue the exact same set_command calls a home-made gamepad client would, at the same clamped envelope. This session's live values (vx=0.84, yaw=-0.39) are a real robot mid-walk, mid-turn — next to it, the Family panel (switches which registered task/driver is running — see the rugiar skill's "Family panel" section) and a live robot-POV Camera feed.

The reference implementation

examples/joystick_controller.py is a complete, working client — connects, authenticates with a token, reads a USB gamepad via pygame, and streams set_command at a fixed rate (--hz, default matches the control loop). Run it with no gamepad at all via --demo to prove the connection works before wiring up hardware — it drives a scripted forward/turn loop instead. It's deliberately short and dependency-light: Python's standard websockets library, nothing else required for --demo. The protocol itself is just JSON text frames over a plain WebSocket — any language with a WebSocket client (JavaScript, C++, an ESP32's Arduino stack, Unity, whatever a phone app is built in) speaks it identically.

Fusing policies

fuse_policies (legged_gym/control/service.py) merges 2+ already-trained local policies' weights into a new one — no further training involved, runs synchronously (seconds, not minutes), and the result is registered exactly like a normally-trained policy: its own policies/<name>/ with checkpoint.pt + train_checkpoint.pt + meta.json, fine-tunable via Clone-from and fusable again. Exposed three ways that all go through the same TrainingManager.fuse_policies() (legged_gym/control/training.py): the control web's "⚛ Fuse policies…" panel (right under "+ New policy…"), the rugiar fuse CLI, and this RPC directly for a custom client.

{"method": "fuse_policies", "params": {
  "names": ["stable_home_made_3", "stable_home_made_4"],
  "out_name": "blended", "weights": [3, 1], "method": "weighted_average"
}, "id": 7}
// weights are normalized to sum to 1 server-side — raw ratios are fine.
// export_task is optional (default: the first source's task) — only
// matters when the sources span more than one task.

Sources must be architecturally compatible — same obs/action dims, hidden dims, and recurrent-or-not (checked by inferring the network shape straight from each source's train_checkpoint.pt tensor shapes, no live env needed). A mismatched task label across sources is only a warning in the response's warnings list, not a hard stop — two different tasks can share an identical network shape and still be worth trying to merge.

The default method is weighted_average — an elementwise weighted sum of matching weights (a.k.a. model soup / SWA-style interpolation). It's cheap and works reasonably well for closely related checkpoints (a fine-tune lineage, or same-seed variants). It has no guarantee for independently-trained policies, though: two networks trained from different random initializations can converge to functionally-equivalent but internally permuted representations — hidden unit i in one network doesn't necessarily correspond to hidden unit i in the other — and naively averaging permuted weights typically lands in a poor region of the loss landscape between the two minima rather than a good one near either (the "mode connectivity" / permutation-symmetry problem).

Working as a team

This fork is developed by a small team split across Mac, Linux, and Windows, with occasional access to a much more powerful (sometimes remote) GPU machine for training. None of what follows is a hard rule — Genesis itself runs identically (just slower) on a GPU-less laptop as it does on a workstation, and Docker erases most of the remaining OS differences — but it's worth being explicit about where each platform is genuinely more comfortable, so people default to the path of least friction instead of fighting their own machine.

Watch a checkpoint before trusting it

A real mistake from this fork's own history: a training run's reward curves looked reasonable enough to plan around, but the policy was actually falling roughly once a second — the reward-curve summary just didn't make that obvious. The fix that should have happened earlier: look at it, not just the numbers, before committing more time to it.

python legged_gym/scripts/play.py --task=g1_crouch --cpu --num_envs=1 \
    --load_run=<run_name> --ckpt=200 \
    --viewer=viser --viser_port=9006
# open http://localhost:9006 — omit --ckpt for the latest checkpoint in that run.
# Genesis's own native window has a rendering bug on this Mac/asset combination,
# so --viewer=viser (a web view) is what actually works, same as the swap demo.

Docker, kept current

The Docker Compose setup this whole demo runs on — Dockerfile, docker-compose.yml, docker-entrypoint.sh's auto-discovery of every checkpoint in ./policies/, and working CUDA passthrough — was contributed by Ramiro R. C. (RawthiL), not the original author: real evidence the "everyone adds what they can" model below is already working, not just an aspiration. See the root README's Docker Compose section for the full up-to-date instructions (GPU overlay, environment variables, what gets mounted) — it's kept current as the control stack grows, most recently to auto-discover .onnx checkpoints (community releases that only ship ONNX) exactly like .pt ones.

Who's comfortable where

Training a stable policy

The reward curve climbed, crashed, climbed again, crashed again. Is the entropy too high?

Usually no — and this section exists because that question comes up naturally the first time anyone stares at a real training curve. Two completely different problems both look like "the curve is going up and down," and telling them apart is the single most useful diagnostic skill for anyone using the Create Policy panel (§12). Everything below is grounded in two things: this fork's own training history (real logs, real numbers, no hypotheticals) and what the wider legged-locomotion RL community — ETH Zürich's legged_gym, Unitree's own unitree_rl_gym, and a few research papers — has already learned the hard way, cited as we go.

export SIMULATOR=genesis   # every rugiar command refuses to import without this set
rugiar train --list_tasks  # sanity check — should print g1, go2, g1_crouch, ...

rugiar train --task g1 --name my_first_walk --max_minutes 15 --push_robots off

rugiar train --list_policies                       # what's already local & fine-tunable?
rugiar train --task g1 --name my_first_walk_v2 \
    --from_policy my_first_walk --max_iterations 500

The browser-side version of step 1/2 above: + New policy… opens the same guided form rugiar train's flags map onto (command envelope, push disturbances, reward weights) without composing a command by hand.

The three vitals, every run

The Create Policy panel's result tiles (EPISODE LENGTH / MEAN REWARD / ACTION NOISE STD) are the same three numbers rsl_rl prints every iteration. Read them in this order — noise std tells you whether the other two are trustworthy at all:

Two patterns that both look "erratic" — but aren't the same problem

Side by side, from two real runs in this fork's own logs:

Diagnose your own run

How much is a lot? — sourced numbers for G1

"Widen the command range" and "turn on push disturbances" are directions, not destinations — here's what the same robot family actually uses, with sources, so a value on the Create Policy panel means something before you type it in.

One lesson at a time

The stable_home_made_3/stable_home_made_4 run above changed three things at once relative to its base checkpoint: doubled the vx command range past Unitree's own default, turned pushes on, and moved the height target 3cm off the checkpoint's own standing height. Any one of those alone is a reasonable next step; all three together is a hard, unfamiliar situation dropped on the policy in one go — and Pattern B above is what that looks like in the logs.

This mirrors what the wider community already does, not just a house rule: Rudin, Hoeller, Reist & Hutter, "Learning to Walk in Minutes" (CoRL 2022) — the paper behind this whole legged_gym lineage — trains with a curriculum that only raises difficulty once the policy has demonstrated it can handle the current level, rather than presenting the hardest case from step one. Concretely, for the Create Policy panel, that means:

• Change one of {command envelope, push disturbances, target variable} per fine-tuning step, not several.
• Widen a range gradually (e.g. -1 1 → -1.5 1.5 → -2 2 across separate runs), not in one jump — same principle as terrain-difficulty curricula in the source paper above.
• Don't turn on --push_robots until the policy is already stable without it (rising episode length, flat-or-falling noise std) — confirmed as the right order in this fork's own HANDOFF_stability_curriculum.md §5 plan, even though Unitree's own G1 config trains with pushes on from iteration zero (§ above) — the two aren't contradictory: Unitree is training from scratch with a full 10,000-iteration budget where early instability has time to resolve itself; a short fine-tuning step from an already-stable checkpoint has much less room to recover from a fresh disturbance introduced at the same moment as other changes.
• Domain randomization itself is worth keeping, though — Kumar, Fu, Pathak & Malik, "RMA: Rapid Motor Adaptation" (RSS 2021) found it measurably improves real-robot robustness (varied friction, mass, pushes) compared to a policy trained without it — the "one knob at a time" advice is about sequencing changes during fine-tuning, not about avoiding randomization altogether.

# What was actually run — 3 knobs moved at once from stable_home_made_2:
python legged_gym/scripts/web_train.py --task g1 --name stable_home_made_3 \
  --from_checkpoint <stable_home_made_2 checkpoint> \
  --cmd_vx_range -2 2 --cmd_vy_range -1 1 --cmd_yaw_range -1 1 \
  --base_height_target 0.750 --push_robots on --max_minutes 20

# A curriculum-respecting next step from the SAME checkpoint — only the
# command range moves, and only halfway to the eventual target:
python legged_gym/scripts/web_train.py --task g1 --name stable_home_made_2b \
  --from_checkpoint <stable_home_made_2 checkpoint> \
  --cmd_vx_range -1.5 1.5 --cmd_vy_range -1 1 --cmd_yaw_range -1 1 \
  --push_robots off --max_minutes 20
# then, only once THAT one's episode length is rising and noise std is flat/falling:
python legged_gym/scripts/web_train.py --task g1 --name stable_home_made_2c \
  --from_checkpoint <stable_home_made_2b checkpoint> \
  --cmd_vx_range -2 2 --push_robots on --max_push_vel_xy 1.0 --max_minutes 20

Reward weights, if the guided fields don't cover it

The Create Policy panel's guided fields (command envelope, target variable, push disturbances, exploration noise) cover the knobs most fine-tuning steps need. The "Reward weights (advanced)" grid underneath is the raw fallback — every --reward_scale <name> <value> this task's reward function defines, unexplained unless a term has its own note. These are the community's own starting points, not something to invent from scratch:

Anti-patterns worth naming

• Reading "curve goes up and down" as one thing. It's at least two, with opposite fixes (above) — check noise std first, always, before touching --entropy_coef.
• Raising push severity before the policy is stable without pushes. Confirmed the hard way in this fork's own history (HANDOFF_stability_curriculum.md).
• Changing several fine-tuning knobs in the same run and reading the result as one lesson. When it goes wrong, there's no way to tell which change caused it — see "one lesson at a time" above.
• Trusting the reward-curve summary alone. A real past mistake in this fork (§13): reward curves looked fine while the policy fell roughly once a second. Watch a checkpoint live before committing more training time to it.

The toolkit: rugiar CLI, rugiar_mcp, the Claude Code skill, and the control web

Everything in §11–§15 — training, fine-tuning, fusing, switching policies live, driving a real robot — is one engine (TrainingManager / ControlService, §13) reachable through four doors that all stay in sync by construction, not by hand-maintained duplication: a terminal command, an MCP server for AI agents, an AI coding assistant that already knows this system, and a browser tab.

rugiar — the command line

legged_gym/cli/rugiar.py is a thin wrapper that turns argv straight into the same keyword arguments TrainingManager already accepts — nothing about how a policy gets built, fine-tuned, or fused lives twice, so the CLI's flags and the control web's form fields can't drift apart. It never imports ControlService/the driver at all — the cleanest boundary in the repo: this door only ever reaches training, never a live robot session. Same command shape for a Genesis locomotion task and an mjlab motion-tracking one — TrainingManager's backend registry (see the mjlab aside in §11) picks the right interpreter/entrypoint per task, transparently:

# train from scratch, stop after 15 minutes
rugiar train --task g1 --name crouch --max_minutes 15 --base_height_target 0.45

# fine-tune an existing local policy
rugiar train --task g1 --name crouch_v2 --from_policy crouch --max_iterations 500

# an mjlab motion-tracking task — same shape, plus a required reference clip
rugiar train --task Rugiar-G1-Mimic --name mimic_dance --max_iterations 3000 \
    --motion_file resources/reference_motion/unitree_g1/mjlab_run/dance1_subject2.npz

# merge two already-trained policies' weights into a new one (§13's Fuse policies)
rugiar fuse --policies stable_home_made_3 stable_home_made_4 --name blended

# discover what's available before committing to any of the above — works for
# mjlab tasks too, even run from the Genesis venv (a one-shot probe into
# .venv-mjlab fills in real data when this process can't import mjlab itself)
rugiar train --list_tasks
rugiar train --list_motions --task Rugiar-G1-Mimic
rugiar train --list_policies
rugiar fuse --list_fusion_methods
rugiar order --show   # the display order a downstream control layer offers policies in

The live, authoritative reference for every flag is always rugiar <subcommand> --help — see the rugiar skill below for a maintained snapshot, and the root README for the full quick-start.

rugiar_mcp — talking to a live robot from an AI agent

Where the CLI only ever reaches training, rugiar_mcp is the exact mirror image: an MCP server exposing a running session — rugiar_driver.py, already up and driving a robot — as tools any MCP-speaking agent can call, over the identical WebSocket JSON-RPC protocol the control web and examples/joystick_controller.py already speak (§13) — no new wire format, no second implementation of "how to talk to a robot" to keep in sync.

rugiar_mcp is a real, working prototype — built and proven against this same control protocol — currently on its own branch (mcp-base), not yet merged/kept current with the mjlab work in §11. Merging it, and deciding whether it should eventually gain training tools too (the path already exists — Control already forwards those RPCs for the web UI, see above), is open work, not a finished door yet.

The rugiar skill

.claude/skills/rugiar/SKILL.md is a packaged set of instructions Claude Code loads automatically whenever a conversation touches training, fusing, or driving a robot in this repo — the assistant doesn't have to be re-taught this system's gotchas every session. It captures things a plain --help output can't: that reward/episode-length numbers alone don't prove a policy walks (a real past mistake here, see §13), the exact recipe for training a crouched-but-mobile policy without it turning wobbly, how to set up Kaggle cloud training, the fusion methods (weighted average and Git Re-Basin, and which policies each one supports), and the full WebSocket control protocol for anyone building a custom client. Kept next to the code it documents, in the same repo, so it's expected to be updated alongside a feature rather than drifting into stale advice.

The control web

The browser door — rugiar_driver.py --control_port 9013 serves it at http://localhost:9013, backed by the exact same ControlService the CLI and the WebSocket protocol (§13) talk to. A live screenshot from this fork's own development instance:

The info popup for walk_gpu_c4_hient2, open mid-session — the same PROVENANCE block policy_info() (§13's fuse_policies neighbor in the RPC table) hands back for any local policy: which task and simulator trained it, what it was cloned from, and the literal rugiar train command that reproduces it. Nothing about how a policy came to exist is hidden behind this panel.

Everything else in this page's earlier sections is a panel here too: policy switching with cross-fade (§12), pause/restart/E-STOP and manual velocity commands, live telemetry tagged by whether each field is a real sensor reading or simulator-only ground truth (§13), stress stimuli (random pushes/commands) for probing robustness, Create Policy for training/fine-tuning without hand-composing a command (§15), and — the newest addition — Fuse policies (§13), right under "+ New policy…", proposing every fusion method this build knows about — weighted average and Git Re-Basin are both selectable today, working for plain and recurrent policies alike, so the tradeoff between them is discoverable in the UI itself, not just in a code comment.

The Keyboard panel — every WASD/arrow-key hold maps to the same clamped set_command envelope §13 describes for any client, and the number-row shortcuts (0–9) map directly onto request_switch against the Policies list's own top-to-bottom order (below), so a whole session can be driven without touching the mouse.

The Create Policy panel itself — New policy / Fuse / Distill, and the Policies list they all write into — is pictured in §15's Quick start, right where the same three actions are given as rugiar commands.

Glossary

Credits & license

unitree_rl_gym is released under the BSD 3-Clause license and builds on several open projects: legged_gym (ETH Zürich Robotic Systems Lab) for the training environment, rsl_rl (also ETH Zürich) for the PPO implementation, MuJoCo (Google DeepMind) for sim2sim, and unitree_sdk2_python (Unitree Robotics) for real-hardware communication. This fork additionally builds on LeggedGym-Ex (the Genesis/Isaac Sim port) and Genesis itself. See the repo's root README for the full architecture write-up and setup instructions.