Status
Keyboard
Hold to accelerate, release to decelerate smoothly back to a stop — press the opposite key or Esc to change it.
Pause & Restart
Pause/Restart affect the sim run itself, separate from driving it or switching policies. Auto-restart timer is off by default — the robot only resets when you hit Restart (or on a real fall) — turn it on to get the training-style episode timeout back.
Live Telemetry
What the running robot's state actually is, right now — read these before picking a Create Policy target so you're choosing a real number, not guessing. sensor is a real IMU/gyroscope reading, available on both sim and real hardware. sim only exists in simulation — no equivalent sensor exists on the physical robot, so it can't be an input to a real-robot policy, only a training-time target.
Waiting for data…
Camera
What the robot sees, from a camera positioned roughly where its head-mounted sensor would be — a robot-POV RGB feed, separate from the third-person Simulator view. Hidden unless the server was started with --camera.
Command
Drag any indicator, use W/A/S/D and all four arrow keys (↑/↓ are aliases for W/S), or the mouse-look pad to set the velocity command. +X forward, +Y left, +Yaw turns left.
Building your own controller (gamepad, custom hardware)? This panel sends the exact same WebSocket calls a home-made joystick would — see Talking to the robot: the control protocol in the Docs tab.
Turn off if another client (a gamepad, the MCP server, a home-made controller) is driving this robot from elsewhere — this page's own keys/drag/mouse-look stop sending commands and stop echoing the server's command back into a local decel-to-zero coast, so it won't fight the other client.
Fraction of the trained command envelope every client can reach — enforced server-side, so this caps the HUDs/keyboard here AND any other connected client (a gamepad, a future robot) the same way. 100% is the full trained range (this repo's and Unitree's own default, e.g. ±1.0 m/s on G1). Lower it for a more conservative session.
⚠ Past 100% asks the policy for a velocity it was never trained on — deliberately out-of-distribution. Expect it to behave unpredictably or fall. Sim only: this is never available against real hardware.
Family
Which registered task this whole session is running (e.g. "g1" walking vs "g1_target" standing) — different from picking a Policy below, which only switches WITHIN the current family. Switching family restarts this server's process for the new task (10-20s) — the page reconnects on its own once it's back. Only tasks with at least one trained local policy are shown. If the Simulator view stays frozen once it's back, use the ↻ button to reconnect it manually — this isn't automatic because reloading it is CPU-heavy and can itself slow down the robot's control loop, so it's your call when to pay that cost.
Motion
Reference-motion clips this task can track (mjlab's Rugiar-G1-Mimic only —
hidden entirely on a backend with no motion-tracking concept, e.g. Genesis). Picking a clip
restarts this server's process against it, same 10-20s reconnect as a Family switch above.
Clips marked no policy have no local policy trained against them yet — picking one is NOT disabled: the reference-motion "ghost" overlay plays regardless of what's driving the robot mesh, so you can preview a clip (the robot holding a neutral pose, or whatever policy happens to be active) before ever training anything against it.
Policies
Click a policy to switch, or press its shortcut key. Shortcuts are fixed to position, not name — drag ⋮⋮ to reorder: top is 0, then 9 down to 1 (up to 10 policies).
Stress Stimuli
Training runs this policy through random shoves and a velocity command that changes on its own. Turn both off to drive it deliberately instead — like a real operator would, not like a training run.