Skip to main content
Reactor returns predicted joint and gripper targets for each state request. Your application sends camera images and measured robot state, checks the returned targets, and executes them through your controller. It also handles motor timing, limits, and stopping behavior. The model expects DROID’s robot and camera setup. Adding or renaming joint values does not make another robot compatible; check its joint, gripper, and camera conventions before using the policy.
  1. Publish wrist and two exterior RGB views. Record capture times for the images and measured state so you can check how closely they line up.
  2. Encode the seven joint angles in the reference order and radians. Convert measured gripper state to a closed fraction (0 open, 1 closed).
  3. Select an available checkpoint for the session, set the task instruction, and submit one state request. Keep publishing new frames: the model waits for one from each camera after the request.
  4. Check that the reply’s step matches your request’s chunk_id. Check the (32, 8) shape, finite values, robot limits, and how old the observations are.
  5. Each row contains seven target joint angles and a gripper value. The action samples are spaced at 15 Hz; if your controller runs faster, convert them into the intermediate commands it needs. Convert the eighth value (column 7) to your gripper’s command format.
  6. Capture new images and measure the resulting robot state before requesting another chunk. Test how many action rows to execute before requesting a new prediction.
Joint targets are absolute angles in radians: move each joint to the returned angle, rather than adding that angle to its current position. The targets are already converted to robot units; do not apply the model’s training normalization again. Your local software must enforce limits, detect stalled predictions, execute movement, and check for collisions. This guide does not provide the connection to your physical robot’s hardware. Resending an identical request does not produce another reply. Changing the request can trigger a new prediction using new frames, even with the same request identifier. Do not execute two replies for the same intended request. After a disconnect or uncertain reset, start a new session and discard pending actions from the previous session.

Observation timing

Images and robot state travel separately. Record when you capture them, set a maximum age for each, and limit how far apart in time the camera views can be. A matching reply identifier does not prove that the images and state were captured together. Send fresh images; repeatedly publishing an old image does not make it a new observation. Measure how long it takes from capturing observations to executing their predicted actions, and how many actions are waiting in your controller’s queue. Test the complete loop in a simulator or with recorded observations before enabling movement on hardware.