reactor/xwam through the SDK and keep the session open while exchanging observations
and predictions. A camera track is a named stream of successive images from one camera;
publishing it attaches that stream to the session.
API at a glance
Send a new request and match its
chunk_id to the returned step. Your application validates the
output and executes it through its own controller.
New to Reactor? How the API works explains sessions, tracks,
commands, and the difference between an SDK message and the data returned by the example helper. For
a complete script, use Get your first actions.
Camera tracks
Publish all three named video tracks. The reference client sends RGBuint8 arrays with shape
(240, 320, 3) at 15 frames/s. This is the camera publishing rate, not the robot control rate.
Keep each track publishing while awaiting a reply, repeating the current observation when needed.
The model waits for a frame from every view received at least as recently as the request. Frame
arrival does not prove that the cameras captured simultaneously; your client owns capture alignment.
Swapping cameras under the correct track names is not detected by the API.
Commands
Send model commands withawait reactor.send_command(name, payload) after the session is READY.
Prediction request
state_json contains a JSON string, not a nested object.
Omit the seed fields for a new integration. Set them when replaying a recorded request whose seed
components are known. The server normalizes state and denormalizes outputs; send physical state in
the checkpoint’s conventions, without applying training-statistic normalization yourself.
State layout
Indices are zero-based; slice end indices are exclusive.
The checkpoint frame differs from RoboTwin’s raw end-pose frame. Apply the
coordinate mapping before sending state.
Action reply
The SDK message envelope is{"type": "action_prediction", "data": {...}}.
Each action row has this layout:
Each row is relative to the previous step, starting from the request’s state. Accumulate
translation and gripper deltas; compose rotations as
R_next = R_delta @ R_previous. These are
end-effector commands, not joint angles or motor commands. See
action execution.
Ordering and retries
Allow only one request at a time. Matchstep to the outstanding chunk_id; discard other replies
and consume a matching reply only once. A second state update can replace an unanswered request.
An unchanged state_json string is deduplicated. To retry, preserve the observation, chunk_id,
and seed fields and increment retry. This requests another prediction with the same sampling seed;
lossy video transport means numerical equality is not guaranteed. A retry does not authorize
executing a chunk twice.
At an episode boundary, stop execution, discard queued replies, send reset, and set the new task
and observations. Keep request IDs increasing within the session. A reply already in transit can
still arrive after reset.