> ## Documentation Index > Fetch the complete documentation index at: https://docs.reactor.inc/llms.txt > Use this file to discover all available pages before exploring further. > ## Agent Instructions > To build and serve your own model, start at /deploy/development/quickstart and /deploy/development/overview. Deploying is the default path: reactor init scaffolds a workspace, reactor auth login authenticates, and reactor model deploy registers the model, publishes the release with the weights/ folder, and activates it on Reactor's GPUs, in one command from that workspace. Docker must be running, because the publish step builds the image locally. Bump model.version in reactor.yaml before redeploying a change, because a release that already has an image is reactivated as it is. Deployment access is granted per account, so contact team@reactor.inc if a deploy is refused. Every key in reactor.yaml is documented at /deploy/platform/reactor-yaml. Model code imports reactor_runtime; Python client code imports reactor_sdk. The runtime overview explains the model interface. Running the model on your own machine with reactor run is optional and needs a GPU you attach with --gpus; /deploy/development/local-testing covers that loop and pairs a complete brightness model with a Python client test in a separate brightness-test workspace. > Reactor hosts multiple models, each with its own connect slug (modelName) and command/event schema. The video model catalog — slug, typed SDK package, and links to its schema — is at /model-api-reference/overview. Robotics policy documentation starts at /robotics/overview; The launch policies are Cosmos Nano DROID (/robotics/cosmos/nano-policy-droid/), DreamZero DROID (/robotics/dreamzero/droid/), and FLUX Action DROID (/robotics/flux-action/droid/), organized by model family and embodiment. Some models expose one slug per experience (e.g. HappyOyster); always take the slug from the model's own pages, never guess it. > Fastest path to a working app: `npx create-reactor-app my-app --model=` scaffolds a complete app with secure auth wired up. Typed TypeScript SDKs are published as @reactor-models/; Python uses the base reactor-sdk package. > Auth: exchange an API key (rk_...) for a JWT via POST https://api.reactor.inc/tokens from your server. Never put the API key in client-side code. > Append .md to any docs URL for clean Markdown. Search these docs via the MCP server at https://docs.reactor.inc/mcp. # Observations and actions > Camera tracks, robot state, action formats, and requesting the next prediction for Praxis-1 DROID. Connect to `reactor/praxis-1-droid` through the SDK. Keep the session open while you send observations and receive predictions. A **camera track** is a named stream of images from one camera. When you publish a track, the SDK attaches that stream to the session. ## API at a glance | Direction | What crosses the connection | | - | - | | Your client → model | Two camera streams, a task instruction, the arm's measured state, and an executed-step echo. | | Model → your client | `action_prediction` messages with end-effector pose and gripper targets. | The first chunk arrives when the session has a task, enough camera frames, and enough robot state. After your robot executes a chunk, echo its `step` to receive the next chunk. Your application validates the output and executes it with its own controller. New to Reactor? [How the API works](https://docs.reactor.inc/robotics/how-the-api-works.md) explains sessions, tracks, commands, and messages. ## Camera tracks Publish both tracks after `READY` at 15 fps. Supply RGB `uint8` frames no larger than 512 × 288 (width × height). | Track | Camera | | - | - | | `exterior` | Exterior camera | | `wrist` | Wrist camera | If a frame is larger, scale it down and keep its aspect ratio before you publish it. The session does not use a frame that is larger than 512 × 288. It also starts that camera's window again, so it waits for 25 new frames that fit. Each call uses the last 25 frames of every camera (1.6 seconds at 15 fps). When a camera changes resolution, its window starts again. Video and state travel separately, and replies contain no frame timestamp. Keep the cameras streaming while the robot moves. ## Commands | SDK command | Payload | Constraint | | - | - | - | | `set_task` | `{"task": "Put the red block in the bowl."}` | String, at most 300 characters and 64 tokens | | `set_pose_frame` | `{"pose_frame": "fr3_hand"}` | One of the values under [Pose frame](#pose-frame) | | `set_robot_state_json` | `{"robot_state_json": ""}` | String, at most 16,000 characters | | `set_executed_step` | `{"executed_step": 0}` | The `step` of the chunk your robot finished executing | | `reset_context` | `{}` | Starts a new episode; replies `context_reset` | | `get_status` | `{}` | Replies `session_status` | A changed task applies to the next `action_prediction`. Write the task in plain words, as a sentence. A task can be within 300 characters and still be too long. The model accepts at most 64 tokens. A longer task fails with `task_too_long`. ## Robot state `robot_state_json` is a JSON string with the shape `{"ee_pose": [[x, y, z, qw, qx, qy, qz], ...], "grip": [g, ...]}`. Send one row for each control step at 15 Hz. Put the oldest row first and the current state last. Send at least the last 25 rows. To keep it short, the example shows one row. ```json theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}} { "ee_pose": [ [0.45, 0.0, 0.35, 0.0, 1.0, 0.0, 0.0] ], "grip": [0.0] } ``` Positions are in meters. Quaternions are scalar first (`qw, qx, qy, qz`), and they cannot be zero. Gripper values are absolute, and `0` is open. Send measured values, not the last requested targets. Send the state before `set_executed_step`. If a row contains a non-finite value, the session cannot use the state. ## Pose frame Select the frame that your robot uses to report its end-effector pose. The model converts your poses to the frame of its training data. It converts the returned targets back to your frame, so the targets use the frame that you sent. | `pose_frame` | Frame | | - | - | | `fr3_hand` | The Franka FR3 hand frame (default). | | `robolab_droid` | The DROID robot frame of NVIDIA RoboLab. | | `molmo_spaces_droid` | The DROID robot frame of MolmoSpaces. | ## Action reply The `data` object of the `action_prediction` message contains: | Field | Meaning | | - | - | | `step` | Chunk counter for the session: `0`, `1`, `2`, … | | `poses` | One target end-effector pose per row: `[x, y, z]` in meters, then the 3 × 3 rotation matrix row by row (12 values) | | `grips` | One target gripper value per row, absolute: `0` open | | `timestamps` | Time offset of each row in seconds, from the first row | | `control_json` | The control law to run the chunk with, as a JSON string; empty when your arm's own control law applies | A chunk has 21 rows when it starts an episode and 24 rows (1.6 seconds at 15 Hz) after that. This is the control cadence. It is not a reply rate or a limit on inference latency. The reply does not include the predicted video. Pose targets are **absolute**, in the frame that `pose_frame` selects. They are not joint angles, velocities, or increments. Use your controller to calculate the joint motion. `control_json` contains the impedance gains of the DROID demonstrations: the joint and Cartesian stiffness and damping. Its format is `{"type": "impedance", "kq": [7 values], "kqd": [7 values], "kx": [6 values], "kxd": [6 values]}`. The gains are the same for all rows of a chunk. If your controller supports impedance control, apply them. If not, use them as a reference to tune your controller. ## Other messages | Message | When | Fields | | - | - | - | | `prediction_failed` | A chunk could not be computed | `step`, `code`, `reason` | | `session_status` | Reply to `get_status` | `waiting_for`, `next_step`, `awaiting_step` | | `context_reset` | Reply to `reset_context` | `next_step` | `prediction_failed.code` is one of: | Code | Meaning | | - | - | | `timeout` | The prediction did not finish in time. | | `busy` | The previous attempt is still running. | | `too_large` | A camera frame exceeds 512 × 288. | | `rejected` | Reactor refused the prediction. | | `task_too_long` | The task has more than 64 tokens. | | `invalid_input` | The robot state or the frames could not be used. | After a failure, the session tries again about half a second later. That chunk starts a new episode. The session stays open. ## Advance the loop 1. Send the task, publish all camera tracks, and then send the measured robot state. The first prediction does not need an echo. It returns `step: 0`. 2. Before you execute the chunk, check the rows, the values, and the `step`. All values must be finite. Execute the chunk with your controller. 3. While the robot moves, keep publishing camera frames and send the measured state. 4. Send `set_executed_step` with the received `step`. The session then calculates the next chunk from the frames and state of the executed chunk. The next reply has the next `step`. 5. Repeat for each chunk. Make sure that `step` increases by one. Keep only one chunk outstanding at a time. Until you echo the last `step`, the session waits. `get_status` shows this in `waiting_for` and `awaiting_step`. An echo tells Reactor that your client made progress. Reactor does not check that the robot moved. ## Episodes and reset `reset_context` starts a new episode. The next chunk does not use the chunks that the robot executed before. It does not wait for `set_executed_step`, and it has 21 rows. The `step` counter does not restart. `context_reset.next_step` gives the next value. A new session starts at `step: 0`. A reset of the model does not reset or stop the physical robot. Use your controller to do that. This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.