> ## Documentation Index > Fetch the complete documentation index at: https://docs.reactor.inc/llms.txt > Use this file to discover all available pages before exploring further. > ## Agent Instructions > To build and serve your own model, start at /deploy/development/quickstart and /deploy/development/overview. Deploying is the default path: reactor init scaffolds a workspace, reactor auth login authenticates, and reactor model deploy registers the model, publishes the release with the weights/ folder, and activates it on Reactor's GPUs, in one command from that workspace. Docker must be running, because the publish step builds the image locally. Bump model.version in reactor.yaml before redeploying a change, because a release that already has an image is reactivated as it is. Deployment access is granted per account, so contact team@reactor.inc if a deploy is refused. Every key in reactor.yaml is documented at /deploy/platform/reactor-yaml. Model code imports reactor_runtime; Python client code imports reactor_sdk. The runtime overview explains the model interface. Running the model on your own machine with reactor run is optional and needs a GPU you attach with --gpus; /deploy/development/local-testing covers that loop and pairs a complete brightness model with a Python client test in a separate brightness-test workspace. > Reactor hosts multiple models, each with its own connect slug (modelName) and command/event schema. The video model catalog — slug, typed SDK package, and links to its schema — is at /model-api-reference/overview. Robotics policy documentation starts at /robotics/overview; The launch policies are Cosmos Nano DROID (/robotics/cosmos/nano-policy-droid/), DreamZero DROID (/robotics/dreamzero/droid/), and FLUX Action DROID (/robotics/flux-action/droid/), organized by model family and embodiment. Some models expose one slug per experience (e.g. HappyOyster); always take the slug from the model's own pages, never guess it. > Fastest path to a working app: `npx create-reactor-app my-app --model=` scaffolds a complete app with secure auth wired up. Typed TypeScript SDKs are published as @reactor-models/; Python uses the base reactor-sdk package. > Auth: exchange an API key (rk_...) for a JWT via POST https://api.reactor.inc/tokens from your server. Never put the API key in client-side code. > Append .md to any docs URL for clean Markdown. Search these docs via the MCP server at https://docs.reactor.inc/mcp. # Observations and actions > Camera tracks, robot state, action formats, and requesting the next prediction for Praxis-1 YAM. Connect to `reactor/praxis-1-yam` through the SDK. Keep the session open while you send observations and receive predictions. A **camera track** is a named stream of images from one camera. When you publish a track, the SDK attaches that stream to the session. ## API at a glance | Direction | What crosses the connection | | - | - | | Your client → model | Three camera streams, a task instruction, both arms' measured state, and an executed-step echo. | | Model → your client | `action_prediction` messages with end-effector pose and gripper targets. | The first chunk arrives when the session has a task, enough camera frames, and enough robot state. After your robot executes a chunk, echo its `step` to receive the next chunk. Your application validates the output and executes it with its own controller. New to Reactor? [How the API works](https://docs.reactor.inc/robotics/how-the-api-works.md) explains sessions, tracks, commands, and messages. ## Camera tracks Publish all three tracks after `READY` at 15 fps. Supply RGB `uint8` frames in 16:10 aspect, at most 416 pixels on the longer side. | Track | Camera | | - | - | | `top` | Top camera | | `wrist_left` | Left wrist camera | | `wrist_right` | Right wrist camera | Scale each view down and keep its aspect ratio, for example from 1920 × 1200 to 416 × 260. The model was trained on 16:10 views, and it refuses other aspect ratios. The session does not use a frame that is longer than 416 pixels on its longer side. It also starts that camera's window again, so it waits for 25 new frames that fit. Each call uses the last 25 frames of every camera (1.6 seconds at 15 fps). When a camera changes resolution, its window starts again. Video and state travel separately, and replies contain no frame timestamp. Keep the cameras streaming while the robot moves. ## Commands | SDK command | Payload | Constraint | | - | - | - | | `set_task` | `{"task": "Pick up the pen from the center of the table, then place the pen in the bag"}` | String, at most 300 characters and 64 tokens | | `set_robot_state_json` | `{"robot_state_json": ""}` | String, at most 32,000 characters | | `set_executed_step` | `{"executed_step": 0}` | The `step` of the chunk your robot finished executing | | `reset_context` | `{}` | Starts a new episode; replies `context_reset` | | `get_status` | `{}` | Replies `session_status` | A changed task applies to the next `action_prediction`. Write the task in the style of the training captions: plain clauses joined with `then`, with no closing period. A task can be within 300 characters and still be too long. The model accepts at most 64 tokens. A longer task fails with `task_too_long`. ## Robot state `robot_state_json` is a JSON string with the shape `{"left": {"ee_pose": [[x, y, z, qw, qx, qy, qz], ...], "grip": [g, ...]}, "right": {...}}`. Send one row for each control step at 15 Hz. Put the oldest row first and the current state last. Send at least the last 25 rows of each arm. To keep it short, the example shows one row. ```json theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}} { "left": { "ee_pose": [ [0.3, 0.2, 0.25, 1.0, 0.0, 0.0, 0.0] ], "grip": [0.0] }, "right": { "ee_pose": [ [0.3, -0.2, 0.25, 1.0, 0.0, 0.0, 0.0] ], "grip": [0.0] } } ``` Positions are in meters. Quaternions are scalar first (`qw, qx, qy, qz`), and they cannot be zero. Gripper values are absolute, and `0` is open. Send measured values, not the last requested targets. Send the state before `set_executed_step`. If a row contains a non-finite value, the session cannot use the state. ## Pose frame Report the poses of both arms in the world frame of the rig. Both arms were recorded in this frame during training. The model does not convert frames. The returned targets use the same world frame. ## Action reply The `data` object of the `action_prediction` message contains: | Field | Meaning | | - | - | | `step` | Chunk counter for the session: `0`, `1`, `2`, … | | `left_poses` | One target end-effector pose per row for the left arm: `[x, y, z]` in meters, then the 3 × 3 rotation matrix row by row (12 values) | | `left_grips` | One target gripper value per row for the left arm, absolute: `0` open | | `right_poses` | The right arm's targets, in the same format as `left_poses` | | `right_grips` | The right arm's gripper targets, in the same format as `left_grips` | | `timestamps` | Time offset of each row in seconds, from the first row | A chunk has 21 rows when it starts an episode and 24 rows (1.6 seconds at 15 Hz) after that. This is the control cadence. It is not a reply rate or a limit on inference latency. The reply does not include the predicted video. Pose targets are **absolute**, in the world frame of the rig. They are not joint angles, velocities, or increments. Use the controller of each arm to calculate its joint motion. The rows of the two arms with the same index have the same timestamp. Execute them together. There is no control-law field. Run the targets with the controller of each arm. ## Other messages | Message | When | Fields | | - | - | - | | `prediction_failed` | A chunk could not be computed | `step`, `code`, `reason` | | `session_status` | Reply to `get_status` | `waiting_for`, `next_step`, `awaiting_step` | | `context_reset` | Reply to `reset_context` | `next_step` | `prediction_failed.code` is one of: | Code | Meaning | | - | - | | `timeout` | The prediction did not finish in time. | | `busy` | The previous attempt is still running. | | `too_large` | A camera frame exceeds 416 pixels on its longer side. | | `rejected` | Reactor refused the prediction. | | `task_too_long` | The task has more than 64 tokens. | | `invalid_input` | The robot state or the frames could not be used. | After a failure, the session tries again about half a second later. That chunk starts a new episode. The session stays open. ## Advance the loop 1. Send the task, publish all camera tracks, and then send the measured robot state. The first prediction does not need an echo. It returns `step: 0`. 2. Before you execute the chunk, check the rows, the values, and the `step`. All values must be finite. Execute the chunk with your controller. 3. While the robot moves, keep publishing camera frames and send the measured state. 4. Send `set_executed_step` with the received `step`. The session then calculates the next chunk from the frames and state of the executed chunk. The next reply has the next `step`. 5. Repeat for each chunk. Make sure that `step` increases by one. Keep only one chunk outstanding at a time. Until you echo the last `step`, the session waits. `get_status` shows this in `waiting_for` and `awaiting_step`. An echo tells Reactor that your client made progress. Reactor does not check that the robot moved. ## Episodes and reset `reset_context` starts a new episode. The next chunk does not use the chunks that the robot executed before. It does not wait for `set_executed_step`, and it has 21 rows. The `step` counter does not restart. `context_reset.next_step` gives the next value. A new session starts at `step: 0`. A reset of the model does not reset or stop the physical robot. Use your controller to do that. This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.