> ## Documentation Index
> Fetch the complete documentation index at: https://docs.reactor.inc/llms.txt
> Use this file to discover all available pages before exploring further.

> ## Agent Instructions
> To build and serve your own model, start at /deploy/development/quickstart and /deploy/development/overview. Deploying is the default path: reactor init scaffolds a workspace, reactor auth login authenticates, and reactor model deploy registers the model, publishes the release with the weights/ folder, and activates it on Reactor's GPUs, in one command from that workspace. Docker must be running, because the publish step builds the image locally. Bump model.version in reactor.yaml before redeploying a change, because a release that already has an image is reactivated as it is. Deployment access is granted per account, so contact team@reactor.inc if a deploy is refused. Every key in reactor.yaml is documented at /deploy/platform/reactor-yaml. Model code imports reactor_runtime; Python client code imports reactor_sdk. The runtime overview explains the model interface. Running the model on your own machine with reactor run is optional and needs a GPU you attach with --gpus; /deploy/development/local-testing covers that loop and pairs a complete brightness model with a Python client test in a separate brightness-test workspace.
> Reactor hosts multiple models, each with its own connect slug (modelName) and command/event schema. The video model catalog — slug, typed SDK package, and links to its schema — is at /model-api-reference/overview. Robotics policy documentation starts at /robotics/overview; X-WAM observations, actions, and client integration are under /robotics/xwam/; Cosmos3 Nano Policy DROID is under /robotics/cosmos/nano-policy-droid/. Some models expose one slug per experience (e.g. HappyOyster); always take the slug from the model's own pages, never guess it.
> Fastest path to a working app: `npx create-reactor-app my-app --model=<slug>` scaffolds a complete app with secure auth wired up. Typed TypeScript SDKs are published as @reactor-models/<model>; Python uses the base reactor-sdk package.
> Auth: exchange an API key (rk_...) for a JWT via POST https://api.reactor.inc/tokens from your server. Never put the API key in client-side code.
> Append .md to any docs URL for clean Markdown. Search these docs via the MCP server at https://docs.reactor.inc/mcp.

# Inputs and actions

> Camera tracks, JSON requests, checkpoint commands, and action replies.

Connect to `reactor/flux3-action-droid` through the SDK and keep the session open while exchanging
observations and predictions. A **camera track** is a named stream of successive images from one
camera; publishing it attaches that stream to the session.

## API at a glance

| Direction | What crosses the connection |
| - | - |
| Your client → model | Three camera streams, a task instruction, and one eight-value state request at a time. |
| Model → your client | `action_prediction` messages; `data.actions` contains 32 × 8 absolute joint/gripper targets. |

Send a new state request and match its `chunk_id` to the returned `step`. Select a checkpoint before
the first request. Your application validates the output and executes it through its own controller.

New to Reactor? [How the API works](/robotics/how-the-api-works) explains sessions, tracks,
commands, and the difference between an SDK message and the data returned by the example helper. For
a complete script, use [Get your first actions](/robotics/flux-action/droid/quickstart).

## Camera tracks

| Track | View |
| - | - |
| `wrist_view` | Wrist camera |
| `exterior_view_1` | First exterior camera |
| `exterior_view_2` | Second exterior camera |

Publish RGB `uint8` images in height × width × 3 order. The reference client uses 360 × 640 per
track at 15 fps. The policy composes a 360 × 640 wrist view above two 180 × 320 exterior views;
different aspect ratios are resized, so preserve the reference camera geometry.

Every request waits for a newly delivered frame on each track after the request is accepted. Track
arrival is not proof of simultaneous capture with the supplied state. Keep capture timestamps in
your client.

## Commands

| Command | Payload | Effect |
| - | - | - |
| `set_task_description` | `{"task_description": "put the marker in the cup"}` | Required nonempty instruction, at most 300 characters; applies to the next request. |
| `set_state_json` | `{"state_json": "<JSON string>"}` | One prediction request; string limit 2,000 characters. |
| `get_checkpoint` | `{}` | Returns `checkpoint_selected`. |
| `select_checkpoint` | `{"checkpoint": "base-bf16"}` | Pins an available checkpoint before the first prediction. |
| `reset` | `{}` | Clears episode memory and an unanswered request; retains instruction, tracks, and checkpoint. |

The inner `state_json` object is:

```json theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}}
{ "proprio": [0.0, -0.6, 0.0, -2.2, 0.0, 1.6, 0.8, 0.0], "chunk_id": 1, "seed": 1 }
```

| Field | Meaning |
| - | - |
| `proprio` | Eight finite numbers: seven measured joint angles in radians, then gripper closed fraction (`0` open, `1` closed). |
| `chunk_id` | Nonnegative integer request identifier; returned as `step`. |
| `seed` | Optional nonnegative integer; defaults to `chunk_id`. |

Each distinct request string is answered once. A byte-identical resend produces no second reply.
Unknown inner keys are ignored, so adding `"retry": 1` requests another prediction with the same
identifier and seed. New frames can change its actions; retries are not cached replays. Keep one
request outstanding and consume a request identifier only once in the controller. Invalid requests
can be dropped without a prediction.

## Messages

The SDK receives `{"type": "action_prediction", "data": {...}}`. Its data contains:

| Field | Meaning |
| - | - |
| `actions` | 32 rows × 8 columns: absolute joint targets in radians (`0:7`), then gripper closed fraction (`7`). |
| `step` | Echo of `chunk_id`; discard mismatched replies. |
| `checkpoint` | Checkpoint used for this prediction. |
| `inference_seconds` | Model prediction time, excluding transport. |

Row `k` targets `(k + 1) / 15` seconds after the observation: a full chunk spans about 2.13 seconds.
Validate finite values and your controller limits before use. The API does not execute targets or
compensate for network delay.

`checkpoint_selected` reports `checkpoint`, `locked`, and `available`. `command_error` reports
`command` and a human-readable `reason`; the session stays open. Register handlers before requesting
a selection or prediction.

## Episode boundary

After `reset`, the existing `state_json` value is not answered again. Submit a new request with
current observations. Clear local reply/action queues and do not execute a late reply from the
previous episode. Use a new session when the boundary is uncertain or when changing checkpoints.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.