> ## Documentation Index
> Fetch the complete documentation index at: https://docs.reactor.inc/llms.txt
> Use this file to discover all available pages before exploring further.

> ## Agent Instructions
> To build and serve your own model, start at /deploy/development/quickstart and /deploy/development/overview. Deploying is the default path: reactor init scaffolds a workspace, reactor auth login authenticates, and reactor model deploy registers the model, publishes the release with the weights/ folder, and activates it on Reactor's GPUs, in one command from that workspace. Docker must be running, because the publish step builds the image locally. Bump model.version in reactor.yaml before redeploying a change, because a release that already has an image is reactivated as it is. Deployment access is granted per account, so contact team@reactor.inc if a deploy is refused. Every key in reactor.yaml is documented at /deploy/platform/reactor-yaml. Model code imports reactor_runtime; Python client code imports reactor_sdk. The runtime overview explains the model interface. Running the model on your own machine with reactor run is optional and needs a GPU you attach with --gpus; /deploy/development/local-testing covers that loop and pairs a complete brightness model with a Python client test in a separate brightness-test workspace.
> Reactor hosts multiple models, each with its own connect slug (modelName) and command/event schema. The video model catalog — slug, typed SDK package, and links to its schema — is at /model-api-reference/overview. Robotics policy documentation starts at /robotics/overview; The launch policies are Cosmos Nano DROID (/robotics/cosmos/nano-policy-droid/), DreamZero DROID (/robotics/dreamzero/droid/), and FLUX Action DROID (/robotics/flux-action/droid/), organized by model family and embodiment. Some models expose one slug per experience (e.g. HappyOyster); always take the slug from the model's own pages, never guess it.
> Fastest path to a working app: `npx create-reactor-app my-app --model=<slug>` scaffolds a complete app with secure auth wired up. Typed TypeScript SDKs are published as @reactor-models/<model>; Python uses the base reactor-sdk package.
> Auth: exchange an API key (rk_...) for a JWT via POST https://api.reactor.inc/tokens from your server. Never put the API key in client-side code.
> Append .md to any docs URL for clean Markdown. Search these docs via the MCP server at https://docs.reactor.inc/mcp.

# Genie 4 schema reference

> Genie 4 output tracks, world lifecycle, commands, refusals, and session snapshots.

Genie 4 produces video and audio in a Reactor session. Send commands with `reactor.sendCommand()` or
the typed SDK. This page describes the `reactor/genie-4` contract. See the
[overview](https://docs.reactor.inc/model-api-reference/genie-4/overview.md) for a quick start.

## Tracks

| Name | Type | Content |
| - | - | - |
| `main_video` | Video | 1280 × 720 at 24 fps during a live world. A dark placeholder otherwise. |
| `main_audio` | Audio | 48 kHz mono world sound. Silence outside a live world. |

There are no input tracks. Attach both outputs before starting a world. Handle blocked audio
playback with a user control. A connected track alone does not establish that the world is live or
that the browser plays its sound.

## Session lifecycle

One session holds at most one world at a time. The phases are `idle`, `starting`, `live`, `ending`,
`ended`, and `failed`. Call `start_world` from `idle`, `ended`, or `failed`. The first generated
picture moves the world from `starting` to `live`.

A world lasts up to 60 seconds live. `end_world` stops it early and leaves the Reactor session open.
Starting again creates a new world. Supply the prompt or image again, because a world cannot resume.

Subscribe to `session_state` before connecting, then await `get_state` for a fresh snapshot. Clear
local snapshots when disconnected. After reconnecting, read `get_state` again before enabling world
controls.

## Commands

| Command | Description | Result on the awaited call |
| - | - | - |
| `start_world` | Create a world from text, an image, or both. | No reply body. Observe phase changes. |
| `set_keys` | Replace the held keys for this client. | No reply body. Observe state changes. |
| `instruct` | Change the live scene or ask someone in it to speak. | No reply body. Observe the world. |
| `end_world` | Stop the current world. Safe to repeat. | `session_state` |
| `get_state` | Read the current snapshot. | `session_state` |

The JavaScript SDK reports refused commands through `error`, or React `lastError`, with
`operation: "sendCommand"`. Calls resolve `undefined` on failure and do not reject. For the three
commands without a reply body, success also resolves `undefined`. The Python SDK raises
`ReactorError` for a refusal. See [Refusals](#refusals).

The React examples below render inside `Genie4Provider`. Its provider declares the model and tracks.
See the [tutorial](https://docs.reactor.inc/model-api-reference/genie-4/tutorial.md) for authentication and media playback.

### `start_world`

Start from `idle`, `ended`, or `failed`. Supply a nonblank prompt, an image, or both. The prompt and
image apply only at world start. Use `instruct` for later scene changes.

| Parameter | Type | Required | Meaning |
| - | - | - | - |
| `prompt` | string | No | Up to 2,000 characters. Defaults to `""`. Describe the place, time, surroundings, and camera. |
| `reference_image` | upload reference or null | No | PNG, JPEG, or WEBP, up to 20 MiB and 40 megapixels. Defaults to null. |

Upload the image through `uploadFile()` and use the returned reference. Reference images work best
at a 16:9 aspect ratio; other shapes are center-cropped. Do not pass a file path or a URL in
`reference_image`. For an image-only start, omit `prompt` or pass `prompt: ""`. Do not set it to
`undefined` or `null`.

```typescript JavaScript SDK theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}}
await reactor.sendCommand("start_world", {
  prompt: "First-person view on a cobblestone street in a mountain village.",
});
// For an image: const reference = await reactor.uploadFile(imageFile);
// await reactor.sendCommand("start_world", { reference_image: reference });
```

```tsx React typed SDK theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}}
import { useGenie4 } from "@reactor-models/genie-4";

export function StartButton() {
  const { startWorld } = useGenie4();
  return (
    <button onClick={() => void startWorld({ prompt: "First-person view in a mountain village." })}>
      Start world
    </button>
  );
}
```

```python Python theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}}
await reactor.send_command("start_world", {
    "prompt": "First-person view in a mountain village.",
})
```

Watch for `phase: "live"`. A later failure appears as `phase: "failed"` with `last_error`. A refused
start uses `busy`, `invalid_input`, or `invalid_image`.

### `set_keys`

Replace every key this client holds. Valid keys are `w`, `a`, `s`, `d`, `e`, literal space `" "`,
`arrowup`, `arrowdown`, `arrowleft`, and `arrowright`.

| Parameter | Type | Required | Meaning |
| - | - | - | - |
| `keys` | array of key strings | Yes | The complete held set, up to 10 keys. `[]` releases every key held by this client. |

Send an update whenever a key is pressed or released. Send `[]` when the final key is released.
Release held keys when someone switches tabs or apps, or clicks away from the controls. Also release
them when a mouse or touch press ends or is canceled. Ignore keyboard presses while someone types
in a text field.

Keys sent during `starting` take effect when the world becomes `live`. Keys sent before starting a
world or after it ends are ignored.

```typescript JavaScript SDK theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}}
await reactor.sendCommand("set_keys", { keys: ["w", "arrowleft"] });
await reactor.sendCommand("set_keys", { keys: [] });
```

```tsx React typed SDK theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}}
import { useGenie4 } from "@reactor-models/genie-4";

export function ForwardButton() {
  const { setKeys } = useGenie4();
  const release = () => void setKeys({ keys: [] });
  return (
    <button
      onPointerDown={(event) => {
        event.currentTarget.setPointerCapture(event.pointerId);
        void setKeys({ keys: ["w"] });
      }}
      onPointerUp={release}
      onPointerCancel={release}
      onLostPointerCapture={release}
      onBlur={release}
    >
      Hold to move forward
    </button>
  );
}
```

```python Python theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}}
await reactor.send_command("set_keys", {"keys": ["w", "arrowleft"]})
await reactor.send_command("set_keys", {"keys": []})
```

Disconnecting releases held keys.

### `instruct`

Change the scene or give a character a line while the world is `live`. A new instruction replaces
one still being delivered.

| Parameter | Type | Required | Meaning |
| - | - | - | - |
| `text` | string | Yes | Nonblank instruction, 1 to 2,000 characters. |

```typescript JavaScript SDK theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}}
await reactor.sendCommand("instruct", { text: "It starts to snow heavily." });
```

```tsx React typed SDK theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}}
import { useGenie4 } from "@reactor-models/genie-4";

export function SnowButton() {
  const { instruct } = useGenie4();
  return (
    <button onClick={() => void instruct({ text: "It starts to snow heavily." })}>Snow</button>
  );
}
```

```python Python theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}}
await reactor.send_command("instruct", {"text": "It starts to snow heavily."})
```

Refuses with `not_live` outside `live`, or `invalid_input` for blank text.

### `end_world`

Stop the current world and await its `session_state` reply. This command accepts an empty object. It
is valid in every phase and safe to repeat. If no world runs, it returns the settled state. When it
stops a world, the phase becomes `ended` with `end_reason: "ended_by_client"`.

```typescript JavaScript SDK theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}}
const reply = await reactor.sendCommand("end_world", {});
if (reply) console.log(reply.data);
```

```tsx React typed SDK theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}}
import { useGenie4 } from "@reactor-models/genie-4";

export function EndButton() {
  const { endWorld } = useGenie4();
  return <button onClick={async () => console.log((await endWorld())?.phase)}>End world</button>;
}
```

```python Python theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}}
reply = await reactor.send_command("end_world", {})
print(reply)
```

Typed replies expose the snapshot fields directly: read `reply.phase`. The base JavaScript result
uses `reply.data.phase`. Use the awaited reply for this request. Keep subscriptions for broadcasts.

### `get_state`

Read the current `session_state` snapshot in any phase. This command accepts an empty object.

```typescript JavaScript SDK theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}}
const reply = await reactor.sendCommand("get_state", {});
if (reply) console.log(reply.data);
```

```tsx React typed SDK theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}}
import { useGenie4 } from "@reactor-models/genie-4";

export function RefreshButton() {
  const { getState } = useGenie4();
  return <button onClick={async () => console.log((await getState())?.phase)}>Read state</button>;
}
```

```python Python theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}}
reply = await reactor.send_command("get_state", {})
print(reply)
```

## Refusals

| Code | Command | Meaning |
| - | - | - |
| `busy` | `start_world` | A world is `starting`, `live`, or `ending`. |
| `invalid_input` | `start_world`, `instruct` | Missing world inputs or blank instruction. |
| `invalid_image` | `start_world` | The image cannot be read or fails image limits. |
| `not_live` | `instruct` | The world is outside `live`. |

Schema validation can also report `invalid_command` before the handler runs, for example for a value
with the wrong type. The four codes above are model-handler refusals.

These refusals use the SDK error path. There is no `command_error` broadcast. Keep them separate
from connection status and from later world failures in `last_error`. See
[Surfacing refusals](https://docs.reactor.inc/model-api-reference/genie-4/tutorial.md#surfacing-refusals) for React handling.

## Messages

| Message | When | Meaning |
| - | - | - |
| `session_state` | On connect, on changes, and about once a second while live. Also the reply to `end_world` and `get_state`. | Complete snapshot of the world. |

Broadcasts reach every connected client. A command reply goes to the client that sent the command.
Base SDK messages use `{ type: "session_state", data: snapshot }`. Typed hooks and typed replies
expose snapshot fields directly, such as `state.phase`.

### session\_state payload

| Field | Type | Meaning |
| - | - | - |
| `phase` | string enum | `idle`, `starting`, `live`, `ending`, `ended`, or `failed`. |
| `has_prompt` | boolean | Current or last world started with a prompt. |
| `has_reference_image` | boolean | Current or last world started with an image. |
| `keys` | key string array | Combined held keys from all clients. Empty unless live. |
| `video_receiving` | boolean | World video currently arrives at the model output. |
| `audio_receiving` | boolean | World audio currently arrives at the model output. |
| `started_at` | number or null | Unix seconds when the current or last world became live. |
| `max_seconds` | integer | World duration limit. Currently 60. |
| `elapsed_seconds` | number | Seconds live. Zero before live, frozen after the world ends. |
| `end_reason` | string or null | Reason the world ended. Null while it runs. |
| `last_error` | object or null | World failure, cleared when a new start begins. |

The receiving flags describe the model media output. The browser can still block playback. For a
countdown, use `Math.max(0, max_seconds - elapsed_seconds)`.

### end\_reason values

| Value | Phase | Meaning |
| - | - | - |
| `ended_by_client` | `ended` | A client ended the world, or the session ended. |
| `max_duration` | `ended` | The live duration limit elapsed. |
| `start_failed` | `failed` | The generation service could not start the world. |
| `no_stream` | `failed` | No picture arrived within 20 seconds. |
| `media_lost` | `failed` | Pictures stopped arriving for 10 seconds. |
| `upstream_interrupted` | `failed` | The generation service stopped the world early. |
| `platform_failed` | `failed` | A platform failure stopped the world. |

### last\_error fields

| Field | Type | Meaning |
| - | - | - |
| `origin` | string | `upstream` for the generation service, or `platform`. |
| `code` | string | Stable failure code. |
| `reason` | string | Explanation to show the user. |
| `retryable` | boolean | Starting again unchanged may work. |
| `upstream_code` | string or null, optional | Code from the generation service, when available. |
| `trace_id` | string or null, optional | Identifier to include in a failure report. |

World failure codes include `upstream_capacity`, `upstream_unavailable`, `upstream_rejected`,
`upstream_interrupted`, `upstream_auth`, `upstream_protocol`, `content_policy`, `no_stream`,
`media_stalled`, and `platform_failed`. Show `reason` and use `retryable` when offering another
start.

## Complete example

The [tutorial](https://docs.reactor.inc/model-api-reference/genie-4/tutorial.md) provides a complete Next.js app with scoped
authentication, video and audio playback, snapshots, held keys, live instructions, and end/restart
controls. For a shorter base SDK connection, see the
[overview](https://docs.reactor.inc/model-api-reference/genie-4/overview.md#quick-start).


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.