> ## Documentation Index
> Fetch the complete documentation index at: https://docs.reactor.inc/llms.txt
> Use this file to discover all available pages before exploring further.

> ## Agent Instructions
> To build and serve your own model, start at /deploy/development/quickstart and /deploy/development/overview. Deploying is the default path: reactor init scaffolds a workspace, reactor auth login authenticates, and reactor model deploy registers the model, publishes the release with the weights/ folder, and activates it on Reactor's GPUs, in one command from that workspace. Docker must be running, because the publish step builds the image locally. Bump model.version in reactor.yaml before redeploying a change, because a release that already has an image is reactivated as it is. Deployment access is granted per account, so contact team@reactor.inc if a deploy is refused. Every key in reactor.yaml is documented at /deploy/platform/reactor-yaml. Model code imports reactor_runtime; Python client code imports reactor_sdk. The runtime overview explains the model interface. Running the model on your own machine with reactor run is optional and needs a GPU you attach with --gpus; /deploy/development/local-testing covers that loop and pairs a complete brightness model with a Python client test in a separate brightness-test workspace.
> Reactor hosts multiple models, each with its own connect slug (modelName) and command/event schema. The video model catalog — slug, typed SDK package, and links to its schema — is at /model-api-reference/overview. Robotics policy documentation starts at /robotics/overview; X-WAM observations, actions, and client integration are under /robotics/xwam/; Cosmos3 Nano Policy DROID is under /robotics/cosmos/nano-policy-droid/. Some models expose one slug per experience (e.g. HappyOyster); always take the slug from the model's own pages, never guess it.
> Fastest path to a working app: `npx create-reactor-app my-app --model=<slug>` scaffolds a complete app with secure auth wired up. Typed TypeScript SDKs are published as @reactor-models/<model>; Python uses the base reactor-sdk package.
> Auth: exchange an API key (rk_...) for a JWT via POST https://api.reactor.inc/tokens from your server. Never put the API key in client-side code.
> Append .md to any docs URL for clean Markdown. Search these docs via the MCP server at https://docs.reactor.inc/mcp.

# FastH3-Omni schema

> Messages, tracks, and commands for FastH3-Omni.

This page lists FastH3-Omni's messages, tracks, and commands. Wait for the session to be ready before sending commands.

## Tracks

| Track | Direction | Type |
| - | - | - |
| `main_video` | Output | Video, 24 fps |
| `main_audio` | Output | Audio |

## Session lifecycle

When you connect to FastH3-Omni, the session starts with empty generation and playout queues, autoplay off, and no clip playing. The initial `state_update` shows the session settings, queue capacities, and available commands. Check its `valid_commands` field to see which commands you can send in the current state. The initial `queue_update` shows the clips waiting to generate or play; both lists start empty.

Send `enqueue` to add your first clip to the generation queue. When generation finishes, the model sends `clip_generated` and moves the clip to the playout queue. Call `play` to send it to `main_video` and `main_audio`, or turn on autoplay to play ready clips whenever the player is idle. You can generate another clip while one plays.

Call `reset` to stop playback, clear both queues and any retained clips you could continue from, clear the output tracks, and restore the session defaults.

## Commands

| Command | Description | Reply or event |
| - | - | - |
| `enqueue` | Queue a clip from a prompt, with optional references or continuation. | `clip_queued` |
| `play` | Consume and play a ready clip. | `clip_started` |
| `pop` | Remove a clip from either queue. | `clip_popped` |
| `move` | Reposition a queued clip. | `clip_moved` |
| `stop` | Cut the playing clip. | `clip_stopped` |
| `get_queue` | Read generation and playout queues. | `queue_update` |
| `get_state` | Read defaults, capacities, progress, and valid commands. | `state_update` |
| `set_clip_seconds` | Change the default duration for later enqueues. | `clip_length_accepted` |
| `set_seed` | Change the advancing default seed. | `seed_accepted` |
| `set_autoplay` | Play ready clips when idle. | `autoplay_accepted` |
| `set_flush_on_clip_end` | Choose black or a held last frame at boundaries. | `flush_accepted` |
| `set_canvas` | Choose an output aspect while the session is empty. | `canvas_accepted` |
| `reset` | Clear queues and playback and restore defaults. | `session_reset` |

### `enqueue`

Queue a clip from a prompt, with optional references or continuation. Every clip needs a nonempty prompt. Image references guide characters, scenes, and appearance. Separate `first_frame_image` and `last_frame_image` inputs pin the opening and closing frames; each needs a reference image in the same request. Audio references guide the soundtrack and require an image reference or a retained clip to continue. The model keeps each request's references with its clip. It replies with `clip_queued` when it accepts the request, then sends `clip_generated` or `clip_failed` after generation.

| Parameter | Type | Default | Description |
| - | - | - | - |
| `prompt` | string | `""` | Required, nonempty text describing video and sound. Refused if it exceeds the model's text budget; never truncated. |
| `metadata` | string | `""` | Client string, up to 2,000 characters, echoed in its `ClipInfo`. |
| `reference_image` | upload reference or null | `null` | One JPEG, PNG, or WebP image. Use this or `reference_images`. |
| `reference_images` | upload reference array or null | `null` | Up to nine ordered images; use no more than three for real-time generation. Name them `<Picture 1>`, `<Picture 2>`, and so on in the prompt. |
| `first_frame_image` | upload reference or null | `null` | Image pinned to the clip's first frame. Needs at least one reference image; refused with `continue_from_clip_id`. |
| `last_frame_image` | upload reference or null | `null` | Image pinned to the clip's last frame. Needs at least one reference image; with continuation, requires `continue_mode: "frame"`. |
| `reference_audio` | upload reference or null | `null` | One audio clip, 2 to 15 seconds long. Use this or `reference_audios`. |
| `reference_audios` | upload reference array or null | `null` | Up to three ordered audio clips, each 2 to 15 seconds long; at most two with a continuation. |
| `continue_from_clip_id` | string | `""` | UUID of an already generated clip retained in this session and at this canvas. An unknown or stale ID is refused. |
| `continue_mode` | `"latent"` or `"frame"`, or null | `null` | With `continue_from_clip_id`, null defaults to `latent`, carrying motion, camera, and audio. `frame` pins the source clip's final frame as this clip's first and uses its soundtrack as audio guidance. Refused without a continuation ID. |
| `seed` | nonnegative integer or null | `null` | Null uses and advances the session seed. An explicit seed leaves it unchanged. |
| `seconds` | number or null | `null` | One of 5.875, 8, 10.125, 12.25, or 14.375. Null uses the session default. Other values are refused, not rounded. |
| `position` | nonnegative integer or null | `null` | Position in the generation queue; 0 is next, and omitted or past the end appends. The running build is unaffected. |

<Warning>
  Do not send `reference_image` with `reference_images`, or `reference_audio` with `reference_audios` in the same request.
</Warning>

A continued clip's soundtrack uses one of the three audio labels. Give every uploaded audio clip a role in the prompt, such as a speaker's voice, background sound, or music; without that sentence the model has no instruction for how to use it. Keyframes are separate from ordered reference images and have no `<Picture N>` label.

The model checks file types, counts, prompt budget, and queue capacity before accepting a request. It sends `command_error` if a check fails. Each image must be JPEG, PNG, or WebP, no larger than 25 MiB and no more than 25 million pixels, with an aspect ratio between 1:4 and 4:1. Audio can be WAV, MP3, AAC/M4A, OGG/Opus, FLAC, or WebM. Each file can be up to 25 MiB and 2 to 15 seconds long, in mono or stereo.

The model checks the prompt and image references against the deployment's text and presentation budgets. Handle `command_error` if either limit is exceeded; the model refuses an over-budget request rather than truncating it.

With the base JavaScript SDK 3.1.0 or later, pass `FileRef` values directly, including inside `reference_images` and `reference_audios` arrays. The SDK serializes them for you.

The Sandbox converts browser-readable images, including HEIC, before upload and keeps only the first 15 seconds of longer audio. Direct API callers must use an accepted format and keep audio between 2 and 15 seconds.

### Keyframes and continuation

`reference_image` and `reference_images` guide appearance. To pin a frame, upload a separate image as `first_frame_image` or `last_frame_image`. Every request with a keyframe must also include at least one ordinary reference image. Keyframe images are not numbered `<Picture N>` references.

For an independent clip, you can set either or both keyframes. With `continue_from_clip_id`, the default `latent` mode carries motion, camera, and audio and accepts no keyframes. Set `continue_mode: "frame"` to use the earlier clip's final frame as the new first frame. Do not pass `first_frame_image` with a continuation. A `last_frame_image` with continuation requires `frame` mode and a reference image.

After the SDK reaches `ready`, upload the files and enqueue an independent clip with a pinned start and end:

```typescript theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}}
const scene = await reactor.uploadFile(sceneFile);
const opening = await reactor.uploadFile(firstFrameFile);
const ending = await reactor.uploadFile(lastFrameFile);

await reactor.sendCommand("enqueue", {
  prompt: "<Picture 1> is a ceramic artist in a sunlit studio. She lifts a blue bowl as the wheel hums. No music.",
  reference_image: scene,
  first_frame_image: opening,
  last_frame_image: ending,
  seconds: 8,
});
```

Wait for that clip's `clip_generated` message and save `message.data.clip.clip_id` as `generatedClipId`. To begin the next clip on its final frame, pass that ID with `frame` mode and an ordinary reference image:

```typescript theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}}
await reactor.sendCommand("enqueue", {
  prompt: "Continue with the same artist. She sets the bowl on a shelf; the wheel slows and the room stays quiet.",
  reference_image: scene,
  continue_from_clip_id: generatedClipId,
  continue_mode: "frame",
  seconds: 8,
});
```

`clip_queued` confirms acceptance, `clip_generated` reports readiness, and `clip_failed` reports a generation failure. Each message's `clip` includes the keyframe and continuation fields listed below. Invalid combinations produce `command_error` with a reason.

### `play`, `pop`, `move`, and `stop`

| Command | Parameters | Behavior |
| - | - | - |
| `play` | `clip_id?: string` | Blank selects the front ready clip. Refuses if playback is active or no ready clip matches. |
| `pop` | `clip_id: string` | Removes a clip from either queue; an in-flight result is discarded. Use `stop` for the playing clip. |
| `move` | `clip_id: string`, `position: integer` | Moves within the clip's current queue. Zero is front; past the end appends. Does not interrupt an active build. |
| `stop` | none | Cuts playback without clearing queues. Under autoplay, the next ready clip may start. |

<Warning>
  A clip cannot be replayed after it has played, been stopped, or been removed with `pop`.
</Warning>

### State and defaults

| Command | Parameters | Behavior |
| - | - | - |
| `get_queue` | none | Replies with both complete queues at any time. |
| `get_state` | none | Replies with a current state snapshot at any time. |
| `set_clip_seconds` | `seconds: 5.875 \| 8 \| 10.125 \| 12.25 \| 14.375` | Changes the default for future clips; queued clips keep their duration. |
| `set_seed` | `seed: nonnegative integer` | Changes the next automatic seed; explicit clip seeds do not advance it. |
| `set_autoplay` | `enabled: boolean` | Starts ready clips whenever idle. An empty queue waits; no prompts are generated automatically. |
| `set_flush_on_clip_end` | `enabled: boolean` | True flushes to black. False holds the last frame between clips and may leave silent gaps if generation falls behind. |
| `set_canvas` | `aspect: "16:9" \| "1:1" \| "9:16" \| "4:3"` | Requires empty queues and no playback. Dimensions are 1344 × 768, 768 × 768, 768 × 1344, or 1024 × 768. |
| `reset` | none | Clears queues, retained continuations and tracks, cuts playback, and restores defaults. |

The defaults are 14.375 seconds, seed 1000, aspect ratio 16:9, autoplay off, and flush on. Read `state_update` for the current session values.

## Messages

| Message | When it arrives | Payload |
| - | - | - |
| `state_update` | Connect and state changes; reply to `get_state`. | Full state snapshot. |
| `queue_update` | Connect and queue changes; reply to `get_queue`. | `generation`, `playout`, `history`. |
| `clip_queued` | Correlated `enqueue` reply. | `clip`. |
| `clip_generated` | A build enters the ready queue. | `clip`. |
| `clip_moved` | Correlated `move` reply. | `clip`, `queue`, `position`. |
| `clip_started` | Playback starts. | `clip`. |
| `clip_finished` | Video and audio have been sent. | `clip`, `seconds_sent`. |
| `clip_stopped` | Playback is cut. | `clip`, `seconds_sent`. |
| `clip_popped` | Correlated `pop` reply. | `clip`. |
| `clip_failed` | A build fails and leaves the queue. | `clip`, `reason`. |
| `clip_length_accepted` | Correlated `set_clip_seconds` reply. | `clip_seconds`, `frames`. |
| `seed_accepted` | Correlated `set_seed` reply. | `seed`. |
| `autoplay_accepted` | Correlated `set_autoplay` reply. | `enabled`. |
| `flush_accepted` | Correlated `set_flush_on_clip_end` reply. | `enabled`. |
| `canvas_accepted` | Correlated `set_canvas` reply. | `aspect`, `width`, `height`. |
| `session_reset` | Correlated `reset` reply. | `cleared_clips`, `was_playing`. |
| `command_error` | A command is refused. | `command`, `reason`. |

Command replies, including `clip_queued`, are correlated to the caller. The model broadcasts state, queue, and playback changes to all session clients. New clients receive `state_update` and `queue_update` when they connect.

### `ClipInfo`

Messages about a clip include this `ClipInfo` record:

| Field | Type | Meaning |
| - | - | - |
| `clip_id` | string | Assigned UUID. |
| `prompt` | string | Prompt as submitted. |
| `metadata` | string | Client string as submitted. |
| `frames` | integer | Output frame count. |
| `seconds` | number | Output duration. |
| `seed` | integer | Effective seed. |
| `ready` | boolean | True once built and ready to play. |
| `has_reference_image` | boolean | Whether an image reference was supplied. |
| `reference_image_count` | integer | Number of image references. |
| `has_reference_audio` | boolean | Whether uploaded audio was supplied. |
| `reference_audio_count` | integer | Number of uploaded audio references. |
| `has_first_frame`, `has_last_frame` | boolean | Whether this clip has a pinned opening or closing frame. |
| `continue_mode` | string or null | Continuation mode used for this clip, if any. |
| `prompt_prefix` | string | Text the model prepended to the prompt for frame continuation. |
| `build_seconds` | number or null | Build time excluding queue wait; null before build. |

### `state_update`

This message gives you the current session settings, playback status, queue counts, and available commands. It arrives when you connect or session state changes, and in reply to `get_state`. Use `queue_update` to see the clips in each queue.

| Field | Meaning |
| - | - |
| `clip_seconds` | Default duration for clips enqueued without `seconds`. |
| `clip_seconds_choices` | Allowed durations: 5.875, 8, 10.125, 12.25, and 14.375 seconds. |
| `seed` | Seed that will be assigned to the next clip enqueued without an explicit seed. |
| `autoplay` | Whether a ready clip starts automatically when playback is idle. |
| `flush_on_clip_end` | Whether the video output goes black at a clip boundary. If false, it holds the last frame. |
| `aspect`, `width`, `height` | Current output aspect ratio and video dimensions in pixels. |
| `playing`, `playing_clip_id` | Whether a clip is starting or playing, and its UUID. The ID is null when idle. |
| `generation_queued`, `generation_capacity` | Number of clips waiting or generating, and the maximum generation queue size. |
| `playout_queued`, `playout_capacity` | Number of ready clips waiting to play, and the maximum playout queue size. Generation pauses when the playout queue is full. |
| `clips_played`, `seconds_sent` | Clips finished or stopped, and total seconds sent on the output tracks in this session. |
| `valid_commands` | Commands available in the current queue and playback state. |

### `queue_update`

This message lists the clips waiting to generate or play. It arrives when you connect or either queue changes, and in reply to `get_queue`. Each listed clip is a `ClipInfo` record.

| Field | Meaning |
| - | - |
| `generation` | Clips waiting or generating, front first. |
| `playout` | Generated clips ready to play, front first. |
| `history` | Always an empty array. This field is not a playback log. |

With autoplay off, a new clip appears in `generation` after `enqueue`. When `clip_generated` arrives, it moves to `playout`. Calling `play` removes it from `playout`; its ID then appears in `state_update.playing_clip_id`. Save a generated clip's `clip_id` if you want to continue from it while the session retains it.

## Complete example

This base SDK example queues a clip from a prompt and plays it when ready. See the [tutorial](/model-api-reference/fast-h3-omni/tutorial) for uploads and continuation.

```typescript theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}}
import { Reactor } from "@reactor-team/js-sdk";

const reactor = new Reactor({ modelName: "reactor/fast-h3-omni" });

reactor.on("message", async (message) => {
  if (message.type === "clip_generated") {
    await reactor.sendCommand("play", { clip_id: message.data.clip.clip_id });
  }
});

reactor.on("statusChanged", async (status) => {
  if (status !== "ready") return;
  await reactor.sendCommand("enqueue", {
    prompt: "A blue bowl turns on a pottery wheel. The camera moves toward the artist's hands. The wheel hums and rain taps the window.",
    seconds: 5.875,
  });
});

await reactor.connect(await getToken()); // Your server issues the token.
```


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.