> ## Documentation Index > Fetch the complete documentation index at: https://docs.reactor.inc/llms.txt > Use this file to discover all available pages before exploring further. # Schema > The complete SANA-Streaming command and message schema, with end-to-end examples. This page documents the complete SANA-Streaming wire surface: every command you send, the messages the model emits back, and an end-to-end example. For the conceptual model and a quick start, see the [overview](/model-api-reference/sana-streaming/overview). The recommended way to drive SANA-Streaming from JavaScript or React is the typed [`@reactor-models/sana-streaming`](/sdk-reference/typed-model-sdk) SDK. It wraps every command below as a typed method (`setPrompt({ prompt })`), every message as a typed subscription (`useSanaStreamingState`), and each track as a typed view component, all checked at compile time. Under the hood it speaks the wire protocol this page documents, which is also what you send by name from the base [`Reactor`](/sdk-reference/reactor-class) client, for example in Python, which has no typed package: `send_command("set_prompt", { … })`. The examples below follow one convention: **React** snippets call hooks inside a component rendered under ``; **JavaScript** snippets call methods on a connected `SanaStreamingModel` named `sana`; **Python** snippets use the base `Reactor` client named `reactor`. ## Tracks | Direction | Name | Type | Format | Rate | | --------- | ------------ | ----- | ------------------------ | ----------------------------------- | | Inbound | `camera` | Video | WebRTC video track | Camera capture rate | | Outbound | `main_video` | Video | `(N, H, W, 3)` uint8 RGB | 24-frame chunks, one every \~1-1.5s | Output resolution is 1280 × 704. In **live** mode you publish your camera to the `camera` track; in **file** mode the source arrives by upload (`set_video`), not as a track. Commands that change generation take effect on chunk boundaries. Chunks hold 24 frames through the middle of a run; the first and last chunks of a run can be shorter. In the typed React SDK, render the output with `` and read incoming tracks with `useSanaStreamingTrack("main_video")`; both are pre-bound to these names. There is also a `` that acquires and publishes the webcam for you, but it gives no hook to set the camera's content hint, so publish the `camera` track yourself when you need that (see the warning below). Browsers shrink and grow the camera's resolution mid-stream to cope with bandwidth, and a resolution change mid-chunk **crashes the live session**. Set `track.contentHint = "detail"` on the camera track before you publish it; the browser then holds resolution steady and adapts the frame rate instead, which the model handles fine. ## Session lifecycle SANA-Streaming session states: IDLE, RUNNING, PAUSED, with set commands staging in IDLE, start moving to RUNNING, set_prompt applying mid-stream, pause and resume between RUNNING and PAUSED, and reset returning to IDLE Once the connection reaches **ready**, the session begins in `IDLE`. Stage a source there (a published camera in live mode, `set_video` in file mode) and, to steer the edit, a prompt; `start` transitions to `RUNNING`; `pause` moves to `PAUSED`; `resume` returns to `RUNNING`; `reset` clears the source, prompt, and progress and returns to `IDLE` from any state. While `RUNNING`, a new `set_prompt` lands at the next chunk boundary without stopping the stream, and `set_anchor_interval` turns on (or adjusts) re-grounding on the fly — anchoring is off by default. See [Sessions](/concepts/sessions#connection-lifecycle) for the separate connection-level lifecycle (`disconnected → connecting → waiting → ready`) the session passes through before reaching the states above. A prompt is **optional**: start with no prompt and the model streams a near-reconstruction of the source; set or change one at any time to steer the edit. The source is the only hard requirement to `start` (a published camera in live mode, an accepted clip in file mode). In **file** mode a run also ends on its own: every source frame gets transformed, then `generation_complete` reports the total chunk count and the session returns to `IDLE` with the clip, prompt, and seed still staged. Send `start` again to replay the clip from the top; `reset` is only needed to swap in a different clip or to clear the staging. On completion the `main_video` track freezes on the last transformed frame rather than going dark, so blank or overlay the stage yourself. ## Commands In the typed SDK each command is a method on `useSanaStreaming()` (React) or a `SanaStreamingModel` instance (JavaScript); from the base client you send it by name with `send_command()`. Below are all available commands: | Command | Description | | --------------------- | ------------------------------------------------------------ | | `set_mode` | Choose the input source: live camera or uploaded file | | `set_video` | Set an uploaded clip as the source (file mode only) | | `set_prompt` | Set or change the edit prompt, before or during generation | | `set_seed` | Set the noise seed | | `set_anchor_interval` | Re-ground generation every N chunks to limit drift (0 = off) | | `start` | Begin streaming edited video on `main_video` | | `pause` | Pause generation after the current chunk | | `resume` | Resume generation from a pause | | `reset` | Clear the source, prompt, and progress; return to idle | ### set\_mode Select where the source video comes from: `"live"` (a published `camera` track) or `"file"` (a clip set with `set_video`). Repeating the command is **safe**, even with the same mode, so always send `set_mode` then `start` as a pair; the flow works no matter what was set before. Mode is fixed once a run starts; `reset` to switch. **Parameters:** | Parameter | Type | Required | Description | | --------- | ------ | -------- | --------------------------------------- | | `mode` | string | No | `"live"` or `"file"` (default `"file"`) | **Example:** ```tsx React theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}} const { setMode } = useSanaStreaming(); await setMode({ mode: "live" }); ``` ```typescript JavaScript theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}} await sana.setMode({ mode: "live" }); ``` ```python Python theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}} await reactor.send_command("set_mode", {"mode": "live"}) ``` ### set\_video Set an uploaded clip as the source for **file** mode. Upload the file first with `uploadFile()` to get a [`FileRef`](/sdk-reference/types#fileref), then pass it here. The model stores the upload at once (frames decode during generation, not at upload) and replies with `video_accepted` plus a `state` snapshot with `has_video: true`. Gate your start control on that state, not on the upload finishing. Clips must be **at least 33 frames** long. **Parameters:** | Parameter | Type | Required | Description | | --------- | ------- | -------- | -------------------------------------- | | `video` | FileRef | Yes | The uploaded clip, from `uploadFile()` | **Example:** ```tsx React theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}} const { uploadFile, setVideo } = useSanaStreaming(); const ref = await uploadFile(file); // a video File or Blob await setVideo({ video: ref }); ``` ```typescript JavaScript theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}} const ref = await sana.uploadFile(file); // a video File or Blob await sana.setVideo({ video: ref }); ``` ```python Python theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}} ref = await reactor.upload_file("clip.mp4") await reactor.send_command("set_video", {"video": ref}) ``` A `decode failed` error from `set_video` is sometimes a false alarm on a valid clip. Send the same `setVideo({ video })` again (same `FileRef`, no re-upload); treat it as a real error only if it fails again once or twice. ### set\_prompt Set the edit prompt, before `start` or at any point mid-stream. The model applies the new prompt at the **next chunk boundary**, about one chunk (\~1-1.5s) after you send it, with no re-render and no break in the stream. Acknowledged by `prompt_accepted`. A prompt is **optional**: with no prompt the model streams a near-reconstruction of the source. Set or change one to steer the edit. On an encode failure the model keeps the previous prompt and emits `command_error`. See the [prompt guide](/model-api-reference/sana-streaming/prompt-guide) for how to write edit prompts. **Parameters:** | Parameter | Type | Required | Description | | --------- | ------ | -------- | ----------------- | | `prompt` | string | Yes | The edit to apply | **Example:** ```tsx React theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}} const { setPrompt } = useSanaStreaming(); await setPrompt({ prompt: "Replace the background with a dimly lit speakeasy lounge, leaving the subject unchanged.", }); ``` ```typescript JavaScript theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}} await sana.setPrompt({ prompt: "Replace the background with a dimly lit speakeasy lounge, leaving the subject unchanged.", }); ``` ```python Python theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}} await reactor.send_command("set_prompt", { "prompt": "Replace the background with a dimly lit speakeasy lounge, leaving the subject unchanged.", }) ``` ### set\_seed Set the noise seed for sampling. The model reads it at `start`, so set it before the first `start` for reproducible output; to re-seed a staged run, set it then `reset`. **Parameters:** | Parameter | Type | Required | Description | | --------- | ------- | -------- | ------------------------------------------------------------- | | `seed` | integer | No | Non-negative noise seed. Same seed, same output (default `0`) | ```tsx React theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}} const { setSeed } = useSanaStreaming(); await setSeed({ seed: 42 }); ``` ```typescript JavaScript theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}} await sana.setSeed({ seed: 42 }); ``` ```python Python theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}} await reactor.send_command("set_seed", {"seed": 42}) ``` ### set\_anchor\_interval Re-ground generation on the source every `chunks` chunks. Each anchor resets the sampler's KV/RoPE state and VAE caches so the next chunk is generated like a first chunk, which bounds the drift that builds up as the model conditions on its own generated history over a long run. Valid at any time, including mid-stream (the value is read at chunk boundaries). Each anchor emits an `anchored` message, and the chunk right after it can show a brief visible refresh. Pass `0` to disable anchoring. **Anchoring is off by default.** A new session starts with `anchor_interval` at `0` (no re-grounding) — the `state` snapshot you get on connect confirms it. Anchoring only runs once you call `set_anchor_interval` with a non-zero `chunks`. The `20` below is the value used when you call the command *without* a `chunks` argument; it is not the session default. **Parameters:** | Parameter | Type | Required | Description | | --------- | ------- | -------- | ------------------------------------------------------------------------------------- | | `chunks` | integer | No | Chunks between anchors; `0` disables. Off by default; omitting the argument uses `20` | ```tsx React theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}} const { setAnchorInterval } = useSanaStreaming(); await setAnchorInterval({ chunks: 20 }); ``` ```typescript JavaScript theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}} await sana.setAnchorInterval({ chunks: 20 }); ``` ```python Python theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}} await reactor.send_command("set_anchor_interval", {"chunks": 20}) ``` ### start Begin generating edited video on `main_video`. A prompt is **optional** in both modes. The only hard precondition is in **file** mode, which requires an accepted clip (`set_video`); calling `start` there with no source is rejected with a `command_error` (`"No source video set"`). **Live** mode has no such check — `start` always proceeds and transforms whatever camera frames have arrived, so an as-yet-unpublished or empty `camera` track is not an error. Always send `set_mode` first; the pair keeps the start flow self-contained. Calling `start` while a run is already generating has no effect. After a file-mode run completes, `start` works without a `reset` and replays the retained clip from the beginning with the same prompt and seed. Emits `generation_started` plus a `state` snapshot, or `command_error`. ### pause / resume `pause` halts generation after the current chunk finishes (emitting `generation_paused`); `resume` continues (emitting `generation_resumed`). `pause` is only valid while generating; `resume` only while paused. ### reset Abort the current run and clear the model's source video, prompt, and progress, returning to the idle waiting state. Emits `generation_reset`. After a reset, set a source and prompt again before the next `start`. The model latches its source when `start` fires, so `reset` is also how you swap clips between file-mode runs. ```tsx React theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}} const { start, pause, resume, reset } = useSanaStreaming(); await start(); await pause(); await resume(); await reset(); ``` ```typescript JavaScript theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}} await sana.start(); await sana.pause(); await sana.resume(); await sana.reset(); ``` ```python Python theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}} await reactor.send_command("start", {}) await reactor.send_command("pause", {}) await reactor.send_command("resume", {}) await reactor.send_command("reset", {}) ``` ## Messages The model emits the following messages. In React, subscribe to each with its typed hook (shown in the table); in JavaScript, with `on` on a `SanaStreamingModel` instance; in Python, with `@reactor.on_message("")`. React also offers `useSanaStreamingMessage` for one handler over the whole union. The typed hooks hand you a flat, fully-typed object (`msg.running`, `msg.reason`); the base client delivers the same data as JSON `{ "type": "", "data": { … } }`. | Message | React hook | When | Payload | | --------------------- | ------------------------------------ | -------------------------------------------------------------- | ---------------------------------------------------------------------- | | `state` | `useSanaStreamingState` | On connect, after every command, after each chunk | Full session snapshot (see below) | | `prompt_accepted` | `useSanaStreamingPromptAccepted` | A `set_prompt` was accepted and encoded | `{ prompt: string }` | | `video_accepted` | `useSanaStreamingVideoAccepted` | A `set_video` source was accepted and probed | `{ width: int, height: int, num_frames: int, num_latent_frames: int }` | | `conditions_ready` | `useSanaStreamingConditionsReady` | After `set_prompt` / `set_video`; whether `start` will succeed | `{ has_video: bool, has_prompt: bool }` | | `generation_started` | `useSanaStreamingGenerationStarted` | `start` succeeded, frames begin | `{ prompt: string, chunk_num: int, frame_num: int }` | | `chunk_complete` | `useSanaStreamingChunkComplete` | Once per completed chunk of `main_video` | `{ chunk_index: int, frames_emitted: int, active_prompt: string }` | | `anchored` | `useSanaStreamingAnchored` | Generation re-grounded on the source | `{ chunk_index: int }` | | `generation_paused` | `useSanaStreamingGenerationPaused` | `pause` took effect | `{ chunk_index: int }` | | `generation_resumed` | `useSanaStreamingGenerationResumed` | `resume` took effect | `{ chunk_index: int }` | | `generation_complete` | `useSanaStreamingGenerationComplete` | A file-mode source played through to its end | `{ total_chunks: int }` | | `generation_reset` | `useSanaStreamingGenerationReset` | In response to `reset` | `{ reason: string }` | | `command_error` | `useSanaStreamingCommandError` | A command was rejected (bad precondition) | `{ command: string, reason: string }` | `video_accepted` reports the decoded source's pre-crop `width` and `height`; `num_frames` and `num_latent_frames` arrive as `0` because the clip is decoded and VAE-encoded lazily during generation, not at upload. `conditions_ready` lets you check that `start` will succeed (only a video is required; a prompt is optional). ### `state` payload `state` is the single source of truth for the session's observable state. Subscribe once and gate your UI on it (start buttons, mode toggles, transport controls) rather than tracking individual acknowledgements yourself. | Field | Type | Meaning | | ----------------- | ---------------- | ------------------------------------------------------------------------------------ | | `running` | `bool` | The chunk loop is actively producing frames | | `started` | `bool` | True once `start` is accepted; false after `reset` or when a file-mode run completes | | `paused` | `bool` | True while generation is paused | | `current_chunk` | `int` | Zero-based index of the last completed chunk; `0` before the first | | `current_prompt` | `string \| null` | The edit prompt now driving generation, or `null` if none set | | `has_prompt` | `bool` | Whether an edit prompt has been set and encoded | | `has_video` | `bool` | Whether a source clip has been accepted | | `anchor_interval` | `int` | Chunks between re-grounds (`0` = never, the default); set via `set_anchor_interval` | | `seed` | `int` | Current seed value | `current_prompt` is typed `unknown` on the wire (the field is free-form), but the model only ever sends a string or `null`; narrow it on your side. One timing quirk: right after a restart, `state` still reports the previous run's final `current_chunk` until the new run's first `chunk_complete` lands (indices restart at 0). Don't drive progress UI from `current_chunk` in that window. ## Complete example Live mode in React, file mode from Python: ```tsx React theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}} "use client"; import { SanaStreamingProvider, useSanaStreaming, useSanaStreamingChunkComplete, SanaStreamingMainVideoView, } from "@reactor-models/sana-streaming"; import { useEffect } from "react"; // Token minted on your server; see Authentication. async function getJwt() { const r = await fetch("/api/reactor/token"); const { jwt } = await r.json(); return jwt; } function Editor() { const { status, publish, setPrompt, setMode, start } = useSanaStreaming(); useSanaStreamingChunkComplete((msg) => console.log("chunk", msg.chunk_index, msg.active_prompt), ); useEffect(() => { if (status !== "ready") return; void (async () => { // Publish the camera as the live source. Pin the resolution first (see Tracks above). const cam = await navigator.mediaDevices.getUserMedia({ video: true }); const track = cam.getVideoTracks()[0]; track.contentHint = "detail"; await publish("camera", track); // Describe the edit, then start. await setPrompt({ prompt: "Van Gogh oil painting, swirling brushstrokes, vivid colors" }); await setMode({ mode: "live" }); await start(); })(); }, [status]); return ( ); } export default function App() { return ( ); } ``` ```python Python theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}} import asyncio, os from reactor_sdk import Reactor async def main(): reactor = Reactor(model_name="reactor/sana-streaming", api_key=os.environ["REACTOR_API_KEY"]) @reactor.on_message("chunk_complete") def _(msg): print(f"chunk {msg['chunk_index']}: {msg['active_prompt']}") await reactor.connect() # Upload a clip as the file-mode source, describe the edit, then start. ref = await reactor.upload_file("clip.mp4") # at least 33 frames await reactor.send_command("set_video", {"video": ref}) await reactor.send_command("set_prompt", { "prompt": "Apply a loose watercolor wash, bleeding edges, soft pastel palette", }) await reactor.send_command("set_mode", {"mode": "file"}) await reactor.send_command("start", {}) await asyncio.Event().wait() asyncio.run(main()) ``` Prefer the plain-JavaScript client over React? Construct `new SanaStreamingModel()`, `await sana.connect(jwt)`, then call the same typed methods (`sana.setPrompt({ … })`, `sana.start()`) and subscribe with `sana.onState(…)` / `sana.onChunkComplete(…)`. The model name and tracks are baked into the class.