This page documents the complete SANA-Streaming wire surface: every command you send with
reactor.sendCommand(), the messages the model emits back, and an end-to-end example. For the
conceptual model and a quick start, see the
overview.
SANA-Streaming has no typed model SDK yet, so there are no named
command methods or React hooks for it. Drive it from the base
Reactor class (JavaScript) or Reactor client (Python) instead:
send commands by name with sendCommand(), and read the messages coming back as plain JSON in the
shapes this page documents.
Tracks
| Direction | Name | Type | Format | Rate |
|---|
| Inbound | camera | Video | WebRTC video track | Camera capture rate |
| Outbound | main_video | Video | (N, H, W, 3) uint8 RGB | 24-frame chunks, one every ~1-1.5s |
camera track; in
file mode the source arrives by upload (set_video), not as a track. Commands that change
generation take effect on chunk boundaries. Chunks hold 24 frames through the middle of a run; the
first and last chunks of a run can be shorter.
Browsers shrink and grow the camera’s resolution mid-stream to cope with bandwidth, and a
resolution change mid-chunk crashes the live session. Set track.contentHint = "detail" on
the camera track before you publish it; the browser then holds resolution steady and adapts the
frame rate instead, which the model handles fine.
Session lifecycle
Once the connection reaches ready, the session begins in IDLE. Stage a source there (a
published camera in live mode, set_video in file mode) and a prompt; start transitions to
RUNNING; pause moves to PAUSED; resume returns to RUNNING; reset clears the source,
prompt, and progress and returns to IDLE from any state. While RUNNING, a new set_prompt lands
at the next chunk boundary without stopping the stream. See
Sessions for the separate connection-level lifecycle
(disconnected → connecting → waiting → ready) the session passes through before reaching the
states above.
In file mode a run also ends on its own: every source frame gets transformed, then
generation_complete reports the total chunk count and the session returns to IDLE with the clip,
prompt, and seed still staged. Send start again to replay the clip from the top; reset is only
needed to swap in a different clip or to clear the staging. On completion the main_video track
freezes on the last transformed frame rather than going dark, so blank or overlay the stage
yourself.
Commands
Send commands to the model using reactor.sendCommand(). Below are all available commands:
| Command | Description |
|---|
set_mode | Choose the input source: live camera or uploaded file |
set_video | Set an uploaded clip as the source (file mode only) |
set_prompt | Set or change the edit prompt, before or during generation |
set_seed | Set the noise seed |
start | Begin streaming edited video on main_video |
pause | Pause generation after the current chunk |
resume | Resume generation from a pause |
reset | Clear the source, prompt, and progress; return to idle |
set_mode
Select where the source video comes from: "live" (a published camera track) or "file" (a clip
set with set_video). Repeating the command is safe, even with the same mode, so always send
set_mode then start as a pair; the flow works no matter what was set before.Parameters:| Parameter | Type | Required | Description |
|---|
mode | string | Yes | "live" or "file" |
await reactor.sendCommand("set_mode", { mode: "live" });
set_video
Set an uploaded clip as the source for file mode. Upload the file first with uploadFile() to
get a FileRef, then pass it here. The model stores the upload at
once (frames decode during generation, not at upload) and replies with video_accepted plus a
state snapshot with has_video: true. Gate your start control on that state, not on the upload
finishing.Clips must be at least 33 frames long.Parameters:| Parameter | Type | Required | Description |
|---|
video | FileRef | Yes | The uploaded clip, from uploadFile() |
const ref = await reactor.uploadFile(file); // a video File or Blob
await reactor.sendCommand("set_video", { video: ref });
A decode failed error from set_video is sometimes a false alarm on a valid clip. Send the same
set_video again (same FileRef, no re-upload); treat it as a real error only if it fails again
once or twice.
set_prompt
Set the edit prompt, before start or at any point mid-stream. The model applies the new prompt at
the next chunk boundary, about one chunk (~1-1.5s) after you send it, with no re-render and no
break in the stream. Acknowledged by prompt_accepted.See the prompt guide for how to write edit
prompts.Parameters:| Parameter | Type | Required | Description |
|---|
prompt | string | Yes | The edit to apply |
await reactor.sendCommand("set_prompt", {
prompt:
"Replace the background with a dimly lit speakeasy lounge, leaving the subject unchanged.",
});
start
Begin generating edited video on main_video. Requires a source for the active mode (a published
camera track in live mode, an accepted clip in file mode). Always send set_mode first; the pair
keeps the start flow self-contained. After a file-mode run completes, start works without a
reset and replays the retained clip from the beginning with the same prompt and seed.pause / resume
pause halts generation after the current chunk finishes; resume continues. pause is only valid
while generating; resume only while paused.reset
Abort the current run and clear the model’s source video, prompt, and progress, returning to the
idle waiting state. Emits generation_reset. After a reset, set a source and prompt again before
the next start. The model latches its source when start fires, so reset is also how you swap
clips between file-mode runs.await reactor.sendCommand("start", {});
await reactor.sendCommand("pause", {});
await reactor.sendCommand("resume", {});
await reactor.sendCommand("reset", {});
set_seed
Set the noise seed for sampling. Useful before the first start for reproducible output.Parameters:| Parameter | Type | Required | Description |
|---|
seed | integer | Yes | Noise seed. Same seed, same output |
await reactor.sendCommand("set_seed", { seed: 42 });
Messages
The model emits the following messages. Subscribe with reactor.on("message", ...) in JavaScript,
useReactorMessage in React, or @reactor.on_message in Python. Every message is delivered as JSON
{ "type": "<name>", "data": { … } }.
| Message | When | Payload |
|---|
prompt_accepted | A set_prompt was accepted | { prompt: string } |
video_accepted | A set_video upload probe succeeded | { width: int, height: int, num_frames: int, num_latent_frames: int } |
generation_started | start succeeded, frames begin | { prompt: string, chunk_num: int, frame_num: int } |
chunk_complete | Once per completed chunk of main_video | { chunk_index: int, frames_emitted: int, active_prompt: string } |
generation_complete | A file-mode source played through to its end | { total_chunks: int } |
generation_reset | In response to reset | { reason: string } |
command_error | A command was rejected (bad precondition) | { command: string, reason: string } |
state | On connect, after every command, after each chunk | Full session snapshot (see below) |
state payload
state is the single source of truth for the session’s observable state. Subscribe once and gate
your UI on it (start buttons, mode toggles, transport controls) rather than tracking individual
acknowledgements yourself.
| Field | Type | Meaning |
|---|
running | bool | Frames are actively streaming |
started | bool | True once start is accepted; false after reset or when a file-mode run completes |
paused | bool | True while generation is paused |
current_chunk | int | Chunks completed since start |
current_prompt | string | null | The edit prompt now driving generation, or null if none set |
has_prompt | bool | Whether an edit prompt has been set |
has_video | bool | Whether a file-mode source clip has been accepted |
num_source_frames | int | Frame count of the accepted source clip (file mode) |
seed | int | Current seed value |
state still reports the previous run’s final
current_chunk until the new run’s first chunk_complete lands (indices restart at 0). Don’t drive
progress UI from current_chunk in that window.
Complete example
Live mode in the browser, file mode from Python:
import { Reactor } from "@reactor-team/js-sdk";
const video = document.querySelector("video")!;
const reactor = new Reactor({ modelName: "sana-streaming" });
// Render the edited stream as frames arrive.
reactor.on("trackReceived", (name, track, stream) => {
if (name !== "main_video") return;
video.srcObject = stream;
void video.play();
});
reactor.on("message", (msg) => {
if (msg.type === "command_error") console.error(msg.data.command, msg.data.reason);
});
reactor.on("statusChanged", async (status) => {
if (status !== "ready") return;
// Publish the camera as the live source. Pin the resolution first (see Tracks above).
const cam = await navigator.mediaDevices.getUserMedia({ video: true });
const track = cam.getVideoTracks()[0];
track.contentHint = "detail";
await reactor.publishTrack("camera", track);
// Describe the edit, then start.
await reactor.sendCommand("set_prompt", {
prompt: "Van Gogh oil painting, swirling brushstrokes, vivid colors",
});
await reactor.sendCommand("set_mode", { mode: "live" });
await reactor.sendCommand("start", {});
});
const jwt = await getToken(); // token minted on your server
await reactor.connect(jwt);
