Skip to main content
This page documents the complete FastH3 wire surface: the two media tracks the model produces, the session lifecycle, every command you can send, and every message the model emits back. For what FastH3 is and a quick start, see the overview.

Tracks

There are no inbound tracks: the model reads no camera and no microphone. The video track keeps one size for the length of a queue — set_canvas chooses it and is only accepted while the queue is empty and nothing plays, since queued clips are already built at the size in force. The audio track is generated jointly with the video on every clip, so the two never drift.

Session lifecycle

Once the connection reaches ready, the session begins idle against a black output. Adding clips doesn’t start any playback: a built clip only reaches the tracks when a play (or, with set_autoplay on, the playout queue’s front-most ready clip) asks for it. By default the stream flushes to black the moment a clip ends or is stopped and holds there until the next play. Two things change that boundary behavior, and they’re separate:
  • set_flush_on_clip_end with enabled: false makes the stream hold the last frame across a normal end or stop instead of going black.
  • A clip enqueued with continue_from_clip_id chains off its source clip — when autoplay moves from the source into it, the hand-off is seamless regardless of the flush setting: the model builds the sequel to open on the source’s last frame, so there is no boundary to see.
Single session, shared state. Several clients may attach to one session and they all see the same queues and the same stream: an enqueue or a stop from any client affects everyone, and every client receives every state_update and queue_update. Generation is gated on having an audience — with nobody connected no new build starts, though a build already underway finishes into the queue. state_update.valid_commands names exactly what the session would accept at that moment, so a frontend enables and greys out controls from the snapshot instead of re-deriving these rules. The diagram above shortens the valid lists to illustrative commands for that reason — the snapshot, not this page, is the authoritative listing at runtime; see the state_update payload table for what it carries.

Commands

Send commands with reactor.sendCommand() on the base SDK. Commands that have a reply resolve their awaited call with it (enqueue → clip_queued, move → clip_moved, and so on); commands without one take effect silently, the change visible in the next broadcast state_update / queue_update. A refused command has no effect and fails its own call with a stable code and a reason; see Refusals for every code and how to read one.

enqueue

Enter a prompt into the generation queue. Each request snapshots the session’s conditions (set_clip_seconds, set_seed, set_canvas) as they stand, receives a UUID, and joins the queue — at the back, or at position if given (0 = next build). The reply, clip_queued, carries the full ClipInfo. The generation queue is bounded (default 20); full, the command is refused with queue_full. A plain enqueue opens a clip from text alone. Two optional fields open it from a picture instead — at most one of them may be set on any enqueue:
  • starting_frame opens the clip from an uploaded still (image-to-video). The frame is fitted to the session canvas and animates forward.
  • continue_from_clip_id opens the clip from the last frame of another clip, chaining the two into a continuing scene. The source can sit in either queue or in queue_update.history, including a clip that hasn’t built yet — the sequel then waits for its anchor and builds out of order relative to strict queue position, without blocking the builds behind it. A chain keeps the exposure of the clip it opened from, so a long chain does not drift brighter or darker link by link.
Two more optional fields close the clip on a picture instead of leaving the ending to the prompt — independent of how the clip opens, so either can pair with either opener, and at most one of them may be set on any enqueue:
  • ending_frame closes the clip on an uploaded still, fitted to the session canvas the same way as starting_frame. Pair it with starting_frame or continue_from_clip_id for first-and-last generation, or send it alone to fix only the ending.
  • ending_from_clip_id closes the clip on another clip’s clean retained last frame — the same source rules as continue_from_clip_id (either queue or queue_update.history, including a clip that hasn’t built yet). It may name the same clip as continue_from_clip_id to create a hold that opens and closes on one generated frame.
Parameters: starting_frame is a FileRef — the object uploadFile() returns, with upload_id, name, mime_type, size, and optional width / height. Upload first, then embed the reference:
Fails with: empty_prompt when the prompt is empty; prompt_too_long when it is over the 1024-token budget (the reason gives the token count, and the model trims nothing for you); queue_full when the generation queue is full; invalid_request when both opener fields come in the same request (“a clip opens from one picture”) or both ending fields do (“a clip closes on one picture”); unknown_clip when continue_from_clip_id or ending_from_clip_id names a clip that is in neither queue nor queue_update.history (the id doesn’t exist anywhere the model can reference); or invalid_image when an uploaded frame is not a decodable image. metadata is for the frontend, not the model. The model stores it and echoes it back on every message that references the clip; it never parses it. Carry whatever your application needs to track — which request produced the clip, who asked for it, text to show while it plays. JSON fits if you want structure.

move

Reposition a clip within whichever queue holds it. position 0 is the front; a value past the back is clamped. The reply, clip_moved, names the queue and the landing position. Fails with unknown_clip when the clip_id is unknown.

play

Stream a built clip. With no clip_id, play takes the playout queue’s front; a clip_id plays that specific clip. Playing consumes the entry — the clip leaves the queue when it starts. clip_started fires as the first frames reach the tracks. Fails with already_playing if another clip is playing, unknown_clip if the id is unknown, clip_not_ready if the named clip is still generating, or no_ready_clip if no clip_id is given and the playout queue is empty. play has no reply, so it resolves the same way whether it starts a clip or not: read a refusal from the error event.

pop

Remove a clip from whichever queue holds it, freeing its slot; a build in flight for it is discarded when it finishes (the GPUs cannot abandon mid-build). Replies clip_popped. Fails with unknown_clip when the id is unknown, or with clip_referenced when a queued (unbuilt) clip is named by another queued clip’s continue_from_clip_id or ending_from_clip_id. Popping the source would leave the dependent scene no frame to open from or close on, so the model refuses until you’ve popped the dependents first (back-to-front for a bulk clear). Built clips don’t hit this: they persist in queue_update.history after being popped and can still anchor continuations.

stop

Cut the playing clip. Both queues are untouched. With autoplay on, stop acts as a skip — the next ready clip starts on its own. Emits clip_stopped. By default the picture goes black within a fraction of a second, and the rest of the clip is discarded. With set_flush_on_clip_end off, the transport first drains what it already holds, up to about two seconds. The picture then freezes on the last frame. Fails with not_playing when nothing is playing.

get_queue

Replies with both queues plus the retained build history — the same payload as queue_update. Front first, each entry a full ClipInfo.

set_autoplay

When enabled is true, the playout queue’s front clip starts on its own whenever nothing is playing; stop then acts as a skip. When false (the default), playback waits for an explicit play. Replies autoplay_accepted.

set_clip_seconds

Default clip length for enqueues that carry no seconds. The value snaps to what the model can produce; the reply, clip_length_accepted, carries the effective value — read it back before assuming a duration.

set_seed

Default seed for enqueues that carry none. Each such enqueue advances the default by one, so seeds are per-clip, not per-session. enqueue with an explicit seed leaves the default untouched. Replies seed_accepted. Re-enqueueing the same prompt with the same seed, length, and canvas reproduces the same clip (approximately — compiled kernels can reorder floating-point operations).

set_canvas

Video size for the session, by aspect (16:9, 1:1, 9:16, 4:3). Only accepted while the queue is empty and nothing is playing, since queued clips were built at the size in force. Replies canvas_accepted carrying the exact pixel width and height. Fails with session_not_empty otherwise, or with invalid_aspect on an unsupported aspect.

set_flush_on_clip_end

Whether the stream cuts to black the moment a clip ends, is stopped, or is followed by a non-continuing clip — enabled: true is the default. With enabled: false, those transitions hold the clip’s last frame instead. This toggle has no say in a genuine continuation: when autoplay moves from a clip into one enqueued with continue_from_clip_id pointing at it, the sequel opens on the source’s last frame by construction and the boundary has no cut either way. set_flush_on_clip_end is what you set for every boundary that isn’t a hand-off. Replies flush_accepted.

reset

Drop the generation queue, the playout queue, and the retained build history; cut any playing clip; and restore every session default (canvas, seed, clip length, autoplay, flush). Replies session_reset, carrying how many clips were dropped and whether a clip was playing.

get_state

Replies with the full state_update snapshot.

Refusals

A refused command changes nothing and fails its own call. Only the client that sent it is told. The failure carries a stable code to branch on and a reason written for your user: Any command can also fail with invalid_command when its payload breaks the schema (a negative seed, say), and a command carrying an upload with unresolved_upload when the file cannot be fetched. On the JavaScript SDK the awaited call resolves undefined, and the error event fires first with operation: "sendCommand" and the code. Every command except play and stop declares a reply, so for those undefined means refused. play and stop resolve undefined on success too, so only the error event tells. On the Python SDK, send_command() raises a ReactorError with the code.
FastH3 also broadcasts a command_error message ({ command, reason }) to every connected client when it refuses a command. Read the refusal off the call instead: the broadcast reaches clients that did not send the command, and a future major release removes it. Releases before 1.9.0 refuse only with that broadcast, and their call resolves as if it succeeded. Treat a reply-declaring command that resolves undefined (None in Python) with no error as refused, as the example’s fallback reason does. See Replies and refusals for the SDK side.

Messages

Every message arrives as JSON { "type": "<name>", "data": { … } }. The Reaches column says whether a message is delivered to every connected client (everyone) or only to the client whose command produced it (the caller). When a build fails, every queued clip that references it fails too. Each of those clips gets its own clip_failed naming the source it lost. A refused command fails its own call; see Refusals. FastH3 also broadcasts command_error ({ command: string, reason: string }) to every connected client when it refuses a command, until a future major release removes it.

queue_update payload

Carries everything a client needs to render the queue: generation (clips waiting to build, front first), playout (built clips waiting to play, front first), and history (built clips no longer queued — played, stopped, or popped — whose last frame is still retained, oldest first). history entries are not playable again (play refuses them), but each is a legal continue_from_clip_id source. The oldest history entries are evicted as new builds finish.

state_update payload

A full snapshot of everything observable except the queue’s contents, so a client can render UI from this alone. Used by a scheduling client to steer; also carries every deployment-published live bound a writer must not hardcode.

ClipInfo payload

Every message that references a clip embeds the full ClipInfo, so a client never has to join a UUID against an earlier message. Example handler (broadcasts only):

Complete example

Enqueue a base scene, then chain a continuation off it so the two play as one shot. Autoplay drives both; the continue_from_clip_id hand-off is seamless.
The continuation prompt re-describes the setting from scratch (“the same foothills… rain”) and opens on a described hard cut rather than inheriting the take — continuity carries the frame, not the description or the camera; the discipline is the whole prompt guide.