Pika H3 Flex schema

This page is the protocol reference for Pika H3 Flex: its tracks, commands, refusals, and messages. The model streams video on main_video and audio on main_audio. Send commands with reactor.sendCommand() once the session reaches ready.

Tracks

Track	Direction	Type
`main_video`	Output	Video
`main_audio`	Output	Audio, synchronized with `main_video`

There are no input tracks. The model takes everything it needs from commands. Both tracks stream only while a clip plays. Between clips, main_video shows black and main_audio is silent. Turn off set_flush_on_clip_end to hold the last frame instead of black. Each session starts at 16:9 and the 768p resolution, which is 1344 × 768 pixels. set_resolution switches between 768p and 480p (832 × 480 at 16:9), and set_canvas changes the aspect ratio. Call them while both queues are empty and nothing plays, for example before you queue the first clip. At any other time the size stays fixed. state_update, canvas_accepted, and resolution_accepted report the size in pixels.

Session lifecycle

A Pika H3 Flex session has two queues:

The generation queue holds clips that wait to generate.
The playout queue holds clips that are ready to play.

When a clip finishes generating, it moves to the back of the playout queue. Playing a clip takes it out of the playout queue and onto the output tracks. Nothing plays until you call play, unless autoplay is on. When you connect, the model sends you a state_update and a queue_update. A new session has both queues empty, autoplay off, and nothing playing. state_update.valid_commands lists the commands the session accepts right now. Use it to turn controls on and off in your UI. Generation keeps going while a clip plays, so you can queue the next clip during playback. It runs only while at least one client is connected. Several clients can connect to one session and share its queues and stream. A command from any client changes the state for all of them. Replies go only to the client that sent the command, while state, queue, and playback messages go to every client. reset stops playback, clears both queues and the clips you could continue from, clears the tracks, and restores the defaults.

Commands

Command	Typed method	Description	Reply
`enqueue`	`enqueue()`	Queue a clip from a prompt, with optional frame images, references, or a continuation.	`clip_queued`
`play`	`play()`	Play the front ready clip, or a named one.	none (`clip_started` follows)
`stop`	`stop()`	Cut the playing clip.	none (`clip_stopped` follows)
`pop`	`pop()`	Remove a clip from either queue.	`clip_popped`
`move`	`move()`	Move a clip within its queue.	`clip_moved`
`set_clip_seconds`	`setClipSeconds()`	Set the default length for later clips.	`clip_length_accepted`
`set_seed`	`setSeed()`	Set the default seed for later clips.	`seed_accepted`
`set_autoplay`	`setAutoplay()`	Play ready clips automatically when nothing plays.	`autoplay_accepted`
`set_flush_on_clip_end`	`setFlushOnClipEnd()`	Choose black or the last frame between clips.	`flush_accepted`
`set_canvas`	`setCanvas()`	Choose the aspect ratio while the session is empty.	`canvas_accepted`
`set_resolution`	`setResolution()`	Choose `480p` or `768p` while the session is empty.	`resolution_accepted`
`reset`	`reset()`	Clear the queues and playback, and restore the defaults.	`session_reset`
`get_queue`	`getQueue()`	Read both queues.	`queue_update`
`get_state`	`getState()`	Read the session state.	`state_update`

The typed SDK, @reactor-models/pika-h3-flex, has one method for each command on PikaH3FlexModel and on the usePikaH3Flex() hook. play and stop resolve undefined. The other methods resolve their reply message, or undefined if the model refuses the command. Parameter and message fields keep their snake_case names. Pass the FileRef that uploadFile returns for each uploaded image, video, or audio clip.

`enqueue`

Add a clip to the generation queue. The model replies with clip_queued when it accepts the request. Later it sends clip_generated when the clip is ready, or clip_failed if generation fails. The inputs decide how the model builds the clip. There is no field that chooses a mode:

Inputs	How the clip is built
A prompt alone	From text
A prompt with `first_frame_image`, `last_frame_image`, or both	From first and last frames
A prompt with `continue_mode: "frame"`	Frame continuation
A prompt with reference images, videos, or audio	From references
A prompt with a latent continuation (the default), with or without references	Latent continuation

Frame images and references never share a clip. The model refuses a request that combines them with invalid_request, and it also refuses a frame continuation that has references or frame images.

Parameter	Type	Default	Description
`prompt`	string	`""`	Required. What the clip shows and sounds like. Up to 2,048 text tokens, including the labels the model adds for references, frame images, and a continued clip. The model never shortens or rewrites the prompt.
`metadata`	string	`""`	Your own string, up to 2,000 characters. The model returns it unchanged in every message about this clip.
`first_frame_image`	upload reference or null	`null`	An image the clip opens from. Not with references or `continue_from_clip_id`.
`last_frame_image`	upload reference or null	`null`	An image the clip ends on. Use it alone or with `first_frame_image`. Not with references or `continue_from_clip_id`.
`reference_image`	upload reference or null	`null`	One reference image. Send this or `reference_images`, not both.
`reference_images`	upload reference array or null	`null`	Up to 9 ordered images. The prompt names them `<Picture 1>`, `<Picture 2>`, and so on.
`reference_video`	upload reference or null	`null`	Experimental. One reference video. Send this or `reference_videos`, not both.
`reference_videos`	upload reference array or null	`null`	Experimental. Up to 3 ordered videos, each 2 to 15 seconds long and at most 15 seconds together. The prompt names them `<Video 1>`, `<Video 2>`, and so on.
`reference_audio`	upload reference or null	`null`	One audio clip, 2 to 15 seconds long. Send this or `reference_audios`, not both.
`reference_audios`	upload reference array or null	`null`	Up to 3 ordered audio clips. Each is 2 to 15 seconds long, and together they last at most 15 seconds. The sound of a reference video does not count toward these limits.
`reference_order`	array of `"image"`, `"video"`, `"audio"`, or null	`null`	The order the references are packed in, with one entry per upload, for example `["video", "audio", "image"]`. Null packs images, then videos, then audio. The order sets the label numbers.
`continue_from_clip_id`	string	`""`	ID of an earlier clip in this session to continue. Leave it blank for an independent clip.
`continue_mode`	`"latent"`, `"frame"`, or null	`null`	How to continue the clip. Null means `latent`. Refused without `continue_from_clip_id`.
`seed`	nonnegative integer or null	`null`	Null uses the session seed and then advances it by one. An explicit seed does not change the session seed.
`seconds`	number or null	`null`	One of the lengths in `state_update.clip_seconds_choices`. Null uses the session default. The model refuses other values and does not round them.
`position`	nonnegative integer or null	`null`	Position in the generation queue. 0 is the next clip to generate. Null or a value past the end adds the clip at the back.

References

Reference images guide how subjects, objects, and settings look. Reference videos guide motion, appearance, and sound. Reference audio guides voices and sound. None of them fixes the clip’s first or last frame. A clip takes up to 9 images, 3 videos, and 3 audio clips, and at most 12 references in total. A request with reference audio must also include a reference image, a reference video, or a latent continuation, or the model refuses it with invalid_request. Each clip keeps its own references, and a later clip does not reuse them. Upload each file with uploadFile() and pass the FileRef it returns. The same file uploaded twice counts as two references with two labels.

Labels

In the prompt, <Picture N> names the Nth image and <Video N> the Nth video, in the order of their own arrays. Audio clips and the sound of reference videos share the <Audio N> numbers, in packed order. A video with sound takes the next audio label just before its own place. With the default order, one image, one video with sound, and one audio clip are named like this:

Upload	Label
The image	`<Picture 1>`
The video’s picture	`<Video 1>`
The video’s sound	`<Audio 1>`
The audio clip	`<Audio 2>`

If the video has no sound, the audio clip is <Audio 1>. If you change reference_order, the labels change too, so update the prompt to match.

Reference videos

Reference videos are experimental. Each video is 2 to 15 seconds long, at most 64 MiB, and has an aspect ratio between 1:4 and 4:1. The model samples each video at 24 frames per second, cuts it to the clip’s length, and fits it to its own aspect ratio at the session’s resolution. If the video has sound, the sound guides the clip too. The model refuses a video that does not decode or is out of range with invalid_request.

First and last frames

first_frame_image sets how the clip opens, and last_frame_image sets how it ends. Send one or both. The model center-crops each image and resizes it to the canvas. It rebuilds the image rather than pasting it, so the first or last frame is close to the image but not a pixel copy. Describe the action between the two frames in the prompt. In the prompt, a single frame image is <Picture 1>. With both, the first frame is <Picture 1> and the last frame is <Picture 2>. The prompt guide shows the alignment line Pika recommends.

Continuation

continue_from_clip_id continues a clip from this session, whichever way it was built. continue_mode says how:

	`latent` (the default)	`frame`
How the clip is built	From references	From the source clip’s last frame
What carries over	The last 2.33 seconds of the source clip, picture and sound	The source clip’s last frame only. The sound may restart at the boundary.
Which clips can be used	One of the 8 most recently built clips, at the current canvas	Any clip in either queue or in `queue_update.history`, built or not. The new clip waits for it.
References of its own	Allowed	Not allowed: the prompt only, with no references and no frame images

In a latent continuation, the end of the source clip is packed last, as a video reference. Its sound takes the next audio label after all your other audio labels, but it does not use an audio slot. You can still upload 3 audio clips: with three uploads and no reference video, the continued sound is <Audio 4>. The end of the source clip does count as one of the 3 reference videos, so a latent continuation accepts at most 2 reference videos, at most 12.67 seconds together. Wait for the source clip’s clip_generated before you queue a latent continuation. You can queue a frame continuation right away, even before its source has generated. It waits in the generation queue until its source is built, and the clips behind it keep generating. queue_update.history lists the built clips that left both queues. You can no longer play them, but you can continue them. reset and the end of the session remove them.

Lengths

seconds and set_clip_seconds accept 25 values: 5, 5.167, 5.875, 6, 6.583, 7, 7.292, 8, 8.708, 9, 9.417, 10, 10.125, 10.833, 11, 11.542, 12, 12.25, 12.958, 13, 13.667, 14, 14.375, 15, and 15.083 seconds. The whole numbers are shortcuts: each one gives the shortest generated length at or above it, so 5 gives 5.167 seconds, 8 gives exactly 8, and 15 gives 15.083. clip.seconds reports the length the clip actually has. The model refuses any other value and does not round it. Read the list from state_update.clip_seconds_choices rather than hard-coding it.

Limits

The model checks each request before it accepts it, and Refusals lists the codes. Each image can have at most 25 million pixels, with an aspect ratio between 1:4 and 4:1. Each upload can be at most 64 MiB, and a clip’s uploads at most 128 MiB together. The model refuses any field it does not take with invalid_request.

const portrait = await reactor.uploadFile(portraitFile);
const voice = await reactor.uploadFile(voiceFile);

const queued = await reactor.sendCommand("enqueue", {
  prompt:
    "<Picture 1> is a singer in a small rehearsal room. <Audio 1> is her voice. She faces the microphone and sings, “Stay with the morning light.” A soft piano plays under her voice.",
  reference_images: [portrait],
  reference_audios: [voice],
  seconds: 8,
  metadata: "verse-1",
});

import { usePikaH3Flex } from "@reactor-models/pika-h3-flex";

const { uploadFile, enqueue } = usePikaH3Flex();
const portrait = await uploadFile(portraitFile);
const voice = await uploadFile(voiceFile);

await enqueue({
  prompt:
    "<Picture 1> is a singer in a small rehearsal room. <Audio 1> is her voice. She faces the microphone and sings, “Stay with the morning light.” A soft piano plays under her voice.",
  reference_images: [portrait],
  reference_audios: [voice],
  seconds: 8,
  metadata: "verse-1",
});

portrait = await reactor.upload_file("singer.png")
voice = await reactor.upload_file("voice.wav")

await reactor.send_command("enqueue", {
    "prompt": "<Picture 1> is a singer in a small rehearsal room. <Audio 1> is her voice. "
    "She faces the microphone and sings, “Stay with the morning light.” "
    "A soft piano plays under her voice.",
    "reference_images": [portrait],
    "reference_audios": [voice],
    "seconds": 8,
    "metadata": "verse-1",
})

To continue that clip, wait for its clip_generated message and keep its clip.clip_id, then enqueue the next clip with that ID. This request uploads one audio clip, so the continued clip’s soundtrack becomes <Audio 2>:

await reactor.sendCommand("enqueue", {
  prompt:
    "<Audio 1> is the singer's voice. She holds the last note, then smiles and steps back from the microphone. <Audio 2> continues the piano.",
  reference_audios: [voice],
  continue_from_clip_id: generatedClipId,
  seconds: 5.875,
});

await enqueue({
  prompt:
    "<Audio 1> is the singer's voice. She holds the last note, then smiles and steps back from the microphone. <Audio 2> continues the piano.",
  reference_audios: [voice],
  continue_from_clip_id: generatedClipId,
  seconds: 5.875,
});

await reactor.send_command("enqueue", {
    "prompt": "<Audio 1> is the singer's voice. She holds the last note, then smiles "
    "and steps back from the microphone. <Audio 2> continues the piano.",
    "reference_audios": [voice],
    "continue_from_clip_id": generated_clip_id,
    "seconds": 5.875,
})

To build a clip between two images, send the frame images and no references. This prompt follows Pika’s alignment line for an 8-second clip:

const opening = await reactor.uploadFile(openingFile);
const ending = await reactor.uploadFile(endingFile);

await reactor.sendCommand("enqueue", {
  prompt:
    "How the reference pictures align with the target video — <Picture 1> (from [Shot 1]) aligns with the 0.00-second mark of the target video; <Picture 2> (from [Shot 1]) aligns with the 8.00-second mark of the target video.\n\nintegrated_multimodal_description: [Shot 1] Begin aligned with <Picture 1>. Her hands raise the wet clay into a tall cylinder, then open it into a wide bowl. End aligned with <Picture 2>.\noverall_soundscape: The wheel hums and wet clay squeaks under her fingers.\nnon_diegetic_music: N/A",
  first_frame_image: opening,
  last_frame_image: ending,
  seconds: 8,
});

const opening = await uploadFile(openingFile);
const ending = await uploadFile(endingFile);

await enqueue({
  prompt:
    "How the reference pictures align with the target video — <Picture 1> (from [Shot 1]) aligns with the 0.00-second mark of the target video; <Picture 2> (from [Shot 1]) aligns with the 8.00-second mark of the target video.\n\nintegrated_multimodal_description: [Shot 1] Begin aligned with <Picture 1>. Her hands raise the wet clay into a tall cylinder, then open it into a wide bowl. End aligned with <Picture 2>.\noverall_soundscape: The wheel hums and wet clay squeaks under her fingers.\nnon_diegetic_music: N/A",
  first_frame_image: opening,
  last_frame_image: ending,
  seconds: 8,
});

opening = await reactor.upload_file("clay.png")
ending = await reactor.upload_file("bowl.png")

await reactor.send_command("enqueue", {
    "prompt": "How the reference pictures align with the target video — <Picture 1> (from [Shot 1]) "
    "aligns with the 0.00-second mark of the target video; <Picture 2> (from [Shot 1]) aligns "
    "with the 8.00-second mark of the target video.\n\n"
    "integrated_multimodal_description: [Shot 1] Begin aligned with <Picture 1>. Her hands raise "
    "the wet clay into a tall cylinder, then open it into a wide bowl. End aligned with <Picture 2>.\n"
    "overall_soundscape: The wheel hums and wet clay squeaks under her fingers.\n"
    "non_diegetic_music: N/A",
    "first_frame_image": opening,
    "last_frame_image": ending,
    "seconds": 8,
})

A frame continuation opens on the last frame of another clip and takes a prompt only. The source clip’s last frame is <Picture 1>:

await reactor.sendCommand("enqueue", {
  prompt:
    "For the target video, at 0.00 seconds into the target video, <Picture 1> (from [Shot 1]) is fully referenced.\n\nintegrated_multimodal_description: [Shot 1] Begin from <Picture 1>. She lifts the bowl off the wheel and carries it to a shelf by the window.\noverall_soundscape: Footsteps on a wooden floor and rain on the window.\nnon_diegetic_music: N/A",
  continue_from_clip_id: bowlClipId,
  continue_mode: "frame",
});

await enqueue({
  prompt:
    "For the target video, at 0.00 seconds into the target video, <Picture 1> (from [Shot 1]) is fully referenced.\n\nintegrated_multimodal_description: [Shot 1] Begin from <Picture 1>. She lifts the bowl off the wheel and carries it to a shelf by the window.\noverall_soundscape: Footsteps on a wooden floor and rain on the window.\nnon_diegetic_music: N/A",
  continue_from_clip_id: bowlClipId,
  continue_mode: "frame",
});

await reactor.send_command("enqueue", {
    "prompt": "For the target video, at 0.00 seconds into the target video, <Picture 1> "
    "(from [Shot 1]) is fully referenced.\n\n"
    "integrated_multimodal_description: [Shot 1] Begin from <Picture 1>. She lifts the bowl off "
    "the wheel and carries it to a shelf by the window.\n"
    "overall_soundscape: Footsteps on a wooden floor and rain on the window.\n"
    "non_diegetic_music: N/A",
    "continue_from_clip_id": bowl_clip_id,
    "continue_mode": "frame",
})

`play`

Play a ready clip. Leave out clip_id to play the clip at the front of the playout queue. The model sends clip_started when the first frames reach the tracks and clip_finished when the clip ends.

Parameter	Type	Default	Description
`clip_id`	string	`""`	Optional. ID of a clip in the playout queue.

A clip plays only once, and you cannot play it again afterward. You can continue from a clip in queue_update.history, but you cannot play it.

await reactor.sendCommand("play", { clip_id: generatedClipId });

const { play } = usePikaH3Flex();
await play({ clip_id: generatedClipId });

await reactor.send_command("play", {"clip_id": generated_clip_id})

`stop`

Cut the playing clip and leave the queues as they are. With autoplay on, stop skips to the next ready clip. With flush on, the video cuts to black. With flush off, it holds the last frame once the stream drains.

await reactor.sendCommand("stop", {});

const { stop } = usePikaH3Flex();
await stop();

await reactor.send_command("stop", {})

`pop` and `move`

Command	Parameters	Behavior
`pop`	`clip_id: string`	Removes the clip from whichever queue holds it. You can still continue from it if it finished generating. The model refuses to remove a clip that is still generating while queued clips continue from it.
`move`	`clip_id: string`, `position: integer`	Moves the clip within its own queue. 0 is the front. A position past the end moves the clip to the back. A clip never moves to the other queue.

await reactor.sendCommand("move", { clip_id: clipId, position: 0 });
await reactor.sendCommand("pop", { clip_id: otherClipId });

const { move, pop } = usePikaH3Flex();
await move({ clip_id: clipId, position: 0 });
await pop({ clip_id: otherClipId });

await reactor.send_command("move", {"clip_id": clip_id, "position": 0})
await reactor.send_command("pop", {"clip_id": other_clip_id})

Session settings

Command	Parameters	Behavior
`set_clip_seconds`	`seconds`: a value from `clip_seconds_choices`	Sets the length for clips that you enqueue without `seconds`. Queued clips keep their length. If you send it without `seconds`, it uses 15.083.
`set_seed`	`seed: nonnegative integer`	Sets the seed for clips that you enqueue without `seed`. Each such clip advances it by one. The same prompts with the same seeds give close results, but not always identical ones.
`set_autoplay`	`enabled: boolean`	When on, the front ready clip starts whenever nothing is playing. A clip that continues the one that just finished plays on with no cut. Autoplay does not create clips.
`set_flush_on_clip_end`	`enabled: boolean`	When on, the video goes black between clips. When off, it holds the last frame.
`set_canvas`	`aspect`	Sets the aspect ratio: `21:9`, `16:9`, `3:2`, `4:3`, `5:4`, `1:1`, `4:5`, `3:4`, `2:3`, `9:16`, or `9:21`. The model accepts it only when both queues are empty and nothing plays. The resolution stays the same. `canvas_accepted` reports the width and height in pixels.
`set_resolution`	`resolution: "480p" \| "768p"`	Sets the resolution: `480p` (832 × 480 at 16:9) or `768p` (1344 × 768 at 16:9, the default). The aspect ratio stays the same. The model accepts it only when both queues are empty and nothing plays. `resolution_accepted` reports the width and height in pixels.
`reset`	none	Clears both queues and the kept clips, cuts playback, clears the tracks, and restores the defaults.
`get_queue`	none	Replies with `queue_update`.
`get_state`	none	Replies with `state_update`.

state_update carries the current defaults: clip_seconds is the current default length, and clip_seconds_choices lists the allowed lengths. resolution is the current resolution, and resolution_choices lists the ones you can set.

await reactor.sendCommand("set_canvas", { aspect: "9:16" });
await reactor.sendCommand("set_resolution", { resolution: "480p" });
await reactor.sendCommand("set_clip_seconds", { seconds: 8 });
await reactor.sendCommand("set_autoplay", { enabled: true });

const { setCanvas, setResolution, setClipSeconds, setAutoplay } = usePikaH3Flex();
await setCanvas({ aspect: "9:16" });
await setResolution({ resolution: "480p" });
await setClipSeconds({ seconds: 8 });
await setAutoplay({ enabled: true });

await reactor.send_command("set_canvas", {"aspect": "9:16"})
await reactor.send_command("set_resolution", {"resolution": "480p"})
await reactor.send_command("set_clip_seconds", {"seconds": 8})
await reactor.send_command("set_autoplay", {"enabled": True})

set_seed changes the seed for later clips. set_flush_on_clip_end takes effect at the next clip boundary. get_state and get_queue reply with the same payload that the model broadcasts. reset starts the session again from empty:

await reactor.sendCommand("set_seed", { seed: 42 });
await reactor.sendCommand("set_flush_on_clip_end", { enabled: false });

const state = await reactor.sendCommand("get_state", {});
const queue = await reactor.sendCommand("get_queue", {});

await reactor.sendCommand("reset", {});

const { setSeed, setFlushOnClipEnd, getState, getQueue, reset } = usePikaH3Flex();
await setSeed({ seed: 42 });
await setFlushOnClipEnd({ enabled: false });

const state = await getState(); // state?.clip_seconds is the current default length
const queue = await getQueue(); // queue?.playout lists the ready clips

await reset();

await reactor.send_command("set_seed", {"seed": 42})
await reactor.send_command("set_flush_on_clip_end", {"enabled": False})

state = await reactor.send_command("get_state", {})
queue = await reactor.send_command("get_queue", {})

await reactor.send_command("reset", {})

Refusals

When the model refuses a command, nothing changes and the call fails. Only the client that sent the command gets the error. It carries a stable code and a reason you can show to your user.

Command	Codes
`enqueue`	`empty_prompt`, `metadata_too_long`, `queue_full`, `request_rejected`, `invalid_image`, `invalid_audio`, `invalid_request`
`play`	`already_playing`, `no_ready_clip`
`stop`	`not_playing`
`pop`	`unknown_clip`, `has_dependents`
`move`	`unknown_clip`
`set_clip_seconds`	`invalid_seconds`
`set_canvas`	`session_not_empty`, `invalid_aspect`
`set_resolution`	`session_not_empty`, `invalid_resolution`

request_rejected: the prompt is over the 2,048-token budget. The reason gives the count and the limit.
invalid_image: a reference image or frame image does not decode.
invalid_audio: an audio clip does not decode or is outside 2 to 15 seconds. This code also applies when the audio clips last more than 15 seconds together.
invalid_request: the request breaks a rule that is not about one file. For example:
- a length that is not offered
- too many references: more than 9 images, 3 videos, or 3 audio clips, or more than 12 in all
- frame images together with references, or a frame continuation with references or frame images
- reference audio without an image, a video, or a latent continuation
- a reference video that does not decode or is out of range, or videos that last more than 15 seconds together (the end of a latent-continued clip counts as 2.33 seconds)
- a reference_order that does not name each upload once
- a continuation ID that the model does not hold, or continue_mode without a continuation
- an upload over the size limits, or a field that this model does not take
- a canvas or resolution change while the model prepares the request
no_ready_clip: the ID is unknown, the clip is still generating, or the playout queue is empty.
has_dependents: queued clips continue from this clip, and it has not finished generating.
session_not_empty: a clip is queued or playing, so the size cannot change.

In JavaScript, the error event fires with operation: "sendCommand" and the code, and then the awaited call resolves undefined. Commands that declare a reply resolve undefined only when the model refuses them. play and stop declare no reply and resolve undefined on success too, so for these two only the error event shows a refusal. In Python, send_command() raises a ReactorError with the code.

reactor.on("error", (error) => {
  if (error.operation === "sendCommand") showRefusal(error.code, error.message);
});

const queued = await reactor.sendCommand("enqueue", { prompt });
if (queued === undefined) return; // Refused. The error listener has the reason.

The model also broadcasts a command_error message to every client when it refuses a command. That broadcast is legacy. Read the refusal from the call instead. See Replies and refusals.

Messages

Message	When it arrives	Payload
`state_update`	On connect, after each state change, and in reply to `get_state`.	Session state.
`queue_update`	On connect, after each queue change, and in reply to `get_queue`.	`generation`, `playout`, `history`.
`clip_queued`	Reply to `enqueue`.	`clip`.
`clip_generated`	A clip finished generating and joined the playout queue.	`clip`.
`clip_failed`	A clip failed to generate and left the generation queue.	`clip`, `reason`.
`clip_started`	A clip’s first frames reached the tracks.	`clip`.
`clip_finished`	A clip finished streaming.	`clip`, `seconds_sent`.
`clip_stopped`	`stop` or `reset` cut the playing clip.	`clip`, `seconds_sent`.
`clip_moved`	Reply to `move`.	`clip`, `queue`, `position`.
`clip_popped`	Reply to `pop`.	`clip`.
`clip_length_accepted`	Reply to `set_clip_seconds`.	`clip_seconds`, `frames`.
`seed_accepted`	Reply to `set_seed`.	`seed`.
`autoplay_accepted`	Reply to `set_autoplay`.	`enabled`.
`flush_accepted`	Reply to `set_flush_on_clip_end`.	`enabled`.
`canvas_accepted`	Reply to `set_canvas`.	`aspect`, `width`, `height`.
`resolution_accepted`	Reply to `set_resolution`.	`resolution`, `width`, `height`.
`session_reset`	Reply to `reset`.	`cleared_clips`, `was_playing`.
`command_error`	Legacy broadcast of a refusal. Read the refusal from the call instead.	`command`, `reason`.

A failed clip leaves the generation queue, and generation moves on to the next clip. In clip_finished and clip_stopped, seconds_sent is the total for the session, not for one clip.

`ClipInfo`

Every message about a clip carries the full ClipInfo record. The first seven fields are always present. The others are optional.

Field	Type	Meaning
`clip_id`	string	The clip’s UUID.
`prompt`	string	The prompt as you sent it.
`metadata`	string	Your `metadata` string, unchanged.
`frames`	integer	Number of video frames.
`seconds`	number	Clip length in seconds.
`seed`	integer	The seed used for this clip.
`ready`	boolean	True when the clip has finished generating.
`adapter`	string	How the clip was built. `fl2va` means from text, frame images, or a frame continuation. `ref2va` means from references or a latent continuation.
`has_first_frame`, `has_last_frame`	boolean	Whether the clip’s first or last frame is set, by a frame image or by a frame continuation.
`has_reference_image`, `reference_image_count`	boolean, integer	Whether the clip has uploaded images, and how many.
`has_reference_video`, `reference_video_count`	boolean, integer	Whether the clip has uploaded videos, and how many. The end of a continued clip is not counted.
`has_reference_audio`, `reference_audio_count`	boolean, integer	Whether the clip has uploaded audio, and how many. A continued clip’s sound is not counted.
`continue_from_clip_id`	string or null	The clip that this clip continues, or null.
`continue_mode`	string or null	`latent` or `frame` for a continuation, or null.
`prompt_prefix`	string	Always empty. The model does not change your prompt.
`build_seconds`	number or null	How long generation took. Null until the clip is ready.
`attention`, `attention_used`	string, string or null	The attention setting the clip was queued with, and the one the build used. `attention_used` is null until the clip is ready. The service sets both; you cannot choose them.

`state_update`

Field	Meaning
`clip_seconds`	Length of clips that you enqueue without `seconds`.
`clip_seconds_choices`	The allowed lengths.
`seed`	Seed for the next clip that you enqueue without `seed`.
`autoplay`	Whether ready clips start automatically.
`flush_on_clip_end`	Whether the video goes black between clips.
`aspect`, `width`, `height`	Aspect ratio and video size in pixels.
`resolution`, `resolution_choices`	The current resolution, `480p` or `768p`, and the ones `set_resolution` accepts.
`playing`, `playing_clip_id`	Whether a clip plays, and its ID. The ID is null when nothing plays.
`generation_queued`, `generation_capacity`	Clips in the generation queue, and its maximum size.
`playout_queued`, `playout_capacity`	Clips in the playout queue, and its maximum size. Generation pauses while the playout queue is full.
`clips_played`, `seconds_sent`	Clips played and seconds streamed in this session. `reset` does not clear these counters.
`valid_commands`	The commands that the session accepts now.

`queue_update`

Field	Meaning
`generation`	Clips that wait to generate, front first.
`playout`	Clips that are ready to play, front first.
`history`	Built clips that left both queues because they played, stopped, or were popped. You cannot play them again. You can frame-continue any of them, and latent-continue the ones among the 8 most recently built clips. The oldest clip is first and expires next.

Complete example

This example queues a clip from a prompt and plays it when it is ready. The tutorial adds references and continuation.

import { Reactor } from "@reactor-team/js-sdk";

const reactor = new Reactor({ modelName: "reactor/pika-h3-flex" });

reactor.on("error", (error) => {
  if (error.operation === "sendCommand") console.error(error.code, error.message);
});

reactor.on("message", async (message) => {
  if (message.type === "clip_generated") {
    const { clip } = message.data as { clip: { clip_id: string } };
    await reactor.sendCommand("play", { clip_id: clip.clip_id });
  }
  if (message.type === "clip_failed") console.error((message.data as { reason: string }).reason);
});

reactor.on("statusChanged", async (status) => {
  if (status !== "ready") return;
  await reactor.sendCommand("enqueue", {
    prompt:
      "A paper boat drifts down a rain-filled gutter at dusk. The camera follows at street level. Rain patters and water gurgles. No music.",
  });
});

await reactor.connect(await getToken()); // Your server issues the token.