Skip to main content
This page is the protocol reference for PixVerse R2 Director. It covers the tracks, the session lifecycle, the commands, the refusals, and the messages. The model streams video on main_video. It has no audio track. When the session reaches ready, send commands with reactor.sendCommand().

Tracks

There are no input tracks. The model gets all its input from commands and uploads. The track sends nothing before the session’s first play. After that, it always carries a picture: the playing clip, or the last frame sent.

Session lifecycle

A session holds one world and at most one open run at a time:
  • The world is the place where the model makes clips. create_world makes one from a description, reference images, or both. attach_world makes a saved one current.
  • A run is a sequence of clips that continue one another. The first enqueue on a world starts a run. The run ends with end_run, after idle_seconds with nothing to build, at max_seconds (1,800 seconds), or when the session ends. The next enqueue starts a new run on the same world. The first clip of the new run does not continue the picture of the last run.
Two queues hold the clips:
  • The generation queue holds clips that are queued or generating. They build in order, and each one continues from the last frame of the one before.
  • The playout queue holds built clips that are ready and wait for play.
play sends the oldest ready clip, or a named one, on main_video. stop stops it. set_autoplay plays ready clips on their own, oldest first. Built clips stay ready after their run ends, so you can play them at any time. session_state.phase says where the session is: Several clients can connect to one session. Each client gets session_state when it connects. All clients see the same snapshot and the same tracks, and each one can send play or stop. A reply goes only to the client that sent the command. The other messages go to every client.

Commands

The typed SDK, @reactor-models/pixverse-r2-director, has one method for each command on PixverseR2DirectorModel and on the usePixverseR2Director() hook. createWorld, attachWorld, play, and stop resolve undefined. The other methods resolve their reply message. If the model refuses the command, they resolve undefined. Parameter fields and message fields keep their snake_case names. For each uploaded image, pass the FileRef that uploadFile returns.

create_world

Create the world where the model makes clips, and make it current. Valid in idle, world_ready, ended, and failed. The phase moves to creating_world. After 4 to 12 seconds, it moves to world_ready, with world_id and reference_images set. Clips that you queue in the meantime wait for the world. If the model cannot create the world, the previous world stays current. The model sets last_error and fails the clips that waited for the world. Upload each image with uploadFile(). Then pass the FileRef that it returns.

attach_world

Make a world from an earlier session current, without creating it again. Valid in idle, world_ready, ended, and failed. The phase moves to world_ready at once. reference_images is null, because the model does not know how many images the world has. When the first clip of the next run starts, the model checks the id. If the id is unknown, that run fails with last_error code world_not_found, and the model clears world_id.

enqueue

Queue one clip of the current world. Valid in every phase except idle and ending. The model replies with clip_queued. When the clip is ready, the model sends clip_generated. If the model cannot build the clip, it sends clip_failed. A built clip waits for play. When autoplay is on, the clip plays on its own. set_autoplay turns autoplay on. From world_ready, ended, or failed, enqueue starts a new run. During creating_world, the clip waits for the world.

Reference images

Each reference image of a world has an index from 0, in the order that create_world received the images. Image 0 is the opening image of the world. The first clip of every run opens from image 0, and that clip takes no reference options. From the second clip of a run on, enqueue can tell the model how a clip uses the images: The reference options follow three rules:
  • A nonempty reference_indexes cannot go with a frame reference.
  • Each index is below the reference_images count of the world.
  • Each index appears once.
If a request breaks a rule, enqueue refuses it with invalid_reference. For an attached world, the model does not know the image count. So an index out of range fails that clip instead. Not every generation worker supports last_frame_ref. If the worker cannot build the clip, the clip fails with unsupported_reference.

play

Play a ready clip on main_video. The command plays the clip that clip_id names. If clip_id is blank, it plays the oldest ready clip. Valid in any phase, even after the run of the clip ends. When the first frame goes out, the model sends clip_started. After the model sends the whole clip, it sends clip_finished. If stop cuts the clip, the model sends clip_stopped instead.

stop

Stop playing the current clip, or the clip that play just asked for. The picture holds on the last frame sent. The model sends clip_stopped, and the status of the clip becomes stopped.

set_autoplay

Choose whether ready clips play on their own, oldest first, when no clip plays. Off by default. Valid in any phase. The model replies with autoplay_accepted and sends session_state. This command never fails. To play without gaps, wait until a few clips are ready, then turn autoplay on.

pop

Remove a clip that is queued or ready. Valid in any phase. The model replies with clip_popped and sends session_state. The clip becomes cancelled. The model does not build a queued clip that you remove. To remove the playing clip, send stop instead.

end_run

End the open run. Valid in any phase. The model replies with session_state. Clips that are queued or generating become cancelled. Built clips stay ready, and you can still play them. The next enqueue starts a new run on the same world.

get_state

Read the session state. Valid in any phase. The model replies with session_state. You rarely need it, because the model sends session_state on connect and on every change.

Refusals

When the model refuses a command, nothing changes, and the call fails. Only the client that sent the command gets the error. The error carries a stable code and a reason that you can show to your user. The model never refuses set_autoplay, end_run, or get_state.
  • invalid_input: a blank prompt, a world with no description and no image, more than 6 images, or a malformed world_id.
  • invalid_image: an image that does not decode, or that is over 20 MiB or 40 megapixels.
  • invalid_reference: reference options that the world or the clip cannot take. See Reference images.
  • busy: create_world or attach_world while the model creates a world or while a run is open, or enqueue while a run closes.
  • no_world: enqueue before a world is set.
  • queue_full: enqueue when max_pending_clips clips are building, or when the ready and building clips together reach max_ready_clips.
  • already_playing: play while a clip plays. Send stop first.
  • no_ready_clip: play when no ready clip matches.
  • not_playing: stop when nothing plays.
  • clip_not_found: pop with an id that is not in this session.
  • clip_playing: pop of the playing clip. Use stop.
  • clip_not_pending: pop after the clip is generating or done, or at the moment that the model sends it for generation. The last case is retryable.
In JavaScript, a refused command resolves undefined, and the SDK emits an error event with operation: "sendCommand" and the refusal’s code. A command that has a reply resolves undefined only when the model refuses it. create_world, attach_world, play, and stop have no reply, so they resolve undefined on success too. For these four commands, only the error event shows a refusal. In Python, send_command() raises a ReactorError with the code.
See Replies and refusals.

Failures

A failure that no command caused does not refuse a call. Instead, it sets last_error on session_state. For a clip, it sets error_code on the clip, and the clip arrives in clip_failed. last_error has these fields: A failed clip leaves the generation queue, and the clips after it still build.

Messages

clip payload

Every clip message carries the whole clip under clip, in the same shape as the entries of session_state.clips: cancelled means that pop removed the clip, or that its run ended before the model built it.

session_state payload

ended_by_client, idle, and max_duration come with phase ended. Every other end_reason comes with phase failed and a last_error.

Step results

The model keeps every clip as one step of the session, under /sessions/{id}/steps. The step of a built clip holds three files. output.mp4 is the clip as the track plays it, in H.264. last_frame.png is its last frame. clip.json holds the clip_id, prompt, metadata, seconds, and reference options of the clip. A clip that fails is a failed step. So is a clip that pop or the end of its run cancels. The error of the step says why: <code>: <reason> for a failed clip, cancelled: removed with pop, or cancelled: its run ended (<end_reason>).

Complete example

This example creates a world from a description and queues three clips. When two clips are ready, it turns on autoplay, so the clips play back to back. The tutorial adds reference images, cuts, and world reuse.