Skip to main content
This page is the protocol reference for PixVerse R2 Persona. It covers the tracks, the session lifecycle, the commands, the refusals, and the messages. The model streams the character on main_video and main_audio. It hears you on mic. When the session reaches ready, send commands with reactor.sendCommand().

Tracks

Before a call and between calls, main_video streams black frames in the aspect ratio that you set, at 5 fps, and main_audio carries silence. During a call, from live on, the tracks carry the character at 25 fps. While the video stalls, the last frame repeats. Publish mic before start_call whenever the user may speak. If a published microphone sends nothing, the character hears nothing. The character does not answer speech that comes before speech_ready is true.

Session lifecycle

A session holds one character and at most one call at a time:
  • create_character makes the character from one image, a voice, a name, and a persona. attach_character makes a saved character current.
  • start_call starts a call. The call ends on end_call, on a failure, or when the session ends. It also ends after idle_seconds with nothing said to the character, or at call_max_seconds. The character stays current, so the next start_call starts a call with the same character.
session_state reports the phase: Several clients can join one session. Each client gets session_state when it connects, sees every broadcast message, and hears the character. The character hears the mic track of every client. When a client leaves, the call continues. When the session ends, the call ends. Replies go only to the client that sent the command. question_received, transcript, character_action, input_declined, and session_state go to everyone.

Commands

The typed SDK, @reactor-models/pixverse-r2-persona, has one method for each command on PixverseR2PersonaModel and on the usePixverseR2Persona() hook. startCall resolves undefined. The other methods resolve their reply message. If the model refuses the command, they resolve undefined. Parameter and message fields keep their snake_case names. Pass the FileRef that uploadFile returns as image. PixverseR2PersonaModel also has publishMic(track) and unpublishMic() for the mic track.

list_voices

List every voice, and get default_voice. You can send it in any phase. The model replies with voices. Each voice has voice (the name to pass to create_character), description, and languages.

create_character

Create a character from one image, a voice, a name, and a persona, and make it current. You can send it in idle, character_ready, ended, and failed. The phase moves to creating_character. About 2 to 4 seconds later, the model replies with character_ready, which carries character_id. The phase then becomes character_ready. If the model refuses the request, the previous character stays current, if there is one. Upload the image with uploadFile(). Then pass the FileRef that it returns.

attach_character

Make a character from an earlier session current, without creating it again. You can send it in idle, character_ready, ended, and failed. The model replies with character_ready at once. Its character_name, voice, and aspect_ratio are null, because the model does not know them. The model checks the id when the next call starts. If the id is unknown, that call fails with last_error code character_not_found, and the model clears character_id.

start_call

Start a call with the current character. You can send it in character_ready, ended, and failed. The command resolves when the model accepts the request. Then session_state moves through starting, connecting, and live. speech_ready turns true during connecting, about 2 seconds after the call opens. When capacity is free, start_call to live takes 8 to 10 seconds. A wait for capacity in starting adds to that time. The wait lasts up to 120 seconds.

say

Say something to the character, as text. You can send it while speech_ready is true, during connecting or live. When the model delivers the text, it replies with said, which carries the prompt_id of the turn. Then the character sends question_received and speaks its answer on main_audio. It sends one transcript line for each sentence. If the content policy declines the text, the character sends input_declined and says its refusal reply instead.

end_call

End the call. You can send it in any phase, and you can safely send it again. After the call ends, the model replies with call_ended, which carries end_reason and duration_seconds. The character stays current.

get_state

Read the session state. You can send it in any phase. The model replies with session_state. You rarely need this command, because the model sends session_state when you connect and on every change.

Refusals

When the model refuses a command, nothing changes, and the command fails. Only the client that sent the command gets the error. The error carries a stable code and a reason that you can show to your user. The model never refuses list_voices, end_call, or get_state.
  • invalid_input: a blank name, persona, or text, or an id that is not a character_id.
  • invalid_image: the model cannot read the image, or the image is over 20 MiB or 40 megapixels.
  • invalid_voice: the voice is not in list_voices.
  • text_too_long: the text is over 800 bytes.
  • busy: a call is starting or running, or the model is creating a character.
  • no_character: start_call when no character is current.
  • not_live: say while no call is listening (speech_ready is false).
  • send_failed: the model could not deliver the text. Retryable.
  • content_policy: the content policy declined the image or text.
  • upstream_rejected: the generation service refused the image or text.
create_character, attach_character, and say can also fail with three upstream codes. upstream_unavailable means that the model could not reach the generation service. A retry may work. upstream_auth and upstream_protocol mean a deployment problem on the Reactor side of the contract. Report them with trace_id. In JavaScript, a refusal fires the error event with operation: "sendCommand" and the code. Then the awaited command resolves undefined. A command that declares a reply resolves undefined only when the model refuses it. start_call declares no reply, so it also resolves undefined on success. For start_call, only the error event shows a refusal. In Python, send_command() raises a ReactorError with the code.
See Replies and refusals.

Failures

A failure that no command caused is not a refusal. It ends the call, sets the phase to failed, and sets last_error on session_state. last_error has these fields:

Messages

transcript payload

session_state payload

end_reason is one of seven values. ended_by_client (end_call), idle, and max_duration end in ended. start_failed (the call never went live), media_lost, upstream_interrupted, and platform_failed end in failed with last_error.

Complete example

This example creates a character from an image and starts a call. When speech_ready turns true, it says one line to the character. It also prints the subtitles. The tutorial adds the microphone, subtitle handling, and character reuse.