main_video and main_audio. It hears you on mic. When the session reaches ready, send
commands with reactor.sendCommand().
Tracks
Before a call and between calls,
main_video streams black frames in the aspect ratio that you set,
at 5 fps, and main_audio carries silence. During a call, from live on, the tracks carry the
character at 25 fps. While the video stalls, the last frame repeats.
Publish mic before start_call whenever the user may speak. If a published microphone sends
nothing, the character hears nothing. The character does not answer speech that comes before
speech_ready is true.
Session lifecycle
A session holds one character and at most one call at a time:create_charactermakes the character from one image, a voice, a name, and a persona.attach_charactermakes a saved character current.start_callstarts a call. The call ends onend_call, on a failure, or when the session ends. It also ends afteridle_secondswith nothing said to the character, or atcall_max_seconds. The character stays current, so the nextstart_callstarts a call with the same character.
session_state reports the phase:
Several clients can join one session. Each client gets
session_state when it connects, sees every
broadcast message, and hears the character. The character hears the mic track of every client.
When a client leaves, the call continues. When the session ends, the call ends.
Replies go only to the client that sent the command. question_received, transcript,
character_action, input_declined, and session_state go to everyone.
Commands
The typed SDK,
@reactor-models/pixverse-r2-persona, has one method for each command on
PixverseR2PersonaModel and on the usePixverseR2Persona() hook. startCall resolves undefined.
The other methods resolve their reply message. If the model refuses the command, they resolve
undefined.
Parameter and message fields keep their snake_case names. Pass the FileRef that uploadFile
returns as image. PixverseR2PersonaModel also has publishMic(track) and unpublishMic() for
the mic track.
list_voices
List every voice, and get default_voice. You can send it in any phase. The model replies with
voices. Each voice has voice (the name to pass to create_character), description, and
languages.
create_character
Create a character from one image, a voice, a name, and a persona, and make it current. You can send
it in idle, character_ready, ended, and failed. The phase moves to creating_character.
About 2 to 4 seconds later, the model replies with character_ready, which carries character_id.
The phase then becomes character_ready. If the model refuses the request, the previous character
stays current, if there is one.
Upload the image with
uploadFile(). Then pass the FileRef that
it returns.
attach_character
Make a character from an earlier session current, without creating it again. You can send it in
idle, character_ready, ended, and failed. The model replies with character_ready at once.
Its character_name, voice, and aspect_ratio are null, because the model does not know them.
The model checks the id when the next call starts. If the id is unknown, that call fails with
last_error code character_not_found, and the model clears character_id.
start_call
Start a call with the current character. You can send it in character_ready, ended, and
failed. The command resolves when the model accepts the request. Then session_state moves
through starting, connecting, and live.
speech_ready turns true during connecting, about 2 seconds after the call opens. When capacity
is free, start_call to live takes 8 to 10 seconds. A wait for capacity in starting adds to
that time. The wait lasts up to 120 seconds.
say
Say something to the character, as text. You can send it while speech_ready is true, during
connecting or live. When the model delivers the text, it replies with said, which carries the
prompt_id of the turn. Then the character sends question_received and speaks its answer on
main_audio. It sends one transcript line for each sentence. If the content policy declines the
text, the character sends input_declined and says its refusal reply instead.
end_call
End the call. You can send it in any phase, and you can safely send it again. After the call ends,
the model replies with call_ended, which carries end_reason and duration_seconds. The
character stays current.
get_state
Read the session state. You can send it in any phase. The model replies with session_state. You
rarely need this command, because the model sends session_state when you connect and on every
change.
Refusals
When the model refuses a command, nothing changes, and the command fails. Only the client that sent the command gets the error. The error carries a stablecode and a reason that you can show to
your user. The model never refuses list_voices, end_call, or get_state.
invalid_input: a blankname,persona, ortext, or an id that is not acharacter_id.invalid_image: the model cannot read the image, or the image is over 20 MiB or 40 megapixels.invalid_voice: the voice is not inlist_voices.text_too_long: the text is over 800 bytes.busy: a call is starting or running, or the model is creating a character.no_character:start_callwhen no character is current.not_live:saywhile no call is listening (speech_readyis false).send_failed: the model could not deliver the text. Retryable.content_policy: the content policy declined the image or text.upstream_rejected: the generation service refused the image or text.
create_character, attach_character, and say can also fail with three upstream codes.
upstream_unavailable means that the model could not reach the generation service. A retry may
work. upstream_auth and upstream_protocol mean a deployment problem on the Reactor side of the
contract. Report them with trace_id.
In JavaScript, a refusal fires the error event with operation: "sendCommand" and the code. Then
the awaited command resolves undefined. A command that declares a reply resolves undefined only
when the model refuses it. start_call declares no reply, so it also resolves undefined on
success. For start_call, only the error event shows a refusal. In Python, send_command()
raises a ReactorError with the code.
Failures
A failure that no command caused is not a refusal. It ends the call, sets the phase tofailed, and
sets last_error on session_state. last_error has these fields:
Messages
transcript payload
session_state payload
end_reason is one of seven values. ended_by_client (end_call), idle, and max_duration end
in ended. start_failed (the call never went live), media_lost, upstream_interrupted, and
platform_failed end in failed with last_error.
Complete example
This example creates a character from an image and starts a call. Whenspeech_ready turns true,
it says one line to the character. It also prints the subtitles. The
tutorial adds the microphone, subtitle
handling, and character reuse.