Skip to main content
This walkthrough uses the base JavaScript SDK so that you can see each model command. It publishes a microphone, creates a character from one image, and starts a call. Then it talks to the character by voice and by text, and shows the answers as subtitles. The snippets build on each other in one browser module. Your app supplies three things that this page does not show:
  • getToken(), which asks your server for a session token. The Authentication section shows the server side.
  • portraitFile, which is a File object from a file picker.
  • A page with one <video> element, one <audio> element, and a <p id="subtitle"> element.

Installation and setup

Install the base JavaScript SDK:
To start from a complete example app instead, run npx create-reactor-app my-persona-app --model=pixverse-r2-persona.

How PixVerse R2 Persona works

A session holds one character and one call at a time. create_character or attach_character sets the character. start_call opens a call. Then session_state moves through starting and connecting to live. During the call, the character speaks on main_video and main_audio. While speech_ready is true, the character hears your mic track and your say commands. A command reply tells you only that the model accepted the request. It does not tell you that the character is live or that it answered. For that, wait for the messages that follow, such as session_state with speech_ready and the transcript lines. The session_state snapshot always agrees with these messages, so keep a copy of it and draw your UI from that copy.

Authentication

Create the session token on your server so that your rk_ API key never reaches the browser. This route exchanges the key for a short-lived token that can only start reactor/pixverse-r2-persona sessions. The Python SDK exchanges the key for you, so it needs no route.
See Authentication for token lifetimes and for how to keep a token fresh for a whole session.

1. Connect and show the tracks

Connect to reactor/pixverse-r2-persona. Attach main_video to the video element and main_audio to the audio element. Before a call, main_video streams black frames in the aspect ratio that you set, and main_audio is silent.
Connect from a user action, such as a button click. Because of the user action, the browser lets the page play audio and lets you open the microphone. Before you send commands, wait for the ready status. When you connect, the model sends a session_state with phase idle.

2. Publish the microphone

Publish mic before start_call, so that the first thing you say reaches the character. If a published microphone sends nothing, the character hears nothing. If the user declines the microphone, the call still works with say.
session_state.mic_forwarding turns true after your microphone reaches the character during a call. Use it to show a “listening” indicator.

3. Create a character

Read the voices, upload the image, and create the character. The reply is character_ready with character_id. Save that id, because step 6 uses it again. The helper below waits for a phase. The later steps use it too:
The model creates the character in 2 to 4 seconds. If the model refuses the request, the command resolves undefined, and the error event carries the code, such as invalid_image or content_policy.

4. Start the call

Send start_call. The phase moves to starting, then to connecting. When the first frame of the character arrives, the phase moves to live. When capacity is free, this takes 8 to 10 seconds. speech_ready turns true during connecting, so the character can listen before its video shows.
start_call declares no reply, so the command resolves undefined both when the model accepts it and when the model refuses it. A refusal, such as busy or no_character, arrives on the error event. While the call waits for capacity, session_state.wait_position gives the place in line.

5. Talk and show subtitles

Each sentence that the character speaks arrives as a transcript line. The next line with the same turn_id and sentence_index replaces a line with final false. Use those two fields as the key for your subtitles:
Now speak into the microphone, or send a line of text. say works while speech_ready is true:
The character does not speak first. To open the call, wait until speech_ready and video_receiving are both true. Then immediately send one short line, such as “Hello!”. The character answers that line, so the user hears a greeting instead of silence. The said reply carries a prompt_id. The question_received, transcript, and input_declined messages for that turn carry the same prompt_id. A turn from the microphone has a null prompt_id. question_received arrives about 0.3 seconds after say. The first transcript line arrives about 5 seconds after say, when the character starts to speak. The character also sends character_action, which describes a gesture in words, such as a wave. You can show it or ignore it.

6. End the call, and call again

Send end_call to end the call. The reply is call_ended with end_reason and duration_seconds. The character stays current, so start_call starts another call with it.
A call can also end without end_call. It ends after 300 seconds with nothing said (idle_seconds), at 1,800 seconds (call_max_seconds), or on a failure. Watch session_state to see when a call ends, and offer “Call again”:
In a new session, send attach_character with the saved id instead of uploading the image again. The reply is character_ready at once. The model checks the id when the next call starts. If the id is unknown, that call fails with character_not_found, and the model clears character_id:

Surfacing refusals

When the model refuses a command, only that command fails, and only your client gets the refusal. Read the refusal from the error event whose operation is "sendCommand". Show it next to the control that sent the command. Do not show it as a connection problem, because the session is still connected. A call failure is different. The model accepted start_call, but the call then ended on an error. The phase becomes failed, and session_state.last_error carries the code, a reason, and whether a retry may work.
The schema lists every code.