> ## Documentation Index > Fetch the complete documentation index at: https://docs.reactor.inc/llms.txt > Use this file to discover all available pages before exploring further. > ## Agent Instructions > To build and serve your own model, start at /deploy/development/quickstart and /deploy/development/overview. Deploying is the default path: reactor init scaffolds a workspace, reactor auth login authenticates, and reactor model deploy registers the model, publishes the release with the weights/ folder, and activates it on Reactor's GPUs, in one command from that workspace. Docker must be running, because the publish step builds the image locally. Bump model.version in reactor.yaml before redeploying a change, because a release that already has an image is reactivated as it is. Deployment access is granted per account, so contact team@reactor.inc if a deploy is refused. Every key in reactor.yaml is documented at /deploy/platform/reactor-yaml. Model code imports reactor_runtime; Python client code imports reactor_sdk. The runtime overview explains the model interface. Running the model on your own machine with reactor run is optional and needs a GPU you attach with --gpus; /deploy/development/local-testing covers that loop and pairs a complete brightness model with a Python client test in a separate brightness-test workspace. > Reactor hosts multiple models, each with its own connect slug (modelName) and command/event schema. The catalog of every model — slug, typed SDK package, and links to its schema — is at /model-api-reference/overview. Some models expose one slug per experience (e.g. HappyOyster); always take the slug from the model's own pages, never guess it. > Fastest path to a working app: `npx create-reactor-app my-app --model=` scaffolds a complete app with secure auth wired up. Typed TypeScript SDKs are published as @reactor-models/; Python uses the base reactor-sdk package. > Auth: exchange an API key (rk_...) for a JWT via POST https://api.reactor.inc/tokens from your server. Never put the API key in client-side code. > Append .md to any docs URL for clean Markdown. Search these docs via the MCP server at https://docs.reactor.inc/mcp. # Vidu S2-Avatar schema > Tracks, session phases, commands, replies, and messages for Vidu S2-Avatar. Use these commands to create or reuse an avatar, start a call, and change the character during the conversation. ## Tracks | Track | Direction | Type | Description | | - | - | - | - | | `mic` | Input | Audio | Caller speech during a live call. | | `webcam` | Input | Video | Caller camera video during a video call. | | `main_video` | Output | Video | Generated character video. | | `main_audio` | Output | Audio | Generated character speech. | All four tracks are available when the session connects, so you can set up a player before a call. Character video and speech begin when the call is live. See [Tracks](/concepts/tracks) for how to publish `mic` and `webcam`. ## Session lifecycle A session has one current avatar. You can start, end, and restart calls with it without reconnecting. Ending a call leaves the session open. Reactor sends a `session_state` message when you connect and whenever the avatar or call changes. Its `phase` field reports the current step: | Phase | Meaning | | - | - | | `idle` | No avatar has been created or attached in this session. | | `preparing_avatar` | The model is creating an avatar from your image. | | `avatar_ready` | The avatar is ready for a call. | | `starting` | The call is being created. | | `warming_up` | The call is getting ready. | | `live` | The call is active. You can speak, type, or steer it. | | `ending` | The call is closing. | | `ended` | The call is over. You can start another with this avatar. | | `failed` | The call failed. Check `last_error` before trying again. | If avatar creation fails, the phase returns to `idle`. If a call fails, the phase becomes `failed`. Two limits can end a call without `end_call`. `call_max_seconds` is the maximum call length; its timer begins when the call starts. Reaching it sets `end_reason` to `max_duration`. Two hours without input sets `end_reason` to `idle_timeout`. ## Commands Send commands with `reactor.sendCommand()` in JavaScript or `reactor.send_command()` in Python. A command with a named reply returns that message. Other commands return no value; watch `session_state` for their result. A refused command sends `command_error` without changing the session. See [Command errors](#command_error) for the codes. | Command | Description | Valid in | Reply | | - | - | - | - | | `create_avatar` | Create an avatar from one image. | No call active, no avatar preparing | None; see `session_state` | | `attach_avatar` | Use an avatar saved from an earlier session. | No call active | None; see `session_state` | | `list_voices` | List the voices a call can use. | Any time | `voices` | | `start_call` | Start a call with the current avatar. | `avatar_ready`, `ended`, `failed` | None; see `session_state` | | `say` | Send text for the character to answer. | `live` | None | | `interrupt` | Stop the character mid-sentence. | `live` | None | | `update_call` | Change voice, persona, turn-taking, or reply settings. | `live` | `call_updated` | | `set_reference_images` | Give the character an object, an outfit, or a background. | `live` | `reference_images_applied` | | `clear_reference_images` | Remove active reference images. | `live` | `reference_images_applied` | | `end_call` | End the call and keep the avatar. | `starting` through `live` | `call_ended` | | `get_state` | Get the current session state. | Any time | `session_state` | Omit an optional field you do not set. Do not send it as `null`. A command that carries an explicit `null` is dropped whole, without a `command_error`, so send `{ persona }`, not `{ persona, voice: null }`. ### `create_avatar` Create an avatar from one image of a person. The new avatar becomes the current avatar for this session. Give exactly one of `image` or `image_url`. | Parameter | Type | Default | Description | | - | - | - | - | | `image` | Upload reference or null | `null` | The character's image, uploaded with the SDK's [file upload](/concepts/file-uploads). Use this or `image_url`. | | `image_url` | string or null | `null` | A publicly fetchable URL of the image, instead of an upload. | | `name` | string or null | `null` | Display name, up to 100 characters. Reported back as `session_state.avatar_name`. | An uploaded image can be PNG, JPG, WebP, or HEIC, under 20 MB and at most 50 megapixels. The model applies the image's EXIF orientation, so a phone photo arrives upright. While the image is processed, `session_state.phase` is `preparing_avatar`. When the avatar is ready, the phase becomes `avatar_ready` and `session_state` includes its `avatar_id`. Save the ID to use the avatar in another session. ```typescript JavaScript theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}} const image = await reactor.uploadFile(file); await reactor.sendCommand("create_avatar", { image, name: "Tina" }); ``` ```python Python theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}} ref = await reactor.upload_file("tina.png") await reactor.send_command("create_avatar", {"image": ref, "name": "Tina"}) ``` ### `attach_avatar` Use an avatar you created earlier by passing its `avatar_id`. Avatars are kept for 90 days. | Parameter | Type | Description | | - | - | - | | `avatar_id` | string | Required. An `avatar_id` from an earlier `session_state`. | The model checks the ID before using the avatar. On success, `session_state.phase` becomes `avatar_ready`. ### `list_voices` Send `{}` at any time. The `voices` reply lists built-in voices and names the default voice. Each `system` entry has `voice`, `description`, and `accent`. Pass an entry's `voice` to `start_call` or `update_call`. ```typescript JavaScript theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}} const reply = await reactor.sendCommand("list_voices", {}); const voice = reply?.data.system[0].voice; ``` ```python Python theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}} reply = await reactor.send_command("list_voices", {}) voice = reply["data"]["system"][0]["voice"] ``` ### `start_call` Start a live call with the current avatar. Create or attach an avatar first, then wait for `session_state.phase` to become `avatar_ready`. Only `persona` is required. Who the character is and how it behaves, up to 50,000 characters. A `voice` from `list_voices`. Omit it to use the model's default voice. What the character says or does first, before you speak, up to 200 characters. Language the character speaks and replies in, up to 40 characters. `audio` forwards your `mic` track. `video` also forwards your `webcam` track. This setting is fixed for the call. Send a `transcript` message for what the caller and character say. Expand a short persona into a fuller description before the call starts. Use this when you have only a brief character description. Turn-taking settings. You can change them later with `update_call`. Reply generation settings. You can change them later with `update_call`. `call_mode` controls whether your camera reaches the character. The character's generated video still arrives on `main_video` in both modes. #### Turn-taking: `vad` VAD means voice activity detection. These settings control when caller speech starts or ends a turn. The default `server` mode filters brief acknowledgments and background noise. Use `semantic` when the caller should be able to interrupt the character as soon as they begin speaking. `server` filters back-channel sounds and background noise. `semantic` lets the caller interrupt the character as soon as they start speaking. With `server`, controls how much noise is filtered, from 0 to 1. Raise it in a noisy room; lower it if quiet speech is missed. With `server`, how long the caller must be silent before the character answers, from 200 to 6000 milliseconds. Raise it to allow longer pauses; lower it for faster replies. See the [turn-taking examples](/model-api-reference/vidu-s2-avatar/prompt-guide#tune-turn-taking) for common settings. #### Reply generation: `llm` These settings shape each reply. A token is a piece of text. You can change these settings during a call with `update_call`; the new values apply on the next turn. The maximum length of one reply in tokens. Raise it for longer answers; lower it for brief exchanges. Controls variation, from 0 up to but not including 2. Lower values make replies more consistent; higher values make them more varied. Nucleus sampling, greater than 0 and at most 1. Lower values narrow the next-token choices, making replies less varied. Top-k sampling, from 0 to 100. A lower positive value limits how many next-token choices the model considers. Discourages repeated words and phrases. Raise it when replies repeat themselves; the value must be greater than 0. Encourages new topics instead of returning to ones already covered. Raise it for more topic variety, from 0 to 2. Controls randomness. Use a fixed value to make a demo more repeatable; `-1` picks a random seed. The phase moves through `starting` and `warming_up` to `live`. Once live, `main_video` and `main_audio` carry the character, and your `mic` reaches it. If the call fails, the phase becomes `failed`; inspect `command_error` or `session_state.last_error` for the reason. ```typescript JavaScript theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}} await reactor.sendCommand("start_call", { persona: "You are Tina, a warm museum guide. Answer in two short sentences.", greeting: "Say hello and wave.", call_mode: "audio", vad: { type: "server", silence_duration_ms: 800 }, llm: { max_tokens: 80 }, }); ``` ```python Python theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}} await reactor.send_command("start_call", { "persona": "You are Tina, a warm museum guide. Answer in two short sentences.", "greeting": "Say hello and wave.", "call_mode": "audio", "vad": {"type": "server", "silence_duration_ms": 800}, "llm": {"max_tokens": 80}, }) ``` ### `say` Send text for the character to answer, as if you had spoken it. | Parameter | Type | Description | | - | - | - | | `text` | string | Required. What to tell the character, 1 to 2,000 characters. | There is no reply. The answer arrives on `main_video` and `main_audio`, and as a `transcript` when transcripts are on. ```typescript JavaScript theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}} await reactor.sendCommand("say", { text: "Tell me about the sea." }); ``` ```python Python theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}} await reactor.send_command("say", {"text": "Tell me about the sea."}) ``` ### `interrupt` Send `{}` to stop the character mid-sentence, as speaking over it would. There is no reply. To stop it and give a new instruction, send `interrupt` followed by `say`; `interrupt` takes no text. ```typescript JavaScript theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}} await reactor.sendCommand("interrupt", {}); await reactor.sendCommand("say", { text: "Let's talk about the sea instead.", }); ``` ```python Python theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}} await reactor.send_command("interrupt", {}) await reactor.send_command("say", {"text": "Let's talk about the sea instead."}) ``` ### `update_call` Change the live call without restarting it. Give at least one field. | Parameter | Type | Description | | - | - | - | | `voice` | string | A `voice` from `list_voices`. Takes effect after the current sentence. | | `persona` | string | A new persona, up to 50,000 characters. Replaces the current one after the current sentence. | | `vad` | object | Turn-taking settings, as in `start_call`. Applied on the next turn. | | `llm` | object | Reply settings, as in `start_call`. Applied on the next turn. | The model applies all supplied fields in one update. If it rejects any field, none of the changes take effect. The `call_updated` reply lists the settings that changed. ```typescript JavaScript theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}} const reply = await reactor.sendCommand("update_call", { voice: "Ethan" }); console.log(reply?.data.applied); // ["voice"] ``` ```python Python theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}} reply = await reactor.send_command("update_call", {"voice": "Ethan"}) print(reply["data"]["applied"]) # ["voice"] ``` ### `set_reference_images` Give the character one to three reference images to pick up live: an object to hold, an outfit to wear, or a background. | Parameter | Type | Description | | - | - | - | | `images` | array | Required. One to three reference image entries. | Each entry has these fields: A publicly fetchable image. Reference images are URL-only; uploads are not accepted here. Your own stable id for the image, 1 to 128 characters, unique in the call. Sending an id that is already in effect replaces that image. `object`, `garment`, or `background`. Inferred when omitted. One sentence, up to 200 characters, that describes the change. The change does not interrupt speech. The `reference_images_applied` reply lists every image ID now in effect. ```typescript JavaScript theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}} const reply = await reactor.sendCommand("set_reference_images", { images: [ { image_url: "https://example.com/jacket.png", image_id: "jacket-1", kind: "garment", text: "She puts on the denim jacket.", }, ], }); ``` ```python Python theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}} reply = await reactor.send_command("set_reference_images", { "images": [{ "image_url": "https://example.com/jacket.png", "image_id": "jacket-1", "kind": "garment", "text": "She puts on the denim jacket.", }], }) ``` ### `clear_reference_images` Remove reference images from the live call. Pass the `image_id` values you supplied with `set_reference_images`. Omit `image_ids` to undo the most recent `set_reference_images` command. | Parameter | Type | Default | Description | | - | - | - | - | | `image_ids` | string array or null | `null` | Up to three `image_id` values to clear. Omit to undo the most recent `set_reference_images`. | The `reference_images_applied` reply lists the IDs still in effect. This example removes the jacket from the preceding `set_reference_images` example: ```typescript JavaScript theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}} const reply = await reactor.sendCommand("clear_reference_images", { image_ids: ["jacket-1"], }); console.log(reply?.data.image_ids); ``` ```python Python theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}} reply = await reactor.send_command("clear_reference_images", { "image_ids": ["jacket-1"], }) print(reply["data"]["image_ids"]) ``` ### `end_call` Send `{}` to end the call. The `call_ended` reply arrives once the call is released. Its `duration_seconds` is the time the call was live, or `0` if it never became live. The phase moves through `ending` to `ended`. The avatar remains available, so `start_call` works again. ```typescript JavaScript theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}} const ended = await reactor.sendCommand("end_call", {}); console.log(ended?.data.end_reason); ``` ```python Python theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}} ended = await reactor.send_command("end_call", {}) print(ended["data"]["end_reason"]) ``` ### `get_state` Send `{}` at any time to get the current `session_state`. Reactor also broadcasts this message when the avatar or call changes. ```typescript JavaScript theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}} const state = await reactor.sendCommand("get_state", {}); console.log(state?.data.phase, state?.data.avatar_id); ``` ```python Python theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}} state = await reactor.send_command("get_state", {}) print(state["data"]["phase"], state["data"]["avatar_id"]) ``` ## Messages Every message is broadcast to all connected clients, except command replies, which go only to the client that sent the command. | Message | When | Delivery | Payload | | - | - | - | - | | `session_state` | On connect, on every change, and in reply to `get_state`. | Event or command reply | See [`session_state`](#session_state). | | `transcript` | You or the character finished a sentence, with transcripts on. | Event | `speaker`, `text`, `final` | | `voices` | Reply to `list_voices`. | Command reply | `system`, `default_voice` | | `call_updated` | Reply to `update_call`, once the call took the new settings. | Command reply | `applied` | | `reference_images_applied` | Reply to `set_reference_images` and `clear_reference_images`. | Command reply | `image_ids` | | `call_ended` | Reply to `end_call`, once the call is released. | Command reply | `end_reason`, `duration_seconds` | | `command_error` | A command was refused, or a call failed. | Error | See [`command_error`](#command_error). | For `transcript`, `speaker` is `user` or `character`, and `final` is `true` when the text is settled. Listen for messages before connecting so you receive the first `session_state`: ```typescript JavaScript theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}} reactor.on("message", (message) => { switch (message.type) { case "session_state": console.log("Phase:", message.data.phase); break; case "transcript": if (message.data.final) { console.log(`${message.data.speaker}: ${message.data.text}`); } break; case "command_error": console.error(message.data.code, message.data.reason); break; } }); ``` ```python Python theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}} def on_message(message): data = message["data"] if message["type"] == "session_state": print("Phase:", data["phase"]) elif message["type"] == "transcript" and data["final"]: print(f'{data["speaker"]}: {data["text"]}') elif message["type"] == "command_error": print(data["code"], data["reason"]) reactor.on("message", on_message) ``` ### `session_state` The authoritative snapshot. Drive your UI from this message alone. A client that reconnects mid session gets a full snapshot on connect. Only `phase` is always present; the other fields are `null` or `false` when they do not apply. | Field | Type | Description | | - | - | - | | `phase` | string | The session phase. See [Session lifecycle](#session-lifecycle). | | `avatar_id` | string or null | The current avatar's id. Save it to reuse the avatar with `attach_avatar`. | | `avatar_status` | string or null | `processing`, `ready`, or `failed` for the current avatar. | | `avatar_name` | string or null | The avatar's display name. | | `voice` | string or null | The voice in effect for the current or next call. | | `persona_set` | boolean | A persona has been given for the current call. | | `call_mode` | string or null | `audio` or `video` for the current call. | | `warmup_attempts` | integer | Readiness checks made while `warming_up`. | | `call_started_at` | number or null | Unix time, in seconds, at which the call became `live`. | | `call_max_seconds` | integer or null | The longest the call may run before it ends on its own. | | `call_elapsed_seconds` | integer or null | Seconds since the call became `live`. | | `control_ready` | boolean | Commands such as `say` and `interrupt` are accepted. | | `video_receiving` | boolean | Character video is arriving on `main_video`. | | `audio_receiving` | boolean | Character speech is arriving on `main_audio`. | | `mic_forwarding` | boolean | Your `mic` track has reached the character in this call. | | `camera_forwarding` | boolean | Your `webcam` track has reached the character. Video mode only. | | `last_frame_age_ms` | integer or null | Milliseconds since the last character video frame. Grows when video stalls. | | `reference_images` | array or null | Reference images in effect, `[{ image_id, kind }]`. Null or empty when there are none. | | `end_reason` | string or null | Why the last call ended. Null while a call runs. See below. | | `last_error` | object or null | The most recent `command_error` payload. Cleared when a call starts. | `end_reason` is one of: | Value | Meaning | | - | - | | `ended_by_client` | A client sent `end_call`. | | `max_duration` | The call reached `call_max_seconds`. | | `idle_timeout` | The call had no input for two hours. | | `content_policy` | The call was stopped for its content. | | `upstream_quota` | The account's usage quota ran out. Not the user's fault. | | `upstream_interrupted` | The call was interrupted. Start a new call. | | `media_lost` | Character media stopped arriving. Start a new call. | | `bridge_failed` | The call failed on Reactor's side. Start a new call. | Any value other than `ended_by_client` also leaves a `last_error` that explains it. ### `command_error` Emitted when a command is refused or a call fails. A refused command does not perform the requested action; a call failure moves the phase to `failed`. The payload is also stored as `session_state.last_error`. | Field | Type | Description | | - | - | - | | `command` | string | The command that failed, or `session` when nothing the client sent caused it. | | `origin` | string | Whose fault it is. See below. | | `code` | string | A stable code. Branch on this. | | `reason` | string | A readable sentence. | | `retryable` | boolean | The same command may succeed if you send it again later. Defaults to `false`. | | `upstream_status` | integer or null | HTTP status from the generation service, when `origin` is `upstream`. | | `upstream_code` | string or null | Error code from the generation service, when it gave one. | | `trace_id` | string or null | Quote this when you report the failure. | `origin` is one of `request` (your input was invalid), `state` (the command is not allowed in this phase), `upstream` (the generation service refused or failed), or `platform` (a failure on Reactor's side). | Code | Meaning | | - | - | | `INVALID_INPUT` | A parameter is missing, malformed, or out of range. | | `BUSY` | A call is active, or an avatar is being prepared. | | `NO_AVATAR` | `start_call` was sent with no avatar ready. | | `NOT_LIVE` | The command needs a live call. | | `AVATAR_FAILED`, `AVATAR_TIMEOUT` | The image could not become an avatar. | | `AVATAR_NOT_FOUND` | The `avatar_id` is unknown or has expired. | | `WARMUP_TIMEOUT` | The call did not become live in time. | | `UPSTREAM_CAPACITY` | No capacity is free right now. Retryable. | | `UPSTREAM_QUOTA` | The account's usage quota ran out. | | `CONTENT_POLICY` | The request or the call was stopped for its content. | | `UPSTREAM_REJECTED`, `UPSTREAM_AUTH`, `UPSTREAM_UNAVAILABLE`, `UPSTREAM_DISCONNECTED` | The generation service refused or failed. | | `MEDIA_JOIN_FAILED`, `MEDIA_LOST`, `BRIDGE_CRASHED` | Media could not start or stopped. Start a new call. | The generation service can also return its own codes, which pass through unchanged. Examples are `LIVE_CONN_INIT_FAILED` and codes that start with `PROMPT_OP_`. ### Upload reference `create_avatar.image` takes a reference returned by the Reactor upload protocol. All fields are required. | Field | Type | | - | - | | `upload_id` | UUID string | | `name` | string | | `mime_type` | string | | `size` | integer | ## Multiple clients Several clients can join the same session. `session_state` and `command_error` go to all of them; a command reply goes only to the client that sent the command. A session has one call, so every client steers the same character. A client that disconnects does not end the call. Every new session starts in `idle`, without an avatar.