> ## Documentation Index > Fetch the complete documentation index at: https://docs.reactor.inc/llms.txt > Use this file to discover all available pages before exploring further. # LingBot schema reference > The complete LingBot track, command, and event schema, with an end-to-end example. This page documents the complete LingBot wire surface: the media tracks it produces, the session lifecycle, every command you can send, and the messages the model emits back. For what LingBot is and a quick start, see the [overview](/model-api-reference/lingbot/overview). ## Tracks | Direction | Name | Type | Format | Rate | | --------- | ------------ | ----- | ------------------------ | ----------------------------------- | | Outbound | `main_video` | Video | `(N, H, W, 3)` uint8 RGB | Adaptive, paced to model throughput | Default resolution is 1664 x 960. There are no inbound tracks; all client → model communication is through commands. ## Session lifecycle LingBot runtime states: WAITING, GENERATING, PAUSED, with start/pause/resume/reset transitions and the set_* commands that take effect in each state Once the connection reaches **ready**, the session begins in `WAITING`. `start` transitions to `GENERATING` (provided a prompt and seed image are set); `pause` moves to `PAUSED`; `resume` returns to `GENERATING`; `reset` clears state and returns to `WAITING` from any state. The [Commands](#commands) table below lists each command's preconditions and effects. See [Sessions](/concepts/sessions#connection-lifecycle) for the separate connection-level lifecycle (`disconnected → connecting → waiting → ready`) the session passes through before reaching the runtime states above. When all chunks of a run complete and the session is still `started`, the server automatically kicks off the next run with the same prompt and image. Call `reset` to stop the loop and re-stage with new conditions. ## Commands Send commands to the model using `reactor.sendCommand()`. Setter commands (`set_*`) take effect at the next chunk boundary. Below are all available commands: | Command | Description | | ------------------------ | ------------------------------------------------------------------- | | `set_prompt` | Set or swap the text prompt (works before and during generation) | | `set_image` | Set the seed image that anchors the world (required before `start`) | | `set_movement` | Set WASD movement direction | | `set_look_horizontal` | Set yaw (look left/right) direction | | `set_look_vertical` | Set pitch (look up/down) direction | | `set_rotation_speed_deg` | Set how fast the camera rotates when a look axis is active | | `set_seed` | Set the RNG seed for the next generation | | `start` | Begin generation (requires a prompt **and** a seed image) | | `pause` | Pause after the current chunk | | `resume` | Resume from a pause | | `reset` | Clear all session state and return to `WAITING` | ### set\_prompt Stores the prompt and re-applies it on the next chunk boundary. Works before `start` and mid-generation for live prompt swaps. Emits `prompt_accepted` then `conditions_ready`. **Parameters:** | Parameter | Type | Required | Description | | --------- | ------ | -------- | ------------------------- | | `prompt` | string | Yes | Prompt text, ≤ 1000 chars | ```typescript JavaScript theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}} await reactor.sendCommand("set_prompt", { prompt: "A misty old-growth forest, soft morning light filtering through the canopy.", }); ``` ```tsx React theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}} const { setPrompt } = useLingbot(); await setPrompt({ prompt: "A misty old-growth forest, soft morning light filtering through the canopy.", }); ``` ```python Python theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}} await reactor.send_command("set_prompt", { "prompt": "A misty old-growth forest, soft morning light filtering through the canopy.", }) ``` ### set\_image Decodes the uploaded file and caches it as the seed image. Upload the file first with `uploadFile()`, then pass the returned [`FileRef`](/sdk-reference/types#fileref) as `image`. Rejects non-image MIME types with `command_error`. Emits `image_accepted` then `conditions_ready`. **Parameters:** | Parameter | Type | Required | Description | | --------- | ------- | -------- | ------------------------------------------------------------ | | `image` | FileRef | Yes | A reference to an uploaded image, returned by `uploadFile()` | ```typescript JavaScript theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}} const ref = await reactor.uploadFile(seedImageFile); await reactor.sendCommand("set_image", { image: ref }); ``` ```tsx React theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}} const { uploadFile, setImage } = useLingbot(); const ref = await uploadFile(seedImageFile); await setImage({ image: ref }); ``` ```python Python theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}} ref = await reactor.upload_file("seed.png") await reactor.send_command("set_image", {"image": ref}) ``` ### set\_movement / set\_look\_horizontal / set\_look\_vertical The three drive setters. Movement and look are **persistent state, not pulses**: a value holds across chunks until you send a new one (including `"idle"`). To stop, send `"idle"` explicitly. All apply at the next chunk boundary. **Parameters:** | Command | Value | Default | | --------------------- | ------------------------------------------------------------------ | -------- | | `set_movement` | `"idle" \| "forward" \| "back" \| "strafe_left" \| "strafe_right"` | `"idle"` | | `set_look_horizontal` | `"idle" \| "left" \| "right"` | `"idle"` | | `set_look_vertical` | `"idle" \| "up" \| "down"` | `"idle"` | ```typescript JavaScript theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}} // Start walking forward and panning left await reactor.sendCommand("set_movement", { movement: "forward" }); await reactor.sendCommand("set_look_horizontal", { look_horizontal: "left" }); // Stop moving (look continues until you idle it too) await reactor.sendCommand("set_movement", { movement: "idle" }); ``` ```tsx React theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}} const { setMovement, setLookHorizontal } = useLingbot(); await setMovement({ movement: "forward" }); await setLookHorizontal({ look_horizontal: "left" }); ``` ```python Python theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}} await reactor.send_command("set_movement", {"movement": "forward"}) await reactor.send_command("set_look_horizontal", {"look_horizontal": "left"}) ``` ### set\_rotation\_speed\_deg Degrees the camera rotates per latent frame when a look axis is active. Setting it to `0` disables rotation entirely even if a look axis is non-idle. **Parameters:** | Parameter | Type | Required | Description | | -------------------- | ----- | -------- | ----------------------- | | `rotation_speed_deg` | float | Yes | 0.0 – 30.0. Default 5.0 | ```typescript JavaScript theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}} await reactor.sendCommand("set_rotation_speed_deg", { rotation_speed_deg: 10.0 }); ``` ```tsx React theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}} const { setRotationSpeedDeg } = useLingbot(); await setRotationSpeedDeg({ rotation_speed_deg: 10.0 }); ``` ```python Python theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}} await reactor.send_command("set_rotation_speed_deg", {"rotation_speed_deg": 10.0}) ``` ### set\_seed RNG seed for the next generation. Read once when `start` fires; later changes require `reset` then a new `start`. **Parameters:** | Parameter | Type | Required | Description | | --------- | ---- | -------- | ---------------------------------------- | | `seed` | int | Yes | ≥ 0. Same seed → same output. Default 42 | ```typescript JavaScript theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}} await reactor.sendCommand("set_seed", { seed: 42 }); ``` ```tsx React theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}} const { setSeed } = useLingbot(); await setSeed({ seed: 42 }); ``` ```python Python theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}} await reactor.send_command("set_seed", {"seed": 42}) ``` ### start Begins generation. Requires both a prompt **and** a seed image to be set; fails with `command_error` otherwise. Emits `generation_started`. ### pause / resume `pause` halts after the current chunk finishes (`generation_paused`) and is valid only while `GENERATING`; `resume` continues from the next chunk (`generation_resumed`) and is valid only while `PAUSED`. ### reset Clears the prompt, image, and the `started` flag, returning to `WAITING` from any state. Emits `generation_reset`. ```typescript JavaScript theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}} await reactor.sendCommand("start", {}); await reactor.sendCommand("pause", {}); await reactor.sendCommand("resume", {}); await reactor.sendCommand("reset", {}); ``` ```tsx React theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}} const { start, pause, resume, reset } = useLingbot(); ``` ```python Python theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}} await reactor.send_command("start", {}) await reactor.send_command("pause", {}) await reactor.send_command("resume", {}) await reactor.send_command("reset", {}) ``` ## Messages LingBot emits the following messages. Every message is delivered as JSON `{ "type": "", "data": { … } }`. | Event | When | Payload | | --------------------- | ----------------------------------------------------------------- | ----------------------------------------------------------------------------------------- | | `prompt_accepted` | After `set_prompt` | `{ prompt: string }` | | `image_accepted` | After `set_image` decodes | `{ width: int, height: int }` | | `conditions_ready` | After `set_prompt` or `set_image` | `{ has_prompt: bool, has_image: bool }` | | `generation_started` | After `start` | `{ prompt: string, chunk_num: int, frame_num: int }` | | `chunk_complete` | After each chunk emits | `{ chunk_index: int, frames_emitted: int, active_prompt: string, active_action: string }` | | `generation_paused` | After `pause` | `{ chunk_index: int }` | | `generation_resumed` | After `resume` | `{ chunk_index: int }` | | `generation_complete` | After all chunks of a run finish | `{ total_chunks: int }` | | `generation_reset` | After `reset` | `{ reason: string }` | | `command_error` | When a command is rejected | `{ command: string, reason: string }` | | `state` | On connect, after every command, and after every `chunk_complete` | Full session snapshot | ### `state` payload `state` is the single source of truth for driving UI. Subscribe once and treat it as the authoritative session snapshot; you generally do not need to track individual `set_*` and `chunk_complete` events yourself. | Field | Type | Meaning | | -------------------- | ---------------- | ------------------------------------------------------------------------------------------------------------------------- | | `running` | `bool` | `started && !paused`. Equivalent to "frames are actively streaming" | | `started` | `bool` | True once `start` has been accepted; remains true through pauses and across auto-restarts. Flips back to false on `reset` | | `paused` | `bool` | True while in the `PAUSED` state | | `current_chunk` | `int` | Zero-based index of the last completed chunk; `0` before the first chunk and after `reset` | | `current_prompt` | `string \| null` | The prompt currently driving generation, or `null` if none set | | `has_prompt` | `bool` | Whether a prompt has been set this session | | `has_image` | `bool` | Whether a seed image has been set this session | | `current_action` | `string` | `+`-joined composite of movement and look. E.g. `"w+left"`, or `"still"` when idle | | `movement` | `string` | Current value of the `movement` input field | | `look_horizontal` | `string` | Current value of the `look_horizontal` input field | | `look_vertical` | `string` | Current value of the `look_vertical` input field | | `rotation_speed_deg` | `float` | Current rotation speed (0.0 – 30.0) | | `seed` | `int` | Current seed value (effective only on the next `start`) | ## Complete example Stage a seed image and prompt, wait for the image to decode, start, then drive the camera. ```typescript JavaScript theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}} import { LingbotModel } from "@reactor-models/lingbot"; const lingbot = new LingbotModel(); lingbot.onChunkComplete(({ chunk_index, active_action }) => { console.log(`chunk ${chunk_index}: ${active_action}`); }); lingbot.onCommandError(({ command, reason }) => { console.error(`${command} rejected: ${reason}`); }); await lingbot.connect(jwt); // Wait for the seed image to decode so the first chunk renders from it. const imageAccepted = new Promise((resolve) => lingbot.onImageAccepted(() => resolve())); const ref = await lingbot.uploadFile(seedImageFile); await lingbot.setImage({ image: ref }); await imageAccepted; await lingbot.setPrompt({ prompt: "A misty old-growth forest, soft morning light filtering through the canopy.", }); await lingbot.start(); // Drive the camera (persistent state, holds until you change it). await lingbot.setMovement({ movement: "forward" }); await lingbot.setLookHorizontal({ look_horizontal: "left" }); ``` ```python Python theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}} import asyncio, os from reactor_sdk import Reactor async def main(): reactor = Reactor(model_name="lingbot", api_key=os.environ["REACTOR_API_KEY"]) image_accepted = asyncio.Event() @reactor.on_message("image_accepted") def _(msg): image_accepted.set() await reactor.connect() ref = await reactor.upload_file("seed.png") await reactor.send_command("set_image", {"image": ref}) await image_accepted.wait() await reactor.send_command("set_prompt", { "prompt": "A misty old-growth forest, soft morning light filtering through the canopy.", }) await reactor.send_command("start", {}) # Drive the camera. await reactor.send_command("set_movement", {"movement": "forward"}) await asyncio.Event().wait() asyncio.run(main()) ```