> ## Documentation Index
> Fetch the complete documentation index at: https://docs.reactor.inc/llms.txt
> Use this file to discover all available pages before exploring further.

# X2 Schema Reference

> The complete X2 track, command, and event schema, with an end-to-end example.

This page documents the complete X2 wire surface: the media tracks it consumes and produces, the
session lifecycle, every command you can send, and the messages the model emits back. For what X2 is
and a quick start, see the [overview](/model-api-reference/x2/overview).

## Tracks

| Direction | Name         | Type  | Description                                    |
| --------- | ------------ | ----- | ---------------------------------------------- |
| Inbound   | `source`     | Video | The stream to edit, published by the client    |
| Outbound  | `main_video` | Video | The re-rendered stream, published by the model |

Unlike the generation-only models, X2 takes an inbound track: the client publishes the video to be
edited on `source`, and the model streams the transformed result back on `main_video`. The model's
native source pacing is 24 fps, and the output plays at 24 fps with no frame interpolation and no
upscaling. The model chooses the output resolution (a bucket around 832p) once at the first
generation, from the source stream's aspect ratio. It holds for the whole session and arrives in
`generation_started` and in every `state_update`.

<Tip>
  A still image works as a source too: draw it to a canvas and capture it with
  `canvas.captureStream(24)`, repainting at the stream rate so the capturer keeps emitting frames.
  From the model's side that is indistinguishable from a video of a motionless scene, which is the
  drag-to-animate setup. The [tutorial](/model-api-reference/x2/tutorial#streaming-a-source) shows
  the pattern.
</Tip>

## Session lifecycle

There is no `start` command and no staged or started state machine to drive. Generation begins on
its own once two things are true: a non-empty prompt is set, and source frames are arriving. It runs
until the session ends, a new reference image forces an automatic restart, or you call `reset`.
Reset stops generation and clears the prompt, reference image, and pointer. See
[Sessions](/concepts/sessions#connection-lifecycle) for the connection-level lifecycle
(`disconnected → connecting → waiting → ready`) the session passes through first.

Every control follows block semantics: the model generates one block at a time, a prompt change
applies from the next block, it samples the pointer once per block, and the backlog policy switches
at the next block.

## Commands

Send commands with `reactor.sendCommand()` on the base SDK, or the typed methods on the generated
model client / `useX2()` hook. Below are all available commands:

| Command               | Description                                                         |
| --------------------- | ------------------------------------------------------------------- |
| `set_prompt`          | Set or hot-swap the editing instruction (required for generation)   |
| `set_reference_image` | Set the character/object image that anchors swaps and insertions    |
| `set_pointer`         | Update the drag pointer: position and press state together          |
| `set_pointer_x`       | Update only the pointer's horizontal position                       |
| `set_pointer_y`       | Update only the pointer's vertical position                         |
| `set_pointer_active`  | Update only whether a drag is in progress                           |
| `set_keep_backlog`    | Set the source-frame policy between bounded latency and every frame |
| `reset`               | Stop generation and clear the prompt, reference image, and pointer  |

### `set_prompt`

Set the editing instruction that guides the re-render (for example a character swap or insertion).
Generation needs a non-empty prompt before it begins; setting one while generating applies from the
next block. Emits `prompt_accepted`, then `state_update`.

**Parameters:**

| Parameter | Type   | Required | Description                                                          |
| --------- | ------ | -------- | -------------------------------------------------------------------- |
| `prompt`  | string | No       | The editing instruction. Up to 1000 characters. Default `""` (empty) |

<CodeGroup>
  ```typescript JavaScript theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}}
  await reactor.sendCommand("set_prompt", {
    prompt: "replace the character in the video with the character from the reference image",
  });
  ```

  ```tsx React theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}}
  const { setPrompt } = useX2();
  await setPrompt({
    prompt: "replace the character in the video with the character from the reference image",
  });
  ```

  ```python Python theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}}
  await reactor.send_command("set_prompt", {
      "prompt": "replace the character in the video with the character from the reference image",
  })
  ```
</CodeGroup>

### `set_reference_image`

Provide a reference image of a character or object to insert or swap into the video. Upload the file
first with `uploadFile()`, then pass the returned [`FileRef`](/sdk-reference/types#fileref). Emits
`reference_image_accepted` and `state_update` on success, or `command_error` if the file cannot be
decoded as an image.

The reference can be set before generation or replaced while generating. A mid-run replacement
restarts the stream on its own (`generation_stopped` with reason `reference_image_changed`, then a
fresh `generation_started`) and the new image conditions the edit from its first block. The prompt
stays set across the swap, so the edit resumes without a re-arm; no `reset` is needed.

**Parameters:**

| Parameter         | Type    | Required | Description                                                  |
| ----------------- | ------- | -------- | ------------------------------------------------------------ |
| `reference_image` | FileRef | Yes      | A reference to an uploaded image, returned by `uploadFile()` |

<CodeGroup>
  ```typescript JavaScript theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}}
  const ref = await reactor.uploadFile(imageFile);
  await reactor.sendCommand("set_reference_image", { reference_image: ref });
  ```

  ```tsx React theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}}
  const { uploadFile, setReferenceImage } = useX2();
  const ref = await uploadFile(imageFile);
  await setReferenceImage({ reference_image: ref });
  ```

  ```python Python theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}}
  ref = await reactor.upload_file("character.jpg")
  await reactor.send_command("set_reference_image", {"reference_image": ref})
  ```
</CodeGroup>

### `set_pointer`

Update the drag pointer that steers the edited subject's motion, setting its position and press
state together. Valid at any time; the model samples the pointer once per generated block, and it
only has an effect while `active` is true. Coordinates map to the output frame, so account for any
letterboxing your player introduces when you map pointer events. Emits `pointer_changed`, then
`state_update`.

**Parameters:**

| Parameter | Type    | Required | Description                                                         |
| --------- | ------- | -------- | ------------------------------------------------------------------- |
| `x`       | number  | No       | Horizontal position, `0` (left) to `1` (right). Default `0.5`       |
| `y`       | number  | No       | Vertical position, `0` (top) to `1` (bottom). Default `0.5`         |
| `active`  | boolean | No       | Whether a drag is in progress; ignored while false. Default `false` |

<CodeGroup>
  ```typescript JavaScript theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}}
  // Drag in progress at the frame's center-right.
  await reactor.sendCommand("set_pointer", { x: 0.8, y: 0.5, active: true });

  // Release: the pointer stops steering.
  await reactor.sendCommand("set_pointer", { x: 0.8, y: 0.5, active: false });
  ```

  ```tsx React theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}}
  const { setPointer } = useX2();
  await setPointer({ x: 0.8, y: 0.5, active: true });
  await setPointer({ active: false });
  ```

  ```python Python theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}}
  await reactor.send_command("set_pointer", {"x": 0.8, "y": 0.5, "active": True})
  await reactor.send_command("set_pointer", {"active": False})
  ```
</CodeGroup>

<Tip>
  Because the model samples the pointer once per block, sending positions faster than about 30 Hz
  buys nothing. Throttle pointer-move events, and use a trailing send so the final position of a
  fast gesture still lands. Always send `active: false` on release (and on unmount) so the model
  does not keep steering toward a stale point.
</Tip>

### `set_pointer_x`, `set_pointer_y`, `set_pointer_active`

Single-field variants of `set_pointer`, for updating one pointer field at a time. Same semantics:
sampled once per block, effective only while the pointer is active. Prefer `set_pointer` when you
have all three values; the position and press state then change in one step.

**Parameters (one per command):**

| Command              | Parameter        | Type    | Description                                    |
| -------------------- | ---------------- | ------- | ---------------------------------------------- |
| `set_pointer_x`      | `pointer_x`      | number  | Horizontal position, `0` to `1`. Default `0.5` |
| `set_pointer_y`      | `pointer_y`      | number  | Vertical position, `0` to `1`. Default `0.5`   |
| `set_pointer_active` | `pointer_active` | boolean | Whether a drag is in progress. Default `false` |

<CodeGroup>
  ```typescript JavaScript theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}}
  await reactor.sendCommand("set_pointer_x", { pointer_x: 0.25 });
  await reactor.sendCommand("set_pointer_active", { pointer_active: true });
  ```

  ```tsx React theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}}
  const { setPointerX, setPointerActive } = useX2();
  await setPointerX({ pointer_x: 0.25 });
  await setPointerActive({ pointer_active: true });
  ```

  ```python Python theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}}
  await reactor.send_command("set_pointer_x", {"pointer_x": 0.25})
  await reactor.send_command("set_pointer_active", {"pointer_active": True})
  ```
</CodeGroup>

### `set_keep_backlog`

Pick the source-frame consumption policy, applied from the next block:

* `false` (default): newest frames. The model always reads the most recent source frames and drops
  any backlog, keeping latency bounded when inference runs slower than the source. Right for live
  sources like a webcam, where the edit should track "now".
* `true`: every frame. The model consumes every source frame in order, for smoother motion at the
  cost of a growing delay. Right for pre-recorded clips and for drag-to-animate, where smoothness
  matters more than latency.

Emits `state_update`. `reset` does not clear this setting, unlike the prompt, reference image, and
pointer: it persists until the session ends, so re-sync any UI toggle to the default on disconnect
rather than on reset.

**Parameters:**

| Parameter      | Type    | Required | Description                                    |
| -------------- | ------- | -------- | ---------------------------------------------- |
| `keep_backlog` | boolean | No       | `true` = every frame in order. Default `false` |

<CodeGroup>
  ```typescript JavaScript theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}}
  await reactor.sendCommand("set_keep_backlog", { keep_backlog: true });
  ```

  ```tsx React theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}}
  const { setKeepBacklog } = useX2();
  await setKeepBacklog({ keep_backlog: true });
  ```

  ```python Python theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}}
  await reactor.send_command("set_keep_backlog", {"keep_backlog": True})
  ```
</CodeGroup>

### `reset`

Stop generation and clear the prompt, reference image, and pointer, returning the session to waiting
for new conditions. Valid at any time. Emits `generation_stopped` with reason `reset` if a run was
active, and `state_update`. Takes no arguments.

Note two things `reset` leaves alone: the output resolution (chosen once per session, at the first
generation) and the `keep_backlog` policy (persists until the session ends).

<CodeGroup>
  ```typescript JavaScript theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}}
  await reactor.sendCommand("reset", {});
  ```

  ```tsx React theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}}
  const { reset } = useX2();
  await reset();
  ```

  ```python Python theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}}
  await reactor.send_command("reset", {})
  ```
</CodeGroup>

## Messages

X2 emits the following messages, each arriving as JSON `{ "type": "<name>", "data": { … } }`.

| Event                      | When                                         | Payload                                                                  |
| -------------------------- | -------------------------------------------- | ------------------------------------------------------------------------ |
| `prompt_accepted`          | After `set_prompt`                           | `{ prompt: string }`                                                     |
| `reference_image_accepted` | After `set_reference_image` decodes          | `{ width: int, height: int }`                                            |
| `pointer_changed`          | After `set_pointer`                          | `{ x: number, y: number, active: bool }`                                 |
| `generation_started`       | When a run begins producing `main_video`     | `{ prompt: string, width: int, height: int, has_reference_image: bool }` |
| `generation_stopped`       | When a run ends                              | `{ reason: "reset" \| "reference_image_changed" }`                       |
| `command_error`            | When a command is rejected                   | `{ command: string, reason: string }`                                    |
| `state_update`             | On connect and after every observable change | Full session snapshot (see below)                                        |

A `generation_stopped` with reason `reference_image_changed` is an automatic restart: a fresh
`generation_started` follows at once, so treat it as a hiccup rather than an end state. Only reason
`reset` means the session is back to waiting for conditions; drive any "clear the drafts" UI off
that reason, not off every stop.

### `state_update` payload

`state_update` is the single source of truth for driving UI. It arrives on connect and after every
observable state change, always carrying the full state. Subscribe once and reduce it into your app
state; the only fact it does not carry is the decoded reference image's dimensions, which come from
the discrete `reference_image_accepted` ack.

| Field                 | Type             | Meaning                                                        |
| --------------------- | ---------------- | -------------------------------------------------------------- |
| `prompt`              | `string \| null` | The active editing instruction, or `null` when none is set     |
| `has_reference_image` | `bool`           | Whether a reference image is conditioning generation           |
| `pointer_x`           | `number`         | Current horizontal pointer position, `0` to `1`                |
| `pointer_y`           | `number`         | Current vertical pointer position, `0` to `1`                  |
| `pointer_active`      | `bool`           | Whether a drag is steering the edited subject                  |
| `keep_backlog`        | `bool`           | Current source-frame policy (see `set_keep_backlog`)           |
| `generating`          | `bool`           | Whether a run is producing `main_video` frames                 |
| `width`               | `int \| null`    | Output frame width, or `null` before the resolution is chosen  |
| `height`              | `int \| null`    | Output frame height, or `null` before the resolution is chosen |

**Example handler:**

<CodeGroup>
  ```typescript JavaScript theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}}
  reactor.on("message", (msg) => {
    switch (msg.type) {
      case "state_update":
        console.log(`generating: ${msg.data.generating} · prompt: ${msg.data.prompt}`);
        break;
      case "generation_started":
        console.log(`output: ${msg.data.width}x${msg.data.height}`);
        break;
      case "command_error":
        console.error(`${msg.data.command} rejected: ${msg.data.reason}`);
        break;
    }
  });
  ```

  ```tsx React theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}}
  import { useX2StateUpdate, useX2CommandError } from "@reactor-models/x2";

  function StateReadout() {
    useX2StateUpdate((s) => console.log(`generating: ${s.generating} · prompt: ${s.prompt}`));
    useX2CommandError((e) => console.error(`${e.command} rejected: ${e.reason}`));
    return null;
  }
  ```

  ```python Python theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}}
  @reactor.on_message
  def handle(msg):
      data = msg["data"]
      if msg["type"] == "state_update":
          print(f"generating: {data['generating']} · prompt: {data['prompt']}")
      elif msg["type"] == "command_error":
          print(f"{data['command']} rejected: {data['reason']}")
  ```
</CodeGroup>

## Complete example

Publish a webcam source, set a reference image and prompt, then steer the subject with a short drag.

<CodeGroup>
  ```typescript JavaScript theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}}
  import { X2Model } from "@reactor-models/x2";

  const x2 = new X2Model();

  x2.onGenerationStarted(({ width, height, has_reference_image }) => {
    console.log(`run started at ${width}x${height}, reference: ${has_reference_image}`);
  });
  x2.onCommandError(({ command, reason }) => {
    console.error(`${command} rejected: ${reason}`);
  });
  x2.onMainVideo((track, stream) => {
    video.srcObject = stream;
    void video.play();
  });

  await x2.connect(jwt);

  // Publish the stream to edit.
  const media = await navigator.mediaDevices.getUserMedia({ video: true });
  await x2.publishSource(media.getVideoTracks()[0]);

  // Anchor the swap to a reference image, then arm generation with a prompt.
  const ref = await x2.uploadFile(characterImageFile);
  await x2.setReferenceImage({ reference_image: ref });
  await x2.setPrompt({
    prompt: "replace the character in the video with the character from the reference image",
  });
  // No start command: generation begins now that a prompt is set and frames are flowing.

  // Steer the subject with a short drag to the right.
  await x2.setPointer({ x: 0.5, y: 0.5, active: true });
  await x2.setPointer({ x: 0.8, y: 0.5, active: true });
  await x2.setPointer({ x: 0.8, y: 0.5, active: false });
  ```

  ```python Python theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}}
  import asyncio, os
  from reactor_sdk import Reactor

  async def main():
      reactor = Reactor(model_name="xmax/x2", api_key=os.environ["REACTOR_API_KEY"])

      @reactor.on_message("generation_started")
      def _(msg):
          d = msg["data"]
          print(f"run started at {d['width']}x{d['height']}")

      await reactor.connect()

      ref = await reactor.upload_file("character.jpg")
      await reactor.send_command("set_reference_image", {"reference_image": ref})
      await reactor.send_command("set_prompt", {
          "prompt": "replace the character in the video with the character from the reference image",
      })

      # Steer with a short drag.
      await reactor.send_command("set_pointer", {"x": 0.5, "y": 0.5, "active": True})
      await reactor.send_command("set_pointer", {"x": 0.8, "y": 0.5, "active": True})
      await reactor.send_command("set_pointer", {"x": 0.8, "y": 0.5, "active": False})
      await asyncio.Event().wait()

  asyncio.run(main())
  ```
</CodeGroup>