XMAX X2.1 schema

Publish source video on source and receive edited video on main_video. With the base SDK, send commands through reactor.sendCommand(). The React examples use useXmaxX21 from @reactor-models/xmax-x2-1 inside an XmaxX21Provider.

Tracks

Track	Kind	Client direction	Purpose
`source`	Video	Send	The webcam, clip, or repeated still image to edit.
`main_video`	Video	Receive	The edited output.

The model has no audio track. A still image needs a client that publishes repeated video frames.

Session lifecycle

The model sends state_update on connect. A null prompt means no prompt was set. Call set_prompt to enable generation, then provide source frames. An empty string is a valid prompt for drag-only animation. There is no start command. generation_started reports the output dimensions. Those dimensions stay fixed for the session, including after reset. Prompt changes apply from the next generated block. Use the latest state_update to display model state. Await command replies on the connection that sends them. prompt_accepted, pointers_changed, and reference_image_accepted answer their respective commands. A reply does not establish that the edited frames are visible. Observe main_video and state messages separately.

Commands

Command	Description
`set_prompt`	Set or replace the editing instruction, including an empty instruction.
`set_reference_image`	Supply a reference image for a character or object.
`set_pointers`	Set a four-frame window of drag points.
`reset`	Stop generation and clear the prompt, reference image, and drag points.

`set_prompt`

Set the editing instruction. A prompt enables generation even when its value is "". During a run, a replacement applies from the next block. Returns prompt_accepted and broadcasts state_update.

Parameter	Type	Required	Description
`prompt`	string	No	Up to 1,000 characters. Defaults to `""`.

const reply = await reactor.sendCommand("set_prompt", {
  prompt: "Replace the white ball in the hand with a small orange kitten.",
});

const { setPrompt } = useXmaxX21();
const reply = await setPrompt({ prompt: "" });

reply = await reactor.send_command("set_prompt", {"prompt": ""})

`set_reference_image`

Upload the file first, then pass the returned file reference. A successful command returns reference_image_accepted with the decoded image’s original dimensions and broadcasts state_update. A file that cannot decode as an image fails with invalid_image. Replacing the image during a run applies the reference from the next block. The stream continues without generation_stopped or a new generation_started. The block that applies the reference takes longer, so expect one short pause. The visible edit moves to the new reference over the next few blocks.

Parameter	Type	Required	Description
`reference_image`	FileRef	Yes	An image uploaded through Reactor’s upload protocol.

The image can have any aspect ratio. When its ratio is within 20% of the output ratio, the model scales it to cover and center-crops it. Otherwise, it fits the whole image and adds white padding.

const reference = await reactor.uploadFile(imageFile);
const reply = await reactor.sendCommand("set_reference_image", {
  reference_image: reference,
});

const { uploadFile, setReferenceImage } = useXmaxX21();
const reference = await uploadFile(imageFile);
const reply = await setReferenceImage({ reference_image: reference });

reference = await reactor.upload_file("character.jpg")
reply = await reactor.send_command("set_reference_image", {
    "reference_image": reference,
})

`set_pointers`

Each call contains exactly four frame samples, ordered from oldest to newest. Each sample is a list of zero to eight drag points. For example, to steer two points, send four lists with two points in each list. Each point has x and y coordinates from 0 to 1, measured against the output frame. The origin is the top-left corner.

Parameter	Type	Required	Description
`frames`	Array of four point arrays	Yes	Each point is `{ x: number, y: number }`.

At each source-frame tick, append the current positions and send the last four ticks. The latest call replaces the pending window. A block uses each entry on its corresponding frame. The first one-frame block uses the newest entry. If no new call arrives before the next block, the model holds the previous window’s newest entry on all frames. Send [[], [], [], []] to end the drag. Invalid window length, too many points, or coordinates outside the range fail with invalid_points. The reply, pointers_changed, contains only the newest frame’s points as [x, y] pairs. The state_update.pointers field has the same shape. These payloads differ from the command’s four lists of objects.

await reactor.sendCommand("set_pointers", {
  frames: [
    [{ x: 0.4, y: 0.5 }],
    [{ x: 0.42, y: 0.5 }],
    [{ x: 0.44, y: 0.5 }],
    [{ x: 0.46, y: 0.5 }],
  ],
});

const { setPointers } = useXmaxX21();
await setPointers({ frames: [[], [], [], []] });

await reactor.send_command("set_pointers", {
    "frames": [
        [{"x": 0.40, "y": 0.50}],
        [{"x": 0.42, "y": 0.50}],
        [{"x": 0.44, "y": 0.50}],
        [{"x": 0.46, "y": 0.50}],
    ],
})

`reset`

Stop generation and clear the prompt, reference image, and drag points. This command is valid at any time. If a run is active, it emits generation_stopped with reason reset. It also broadcasts state_update. Reset has no parameters or reply payload. To generate again, publish source frames and call set_prompt. The session keeps its output dimensions.

await reactor.sendCommand("reset", {});

const { reset } = useXmaxX21();
await reset();

await reactor.send_command("reset", {})

Messages

Message	Description	Payload
`state_update`	Full state on connect and after observable changes.	Fields below.
`prompt_accepted`	Reply to `set_prompt`.	`prompt: string`
`pointers_changed`	Reply to `set_pointers`.	`points: number[][]`, for the newest frame.
`reference_image_accepted`	Reply after an uploaded image decodes.	Original `width` and `height`.
`generation_started`	A run starts to produce output.	`prompt`, `width`, `height`, `has_reference_image`.
`generation_stopped`	A run stops producing output.	`reason: string`, currently `reset`.

`state_update` payload

Field	Type	Meaning
`prompt`	string or null	Active instruction. `null` means unset. `""` is valid.
`has_reference_image`	boolean	Whether a reference image conditions generation.
`pointers`	number[][]	Newest drag positions as `[x, y]` pairs.
`generating`	boolean	Whether a run produces output frames.
`width`	integer or null	Output width, or `null` before a run chooses it.
`height`	integer or null	Output height, or `null` before a run chooses it.

Complete example

This browser module requires a <video id="output" autoplay playsinline muted> element and the token route from Authentication. Call stop() when the page closes or the user disconnects.

import { XmaxX21Model } from "@reactor-models/xmax-x2-1";

const output = document.getElementById("output") as HTMLVideoElement;
const model = new XmaxX21Model();
const unsubscribe = model.onMainVideo((_track, stream) => {
  output.srcObject = stream;
  void output.play().catch(console.error);
});
model.on("error", console.error);
model.onStateUpdate((state) => console.log(state.generating, state.width, state.height));

const response = await fetch("/api/reactor/token", { cache: "no-store" });
if (!response.ok) throw new Error(`Token request failed: ${response.status}`);
const { jwt } = await response.json();
await model.connect(jwt);
const camera = await navigator.mediaDevices.getUserMedia({ video: true });
await model.publishSource(camera.getVideoTracks()[0]);
const accepted = await model.setPrompt({ prompt: "" });
if (!accepted) console.error(model.getLastError());

async function stop() {
  await model.reset();
  camera.getTracks().forEach((track) => track.stop());
  unsubscribe();
  await model.disconnect();
  output.srcObject = null;
}

Overview

FastH3-Omni

EdenLive

Vidu S2-Avatar

Vidu S2-Editing

H3 Reference Turbo Realtime

FastH3

Visko Orbis Stable

Visko Orbis Dynamic

HappyOyster

LingBot World 2

LingBot

X2

XMAX X2.1

Helios

LongLive-2.0

SANA-Streaming

LTX

XMAX X2.1 schema

Tracks

Session lifecycle

Commands

`set_prompt`

`set_reference_image`

`set_pointers`

`reset`

Messages

`state_update` payload

Complete example

​Tracks

​Session lifecycle

​Commands

​set_prompt

​set_reference_image

​set_pointers

​reset

​Messages

​state_update payload

​Complete example

Tracks

Session lifecycle

Commands

`set_prompt`

`set_reference_image`

`set_pointers`

`reset`

Messages

`state_update` payload

Complete example