FastH3-Omni tutorial

This walkthrough creates a clip from a text prompt, then continues it with an image reference. It uses the base JavaScript SDK so you can see each model command. See Authentication to issue a session token from your server and File Uploads for the upload flow.

Installation and setup

Add the base JavaScript SDK to an app with a <video> and an <audio> element. This walkthrough assumes your app provides the token helper, file picker, and UI.

npm install @reactor-team/js-sdk

How FastH3-Omni works

The session starts idle. enqueue queues a clip for generation. When clip_generated arrives, play sends it to the output tracks. A command reply confirms that Reactor accepted the request. Watch later messages and the tracks for the result.

Authentication

Issue a session token on your server. The browser’s getToken() helper calls that server route. Do not put an rk_ API key in browser code. See Authentication for the token exchange and limits.

1. Connect and show the tracks

Have the browser request a short-lived token from your server, then connect to reactor/fast-h3-omni. Render main_video in the video element and attach main_audio to the audio element. Connecting does not generate a clip.

import { Reactor } from "@reactor-team/js-sdk";

const reactor = new Reactor({ modelName: "reactor/fast-h3-omni" });
const video = document.querySelector("video")!;
const audio = document.querySelector("audio")!;

reactor.on("trackReceived", (name, _track, stream) => {
  if (name === "main_video") {
    video.srcObject = stream;
    void video.play().catch(console.error);
  }
  if (name === "main_audio") {
    audio.srcObject = stream;
    void audio.play().catch(console.error);
  }
});

await reactor.connect(await getToken()); // Your server issues the token.

Start playback from a user action if the browser blocks audio autoplay. Wait for the SDK’s ready status before sending commands. On connect, the model sends state_update with the current defaults, capacities, and valid_commands. It also sends queue_update with both queues.

2. Queue a clip from text

Choose a supported duration. Register the message listener before enqueue so it catches the first clip’s result. The clip_queued reply gives you the clip ID, but the clip cannot be continued or played until clip_generated arrives.

const firstClipReady = new Promise<string>((resolve, reject) => {
  reactor.on("message", (message) => {
    if (message.type !== "clip_generated" && message.type !== "clip_failed") return;
    const result = message.data as { clip: { clip_id: string }; reason?: string };
    if (message.type === "clip_failed") {
      reject(new Error(result.reason ?? "Clip generation failed"));
    } else {
      resolve(result.clip.clip_id);
    }
  });
});

const queued = await reactor.sendCommand("enqueue", {
  prompt: "A ceramic artist turns a blue bowl under warm studio light. The camera slowly pushes in. A pottery wheel hums and rain taps the window.",
  seconds: 5.875,
  metadata: "opening-shot",
});
if (queued?.type !== "clip_queued") throw new Error("The clip was not queued");
const firstClipId = (queued.data as { clip: { clip_id: string } }).clip.clip_id;

The reply means the clip is queued, not ready to play. If generation fails, clip_failed supplies a reason.

3. Play when ready

Wait for clip_generated, then call the model’s play command. The model sends clip_started when playback begins and clip_finished when both tracks finish. The video and audio elements from step 1 render those tracks.

const generatedClipId = await firstClipReady;
await reactor.sendCommand("play", { clip_id: generatedClipId });

If you enable autoplay with set_autoplay({ enabled: true }), remove the play call. Autoplay plays ready clips when the output is idle; it does not generate new ones.

4. Add a reference and continue

Only continue after firstClipReady resolves. Upload a JPEG, PNG, or WebP image through the SDK and pass its FileRef to reference_image. The default continuation carries the first clip’s motion, camera, and audio. Both clips must use the same canvas, and the model must still retain the first clip.

const imageRef = await reactor.uploadFile(imageFile);

await reactor.sendCommand("enqueue", {
  prompt: "Continue the same studio scene. The artist from <Picture 1> lifts the finished blue bowl toward the window. The wheel slows, and rain continues outside.",
  reference_image: imageRef,
  continue_from_clip_id: firstClipId,
  seconds: 8,
});

Here imageFile comes from your file picker. In the prompt, <Picture 1> names the first reference image; the brackets are prompt text, not API syntax. With the base JavaScript SDK 3.1.0 or later, pass FileRef values directly, including in reference_images and reference_audios arrays. To start the next clip on the previous clip’s final frame instead, set continue_mode: "frame" and keep a reference image in that request. You can also add last_frame_image to pin its ending. A frame continuation supplies its own first frame, so do not send first_frame_image with continue_from_clip_id. See the schema for a keyframe example.

5. Manage state and errors

Read state_update for the generation and playout capacities. A full generation queue refuses enqueue; a full playout queue pauses builds. Use move to reorder a queued clip, pop to remove one, or stop to cut playback. Use reset to clear the session. Show the reason from command_error when the model refuses a command. If generation fails, clip_failed reports the reason and removes the clip from the queue. You can then submit another request. The model refuses stale continuation IDs, unsupported durations, invalid media, and over-budget prompts at enqueue time.

Surfacing command_error

reactor.on("message", (message) => {
  if (message.type === "command_error" || message.type === "clip_failed") {
    console.error(message.data.reason);
  }
});

The schema lists every field and message. The prompt guide has scene and soundtrack examples.

Overview

FastH3-Omni

Vidu S2-Avatar

Vidu S2-Editing

H3 Reference Turbo Realtime

FastH3

Visko Orbis Stable

Visko Orbis Dynamic

HappyOyster

LingBot World 2

LingBot

X2

Helios

LongLive-2.0

SANA-Streaming

LTX

FastH3-Omni tutorial

Installation and setup

How FastH3-Omni works

Authentication

1. Connect and show the tracks

2. Queue a clip from text

3. Play when ready

4. Add a reference and continue

5. Manage state and errors

Surfacing command_error

​Installation and setup

​How FastH3-Omni works

​Authentication

​1. Connect and show the tracks

​2. Queue a clip from text

​3. Play when ready

​4. Add a reference and continue

​5. Manage state and errors

​Surfacing command_error

Installation and setup

How FastH3-Omni works

Authentication

1. Connect and show the tracks

2. Queue a clip from text

3. Play when ready

4. Add a reference and continue

5. Manage state and errors

Surfacing command_error