Pika H3 Flex tutorial

This walkthrough uses the base JavaScript SDK so that you can see each model command. It makes a clip from a prompt, plays it, continues it with a reference image and a voice, and then continues once more from the last frame. The snippets build on each other in one browser module. Your app supplies three things that this page does not show:

getToken(), which asks your server for a session token. The Authentication section shows the server side.
imageFile and voiceFile, which are File objects from a file picker.
A page with one <video> element and one <audio> element.

Installation and setup

Install the base JavaScript SDK:

npm install @reactor-team/js-sdk

To start from a complete example app instead, run npx create-reactor-app my-pika-app --model=pika-h3-flex.

How Pika H3 Flex works

The session starts idle, with two empty queues. enqueue puts a clip in the generation queue. When the clip is ready, the model sends clip_generated and moves it to the playout queue, and play streams it on main_video and main_audio. A command reply tells you that the model accepted the request, not that the clip is ready or playing. Wait for the messages that follow, such as clip_generated and clip_started.

Authentication

Create the session token on your server so that your rk_ API key never reaches the browser. This route exchanges the key for a short-lived token that can only start reactor/pika-h3-flex sessions. The Python SDK exchanges the key for you, so it needs no route.

import { NextResponse } from "next/server";

export async function POST() {
  const response = await fetch("https://api.reactor.inc/tokens", {
    method: "POST",
    headers: {
      "Reactor-API-Key": process.env.REACTOR_API_KEY!,
      "Content-Type": "application/json",
    },
    body: JSON.stringify({
      authorization_details: [
        { type: "session", resources: { models: { match: ["reactor/pika-h3-flex"] } } },
      ],
    }),
  });
  if (!response.ok) {
    return NextResponse.json({ error: await response.text() }, { status: response.status });
  }
  const { jwt, expires_at } = await response.json();
  return NextResponse.json({ jwt, expires_at }, { headers: { "Cache-Control": "no-store" } });
}

import asyncio
import os
from reactor_sdk import Reactor

async def main():
    reactor = Reactor(model_name="reactor/pika-h3-flex", api_key=os.environ["REACTOR_API_KEY"])
    await reactor.connect()  # The SDK exchanges the key for a token.

asyncio.run(main())

See Authentication for token lifetimes and for how to keep a token fresh for a whole session.

1. Connect and show the tracks

Connect to reactor/pika-h3-flex, and attach main_video to the video element and main_audio to the audio element. Connecting on its own does not make a clip.

import { Reactor, type ReactorError } from "@reactor-team/js-sdk";

const reactor = new Reactor({ modelName: "reactor/pika-h3-flex" });
const video = document.querySelector("video")!;
const audio = document.querySelector("audio")!;

reactor.on("trackReceived", (name, _track, stream) => {
  if (name === "main_video") {
    video.srcObject = stream;
    void video.play().catch(console.error);
  }
  if (name === "main_audio") {
    audio.srcObject = stream;
    void audio.play().catch(console.error);
  }
});

let lastRefusal: ReactorError | undefined;
reactor.on("error", (error) => {
  if (error.operation === "sendCommand") lastRefusal = error;
});

await reactor.connect(await getToken());

If the browser blocks audio autoplay, connect from a user action such as a button click. Wait for the ready status before you send commands. On connect, the model sends a state_update with the current settings and valid_commands, plus a queue_update with both queues.

2. Queue a clip from text

Register one message listener before you call enqueue. It records every clip’s result, so nothing is lost if a result arrives before you start waiting. The waitForClip helper then waits for one clip to be ready or to fail:

type ClipResult = { ok: true } | { ok: false; reason: string };
const results = new Map<string, ClipResult>();
const waiters = new Map<string, (result: ClipResult) => void>();

reactor.on("message", (message) => {
  if (message.type !== "clip_generated" && message.type !== "clip_failed") return;
  const data = message.data as { clip: { clip_id: string }; reason?: string };
  const result: ClipResult =
    message.type === "clip_generated"
      ? { ok: true }
      : { ok: false, reason: data.reason ?? "Clip generation failed" };
  results.set(data.clip.clip_id, result);
  waiters.get(data.clip.clip_id)?.(result);
});

async function waitForClip(clipId: string): Promise<void> {
  const result =
    results.get(clipId) ??
    (await new Promise<ClipResult>((resolve) => waiters.set(clipId, resolve)));
  waiters.delete(clipId);
  if (!result.ok) throw new Error(result.reason);
}

const queued = await reactor.sendCommand("enqueue", {
  prompt:
    "A singer stands in a small rehearsal room with a microphone. She closes her eyes and hums a short melody. The camera holds a close shot. Soft room tone. No music.",
  metadata: "opening",
});
if (queued?.type !== "clip_queued") {
  throw new Error(lastRefusal?.message ?? "The model refused this command.");
}
const firstClipId = (queued.data as { clip: { clip_id: string } }).clip.clip_id;
const firstClipReady = waitForClip(firstClipId);

This request sends no seconds, so the clip uses the session default. Read state_update.clip_seconds to see the current default, or send seconds with one of the values in state_update.clip_seconds_choices. The clip_queued reply means the clip is in the queue, not that it is ready to play. If the model refuses the request, the call resolves undefined and the error event carries the code, such as queue_full or request_rejected.

3. Play the clip

Wait for the clip to be ready, then call play. The model sends clip_started when playback starts and clip_finished when the clip ends.

await firstClipReady;
await reactor.sendCommand("play", { clip_id: firstClipId });

To play clips without calling play each time, send set_autoplay with enabled: true. Autoplay starts each ready clip when nothing is playing, but it does not create clips.

4. Continue with a reference image and a voice

Upload the files and pass the references they return. The prompt names the image <Picture 1> and the voice <Audio 1>, and the continued clip’s soundtrack takes the next label, <Audio 2>.

const portrait = await reactor.uploadFile(imageFile);
const voice = await reactor.uploadFile(voiceFile);

const next = await reactor.sendCommand("enqueue", {
  prompt:
    "<Picture 1> is the singer. <Audio 1> is her voice. She opens her eyes and sings, “Stay with the morning light.” <Audio 2> continues the room tone. No music.",
  reference_images: [portrait],
  reference_audios: [voice],
  continue_from_clip_id: firstClipId,
  seconds: 8,
});

This is a latent continuation, the default. The first clip must have finished generating, and the model must still keep it; it keeps the 8 most recent generated clips. Both clips must use the same canvas. You can upload up to 3 audio clips. The continued sound takes the next audio label after all of yours, but it does not use an audio slot. The reference image guides how the singer looks, but it does not set the first frame. The end of the first clip guides the motion, camera, and sound at the start of the new one.

5. Continue from the last frame

A frame continuation opens the next clip on the last frame of an earlier one. It takes a prompt only, with no references. Only the picture carries over, so the sound may restart at the boundary. In the prompt, the earlier clip’s last frame is <Picture 1>. You can queue it right away. It waits in the generation queue until the second clip is built:

const secondClipId = (next?.data as { clip: { clip_id: string } } | undefined)?.clip.clip_id;
if (!secondClipId) throw new Error(lastRefusal?.message ?? "The model refused this command.");

await reactor.sendCommand("enqueue", {
  prompt:
    "For the target video, at 0.00 seconds into the target video, <Picture 1> (from [Shot 1]) is fully referenced.\n\nintegrated_multimodal_description: [Shot 1] Begin from <Picture 1>. The singer steps back from the microphone, smiles, and walks out of the rehearsal room.\noverall_soundscape: Footsteps on a wooden floor and the hum of the room.\nnon_diegetic_music: N/A",
  continue_from_clip_id: secondClipId,
  continue_mode: "frame",
});

To start a clip from your own image instead, send first_frame_image with no continuation. The schema covers first and last frames.

6. Manage the queues

state_update reports the capacity of each queue, and when the generation queue is full, enqueue fails with queue_full. These commands manage the session:

move changes the order of clips within a queue.
pop removes a clip from a queue.
stop cuts the playing clip. With autoplay on, the next ready clip starts.
set_canvas changes the aspect ratio, and set_resolution switches between 768p and 480p. Both work only when both queues are empty and nothing plays.
reset clears everything and restores the defaults.

Use state_update.valid_commands to turn buttons on and off so that your UI matches what the session accepts.

Surfacing refusals

A refused command fails its own call, and only your client gets the refusal. Read it from the error event whose operation is "sendCommand", and show it to the user next to the control that sent the command. Do not show it as a connection problem, because the session is still connected. A generation failure is different: the model accepted the request, but the clip failed later. The model sends clip_failed with a reason and removes the clip from the queue, and generation moves on to the next clip.

reactor.on("error", (error) => {
  // error.code is the model's code, such as "invalid_request".
  if (error.operation === "sendCommand") showRefusal(error.code, error.message);
});

reactor.on("message", (message) => {
  if (message.type === "clip_failed") showClipFailure((message.data as { reason: string }).reason);
});

The model also broadcasts a command_error message for each refusal. That broadcast is legacy, so do not listen for it. The schema lists every code. The schema lists every field and message. The prompt guide has scene and soundtrack examples.

Overview

Pika H3 Flex

FastH3-Omni

EdenLive

Vidu S2-Avatar

Vidu S2-Editing

H3 Reference Turbo Realtime

FastH3

Visko Orbis Stable

Visko Orbis Dynamic

HappyOyster

LingBot World 2

LingBot

X2

XMAX X2.1

Helios

LongLive-2.0

SANA-Streaming

LTX