Skip to main content

Documentation Index

Fetch the complete documentation index at: https://docs.reactor.inc/llms.txt

Use this file to discover all available pages before exploring further.

A guided tour of the open-source Helios Interactive reference app, which demonstrates every important pattern in the Helios SDK. By the end you’ll know how to start scenes from prompts or images, hot-swap prompts mid-stream, snap clips, and surface model errors.

Installation and setup

Get the example running locally before reading further. Every section below points back at code in the repo you just cloned. You will need:
  • Node.js 18+.
  • pnpm (the example pins lockfiles to pnpm; npm or yarn will work but you’ll regenerate the lockfile).
  • A Reactor API key (starts with rk_).
  • Familiarity with the Next.js App Router.
1

Clone the example

The example lives alongside our other reference apps in reactor-team/reactor-experiments.
git clone https://github.com/reactor-team/reactor-experiments
cd reactor-experiments/helios
2

Add your API key

Your rk_… key must never reach the browser; the example reads it server-side and mints a short-lived JWT for the client. We’ll cover the broker pattern below; for now, drop the key into .env:
cp .env.example .env
# then edit .env and set REACTOR_API_KEY to your API key
See a “Setup Required” screen? Your REACTOR_API_KEY isn’t loaded. The check lives in app/page.tsxapp/SetupRequired.tsx.
3

Install dependencies and start the dev server

pnpm install
pnpm dev
Open http://localhost:3000, click Connect, and pick a starting point: a curated prompt, an example image, or your own text.

How Helios works

Building with Helios is different from calling a typical generative API. There’s no prompt-in / video-out request. You open a long-lived connection, send a prompt, and the model begins producing a continuous stream of 33-frame chunks. You steer it by mutating the prompt while it runs and the model applies each change at the next chunk boundary. Opening the connection isn’t instant. Reactor provisions a GPU for your session, so the client moves through four states before media starts flowing:
Connection lifecycle: disconnected → connecting → waiting → ready
The waiting state is when the GPU is being assigned, which typically takes a few seconds. Once the status reaches ready, commands take effect and chunks start arriving. See Sessions for the full breakdown. At ready the model is connected but idle; it won’t produce frames until you set a prompt and call start. From there a small set of SDK methods (setPrompt, start, pause / resume, reset) drives it through the rest of its lifecycle. Two things to note about those methods:
  • They’re asynchronous; events are the source of truth. Calling setImage doesn’t mean the next chunk will use it. The model confirms by emitting image_accepted when the change has actually landed.
  • Errors arrive out-of-band. A broken precondition like start with no prompt surfaces later as a command_error event, not as a thrown exception.

Authentication

Helios is different from most video-generation APIs. Instead of sending your API key in a header and receiving an image from the server, Helios opens a long-lived WebRTC connection that the server needs to trust for hours. Shipping a raw rk_… key to the browser would hand full account access to anyone with devtools open. Instead, the Reactor SDK presents a JWT minted server-side from your API key. The JWT is short-lived (Reactor caps it at 6 hours), scoped to a single session, and safe to hand to the client. Your rk_… key stays on the server. That means every Helios frontend needs one server-side route that mints JWTs. In the example, that route is app/api/reactor/token/route.ts, a Next.js route handler that exchanges your rk_… key for a JWT and hands the JWT back with a Cache-Control header derived from the token’s actual expiry:
import { NextResponse } from "next/server";

// How long the minted token should live, in seconds (Reactor caps this at its
// server max). One minute of skew keeps caches from serving a near-expired JWT.
const TOKEN_LIFETIME_SECONDS = 3600;
const CACHE_SKEW_SECONDS = 60;

// Exposed as GET so the browser's HTTP cache can serve repeat calls.
// POST responses aren't cached. We still POST to Reactor internally.
export async function GET() {
  const apiKey = process.env.REACTOR_API_KEY!;
  // ...error handling omitted...

  const res = await fetch("https://api.reactor.inc/tokens", {
    method: "POST",
    headers: {
      "Reactor-API-Key": apiKey,
      "Content-Type": "application/json",
    },
    body: JSON.stringify({ expires_after: TOKEN_LIFETIME_SECONDS }),
  });

  const { jwt, expires_at } = (await res.json()) as {
    jwt: string;
    expires_at: number;
  };

  // Reactor caps token lifetime at its server max (currently 6h), so derive
  // max-age from the actual expires_at, with a one-minute safety skew.
  const nowSeconds = Math.floor(Date.now() / 1000);
  const maxAge = Math.max(0, expires_at - nowSeconds - CACHE_SKEW_SECONDS);

  // `private` keeps shared caches (CDNs, corporate proxies) from storing
  // per-user JWTs. JWTs must never be reused across users.
  return NextResponse.json({ jwt }, { headers: { "Cache-Control": `private, max-age=${maxAge}` } });
}
HeliosApp.tsx fetches the token once on mount and passes it to <HeliosProvider>:
app/HeliosApp.tsx
import { HeliosProvider } from "@reactor-models/helios";

// Fetches the JWT from our route. The route's Cache-Control header lets
// the browser serve repeat calls from its HTTP cache until the JWT expires.
async function fetchToken(): Promise<string> {
  const r = await fetch("/api/reactor/token");
  if (!r.ok) {
    const body = (await r.json().catch(() => ({}))) as { error?: string };
    throw new Error(body.error ?? `Token fetch failed: ${r.status}`);
  }
  const { jwt } = (await r.json()) as { jwt: string };
  return jwt;
}

export function HeliosApp() {
  const [token, setToken] = useState<string | null>(null);
  // ...error + loading states omitted...

  useEffect(() => {
    fetchToken()
      .then(setToken)
      .catch((e) => setError(String(e)));
  }, []);

  // HeliosProvider needs a valid JWT, don't render it until we have one.
  if (!token) return <Loading />;

  // HeliosProvider owns the WebRTC connection from here; children read
  // session state through SDK hooks.
  return <HeliosProvider jwtToken={token}>{/* ...app tree... */}</HeliosProvider>;
}
After a reload or navigation, that same fetch("/api/reactor/token") is served from the browser’s HTTP cache until the JWT actually expires and never touches your server or Reactor. Once the cache window closes, the next fetch refills it.
The broker pattern (server mints, client consumes) is the standard for any browser-side Reactor app, not just Helios. See Authentication for the full concept page, including the Express equivalent and the Python path that skips the broker entirely.

Starting from a prompt

Generation kicks off in two SDK calls: setPrompt registers the prompt at chunk 0, then start begins producing chunks. PromptComposer.tsx exposes a grid of curated presets and a free-text input, but every button routes through the same send function:
app/components/PromptComposer.tsx
const { setPrompt, start } = useHelios();
const [text, setText] = useState("");
// ...status / ready guard omitted...

// The two-call flow every entry point in this panel shares:
//   1. setPrompt registers the prompt at chunk 0 (required before start)
//   2. start begins generation
async function send(prompt: string) {
  await setPrompt({ prompt: prompt.trim() });
  await start();
}

return (
  <>
    {/* Preset buttons. Text comes from the curated scene library. */}
    {TEXT_SCENES.map((scene) => (
      <button key={scene.id} onClick={() => send(scene.initial.text)}>
        {scene.label}
      </button>
    ))}

    {/* Free-text input. Same send(), different source string. */}
    <textarea value={text} onChange={(e) => setText(e.target.value)} />
    <button onClick={() => send(text)}>Start generating</button>
  </>
);
The preset text comes from app/lib/prompts.ts, a curated scene library that’s the single source of truth for both these prompts and the mid-stream evolutions covered later in the article. Its header comment is worth reading: each prompt is a full paragraph for a reason, and that style is what lets the model hot-swap prompts smoothly later. The example calls set_prompt, a convenience wrapper that picks the chunk index automatically. If you need to queue a prompt for a specific future chunk, reach for schedule_prompt instead.
start requires a prompt at chunk 0. Skip the setPrompt call and you’ll get a command_error event back. See start for the full precondition list; how the example surfaces those errors is covered later in the article.

Starting from an image

Image-to-video adds a second piece of conditioning, and chaining setImage → setPrompt → start directly hides a subtle race. setImage carries an upload that the runtime has to resolve before the model dispatches it; start carries nothing and sails past on the same data channel. The first chunk is then generated from the prompt alone, no image conditioning at all. The image lands a tick later and only applies from chunk 1 onward, so the user sees the scene “correct itself” at the first chunk boundary. setConditioning (Helios SDK 0.9.0+) is the fix: prompt and image ride on a single data-channel message. One message can’t be split or reordered, and the model handles it as one transaction. By the time start reaches the model, both pieces are in place.
Safe Helios image-start flow: send set_conditioning with the image and prompt together, then start, so the first chunk is conditioned on both
That collapses the curated-scene flow in ImageStarter.tsx to three calls:
app/components/ImageStarter.tsx
const { uploadFile, setConditioning, start } = useHelios();

async function startFromExample(scene: Scene & { imageUrl: string }) {
  const blob = await fetch(scene.imageUrl).then((r) => r.blob());
  const ref = await uploadFile(blob, { name: `${scene.id}.jpg` });

  // Atomic: prompt + image commit together, or neither commits.
  await setConditioning({ prompt: scene.initial.text, image: ref });
  await start();
}
If anything in the transaction fails (missing piece, non-image MIME, undecodable bytes), the model emits command_error and mutates nothing. Reach for setConditioning whenever both pieces are known at the same time: curated scene launches, “load this preset” buttons, anything that’s a single user click. The example’s second image path is for custom uploads, where the user’s prompt arrives later from a separate action. It’s shorter still: uploadFile, then setImage. The user types their own prompt in the composer above and clicks Start, at which point PromptComposer fires setPrompt + start. No race here either: by the time the human has typed and clicked, the upload has long since been VAE-encoded.
app/components/ImageStarter.tsx
async function uploadCustomImage(file: File) {
  const ref = await uploadFile(file);
  await setImage({ image: ref });
  // User types in PromptComposer, which fires setPrompt + start.
}
If you’re about to call start() and you need image conditioning, use setConditioning. Only fall back to setImage alone when the prompt arrives later from a separate user action: the custom-upload flow above, or a mid-stream image swap. The example images live in public/ and pair with hand-tuned starting prompts in app/lib/prompts.ts.
See File Uploads for what the SDK does with the bytes you hand it, and set_image for the command reference, including mid-stream image swaps.

Going live

Once generation starts, the UI flips from the setup panel to its Live phase. The example wires three small components into the right-hand sidebar and main pane: a status badge that tracks the connection lifecycle, a “now playing” panel that mirrors the state snapshot and exposes transport controls, and the video pane itself. StatusBadge.tsx is the user’s window into the four-state connection machine. Every state, including the multi-second waiting step where Reactor is provisioning a GPU, gets a visible label and color.
app/components/StatusBadge.tsx
import { useHelios } from "@reactor-models/helios";

const TONE = {
  disconnected: { dot: "bg-zinc-500", label: "Disconnected" },
  connecting: { dot: "bg-amber-400 animate-pulse", label: "Connecting…" },
  waiting: { dot: "bg-amber-400 animate-pulse", label: "Waiting for GPU…" },
  ready: { dot: "bg-active", label: "Connected" },
};

export function StatusBadge() {
  const { status, lastError, connect, disconnect } = useHelios();
  const idle = status === "disconnected";

  return (
    <div>
      <span className={TONE[status].dot} />
      <span>{TONE[status].label}</span>
      {idle ? (
        <button onClick={() => connect()}>Connect</button>
      ) : (
        <button onClick={() => disconnect()}>Disconnect</button>
      )}
      {lastError && <p className="text-red-400">{lastError.message}</p>}
    </div>
  );
}
useHelios() is the only hook needed here: status, connect, disconnect, and lastError all live on it. The button toggles purely on status === "disconnected"; every other state (connecting, waiting, ready) renders the Disconnect button. NowPlaying.tsx is the canonical example of how the rest of the app reads model state: subscribe once with useHeliosState, hold the latest snapshot in useState, read fields off it. No event aggregation, no derived booleans, no useReducer over chunk_complete events.
app/components/NowPlaying.tsx
const { status, pause, resume, reset } = useHelios();
const [snapshot, setSnapshot] = useState<HeliosStateMessage | null>(null);

useHeliosState((msg) => setSnapshot(msg));

// The SDK doesn't emit a final `state` message on disconnect, so we
// clear ourselves. Otherwise the next session inherits the old one.
useEffect(() => {
  if (status !== "ready") setSnapshot(null);
}, [status]);

// Phase switch: while not started (or after reset), render null and
// let the setup panel take over.
if (status !== "ready" || !snapshot?.started) return null;

return (
  <>
    <p>{String(snapshot.current_prompt ?? "")}</p>
    <span>chunk {snapshot.current_chunk}</span>
    <span>{snapshot.current_frame} frames</span>
    {snapshot.running ? (
      <button onClick={() => pause()}>Pause</button>
    ) : (
      <button onClick={() => resume()}>Resume</button>
    )}
    <button onClick={() => reset()}>Reset</button>
  </>
);
pause, resume, and reset are typed SDK methods on useHelios(), same shape as setPrompt and start from earlier sections: each returns a Promise that can reject with a command_error if its preconditions aren’t met. The video pane itself is one component:
app/components/Video.tsx
import { HeliosMainVideoView } from "@reactor-models/helios";

export function Video() {
  return (
    <div className="rounded-lg border bg-black">
      <HeliosMainVideoView className="h-full w-full" videoObjectFit="contain" />
    </div>
  );
}
<HeliosMainVideoView /> is a typed wrapper around <ReactorView track="main_video"> that handles <video> element setup, srcObject binding, and browser autoplay policy quirks. Style the outer container; never reach for the underlying element.

Hot-swapping prompts mid-stream

The most distinctive Helios feature is its ability to change the prompt without restarting. The example’s “evolve the scene” picker matches the active prompt against the prompt library and offers one-click continuations.
app/components/EvolveScene.tsx
const { status, setPrompt } = useHelios();
const [snapshot, setSnapshot] = useState<HeliosStateMessage | null>(null);
useHeliosState((msg) => setSnapshot(msg));

useEffect(() => {
  if (status !== "ready") setSnapshot(null);
}, [status]);

if (status !== "ready" || !snapshot?.started) return null;

// Match the active prompt against `initial` and every `evolutions[i].text`
// in the curated library. If nothing matches (free-text prompt), bail.
const scene = findSceneForPrompt(String(snapshot.current_prompt ?? ""));
if (!scene) return null;

return (
  <>
    <label>Evolve the scene</label>
    {scene.evolutions.map((evolution) => (
      <button key={evolution.title} onClick={() => setPrompt({ prompt: evolution.text })}>
        {evolution.title}
      </button>
    ))}
  </>
);
Each button is a single setPrompt call. No start, no reset, no acknowledgment wait. The model is already generating, and the next 33-frame chunk picks up the new prompt automatically. From the user’s perspective the scene just keeps going; from the model’s perspective the prompt schedule was updated for the next chunk boundary.
set_prompt is the convenience wrapper that targets “the next chunk.” If you know the exact chunk index where you want the change to land (a music cue, a beat counter), reach for schedule_prompt instead.

Snapping a clip

The SDK ships recording primitives so you don’t have to wire up MediaRecorder yourself. The example’s SnapClip.tsx captures the last 10 seconds of the live stream and opens a modal with the SDK’s built-in preview player and a download button.
app/components/SnapClip.tsx
import {
  ClipDownloadButton,
  ClipPlayer,
  RecordingError,
  useReactor,
  type Clip,
} from "@reactor-team/js-sdk";

const { status, reactor } = useReactor((s) => ({
  status: s.status,
  reactor: s.internal.reactor,
}));
const [clip, setClip] = useState<Clip | null>(null);

async function snap() {
  try {
    setClip(await reactor.requestClip(durationSeconds));
  } catch (e) {
    if (e instanceof RecordingError) {
      // render e.code + e.reason (omitted)
    }
  }
}

return (
  <>
    <button onClick={snap}>Snap last {durationSeconds}s</button>
    {clip && (
      <Modal onClose={() => setClip(null)}>
        <ClipPlayer clip={clip} getJwt={getJwt} />
        <ClipDownloadButton clip={clip} getJwt={getJwt} filename={filename} />
      </Modal>
    )}
  </>
);
Notice how the imports are from @reactor-team/js-sdk, not @reactor-models/helios. Recording is a base-SDK feature. It works identically for every Reactor model, and the typed model packages don’t re-export the recording surface. So direct base-SDK imports are idiomatic in this one place, and you can drop the file into any other Reactor example unchanged. reactor.requestClip(durationSeconds) is the whole capture API. It returns a Clip value that you hand to <ClipPlayer> to preview and <ClipDownloadButton> to save. The getJwt prop is a resolver those components call when they need an auth token to fetch the clip. The example reuses the same cached /api/reactor/token route from Authentication, so repeat captures don’t trigger new token mints. Errors come back as a RecordingError with a typed code and reason, distinct from the command_error events covered next.
Clip preview in Chromium and Firefox requires hls.js, already in the example’s package.json. See Recordings for the full feature page, including continuous recording, programmatic capture, and retention policies.

Surfacing command_error

Every Helios command can fail a precondition check (e.g. start before a prompt at chunk 0). The example never lets these fail silently.
app/components/CommandError.tsx
const [error, setError] = useState<{ command: string; reason: string } | null>(null);

useHeliosCommandError((msg) => {
  setError({ command: msg.command, reason: msg.reason });
});

// Clear on the next state snapshot. Any state change implies the user
// has moved on from whatever triggered the error.
useHeliosState(() => {
  setError(null);
});

if (!error) return null;

return (
  <div>
    <span>{error.command} failed</span>
    <p>{error.reason}</p>
  </div>
);
useHeliosCommandError is the typed wrapper for the command_error message: it fires when Helios rejects a command, carrying the failing command name and a human-readable reason. The component sits in the sidebar, renders nothing until an error arrives, and clears itself when the next state snapshot lands so a stale banner can’t pile up.
command_error is one of several messages Helios emits. See the Messages from model table for the full list, including chunk_complete, conditions_ready, and image_accepted.

What’s intentionally left out

Not every Helios feature is surfaced in this demo. See below for a list of what’s missing and the one-line addition that wires each one in.
FeatureHow to add it
Mid-stream image swapHelios supports changing the reference image during generation; the demo only swaps prompts. Drop a Live-phase image picker that calls setImage with the same upload + FileRef pattern from Starting from an image, minus start and the prompt. See set_image.
Custom-prompt evolutionsThe evolution picker hides for free-text prompts because there’s no known continuation set. To support them, generate evolutions on the fly (e.g. a small LLM call seeded from the active prompt) and feed the result into the same setPrompt call EvolveScene already uses.
Reproducible runsuseHelios().setSeed({ seed }). Add it as a Setup-phase control. Helios reads the seed once when start fires. See set_seed.
Exact-chunk prompt schedulinguseHelios().schedulePrompt({ prompt, chunk }). Evolutions land at the next chunk by default; this lets you target a specific one for music cues or beat-synced transitions. See schedule_prompt.
Super-resolution modeuseHelios().setSrScale({ sr_scale }). Toggle between "off", "2x", and "4x". Takes effect on the next chunk; works as either a Setup- or Live-phase control.
Image conditioning strengthuseHelios().setImageStrength({ image_strength }). A 0..1 slider for the Live phase; ignored when no image is set.
For the full design rationale, and the patterns to follow when adding any of the above, read skill/SKILL.md in the example repo.