Installation and setup
Get the example running locally before reading further. Every section below points back at code in the repo you just cloned. You will need:- Node.js 18+.
- pnpm (the example pins lockfiles to pnpm;
npmoryarnwill work but you’ll regenerate the lockfile). - A Reactor API key (starts with
rk_). - Familiarity with the Next.js App Router.
1
Clone the example
The example lives alongside our other reference apps in
reactor-team/js-sdk under examples/.2
Add your API key
Your
rk_… key must never reach the browser; the example reads it server-side and mints a
short-lived JWT for the client. We’ll cover the broker pattern below; for now, drop the key
into .env:3
Install dependencies and start the dev server
http://localhost:3000, click Connect, and pick a starting point: a curated prompt,
an example image, or your own text.How Helios works
Building with Helios is different from calling a typical generative API. There’s no prompt-in / video-out request. You open a long-lived connection, send a prompt, and the model begins producing a continuous stream of 33-frame chunks. You steer it by mutating the prompt while it runs and the model applies each change at the next chunk boundary. Opening the connection isn’t instant. Reactor provisions a GPU for your session, so the client moves through four states before media starts flowing:waiting state is when the GPU is being assigned, which typically takes a few seconds. Once the
status reaches ready, commands take effect and chunks start arriving. See
Sessions for the full breakdown.
At ready the model is connected but idle; it won’t produce frames until you set a prompt and call
start. From there a small set of SDK methods
(setPrompt,
start,
pause /
resume,
reset,
rewind) drives it through the rest of its lifecycle.
Two things to note about those methods:
- They’re asynchronous; events are the source of truth. Calling
setImagedoesn’t mean the next chunk will use it. The model confirms by emittingimage_acceptedwhen the change has actually landed. - Errors arrive out-of-band. A broken precondition like
startwith no prompt surfaces later as acommand_errorevent, not as a thrown exception. The two rewind commands are the exception to the exception: they report onrewind_failedinstead, which names no command.
Authentication
Helios is different from most video-generation APIs. Instead of sending your API key in a header and receiving an image from the server, Helios opens a long-lived WebRTC connection that the server needs to trust for hours. Shipping a rawrk_… key to the browser would hand full account access to
anyone with devtools open. Instead, the Reactor SDK presents a JWT
minted server-side from your API key. Mint it session-scoped (authorization_details with
type: "session"): the token can only start and operate its own Helios sessions, it is short-lived
(1 hour by default, capped at 6), and it is safe to hand to the client. Your rk_… key stays on the
server.
That means every Helios frontend needs one server-side route that mints JWTs. In the example, that
route is app/api/reactor/token/route.ts, a
Next.js route handler
that exchanges your rk_… key for a JWT and hands the JWT back with a Cache-Control header
derived from the token’s actual expiry:
HeliosApp.tsx hands <HeliosProvider> a resolver, not a token string. The SDK calls the
resolver on every Coordinator hop it makes: uploads, clip manifests, ICE refreshes, SDP
renegotiation. A static string 401s those hops the moment it ages out.
app/HeliosApp.tsx
Starting from a prompt
Generation kicks off in two SDK calls:setPrompt registers the prompt at chunk 0, then start
begins producing chunks. PromptComposer.tsx exposes a grid of curated presets and a free-text
input, but every button routes through the same send function:
app/components/PromptComposer.tsx
app/lib/prompts.ts, a curated scene library that’s the single source of
truth for both these prompts and the mid-stream evolutions covered later in the article. Its header
comment is worth reading: each prompt is a full paragraph for a reason, and that style is what lets
the model hot-swap prompts smoothly later.
The example calls set_prompt, a convenience
wrapper that picks the chunk index automatically. If you need to queue a prompt for a specific
future chunk, reach for schedule_prompt
instead.
Starting from an image
Image-to-video adds a second piece of conditioning, and chainingsetImage → setPrompt → start
directly hides a subtle race. setImage carries an upload that the runtime has to resolve before
the model dispatches it; start carries nothing and sails past on the same data channel. The first
chunk is then generated from the prompt alone, no image conditioning at all. The image lands a tick
later and only applies from chunk 1 onward, so the user sees the scene “correct itself” at the first
chunk boundary.
setConditioning (Helios SDK 0.9.0+) is the fix: prompt and image ride on a single data-channel
message. One message can’t be split or reordered, and the model handles it as one transaction. By
the time start reaches the model, both pieces are in place.
ImageStarter.tsx to three calls:
app/components/ImageStarter.tsx
command_error and mutates nothing. Reach for setConditioning whenever both pieces are
known at the same time: curated scene launches, “load this preset” buttons, anything that’s a single
user click.
The example’s second image path is for custom uploads, where the user’s prompt arrives later from a
separate action. It’s shorter still: uploadFile, then setImage. The user types their own prompt
in the composer above and clicks Start, at which point PromptComposer fires setPrompt + start.
No race here either: by the time the human has typed and clicked, the upload has long since been
VAE-encoded.
app/components/ImageStarter.tsx
start() and you need image conditioning, use setConditioning. Only fall
back to setImage alone when the prompt arrives later from a separate user action: the
custom-upload flow above, or a mid-stream image swap.
The example images live in public/ and pair with hand-tuned starting prompts in
app/lib/prompts.ts.
Going live
Once generation starts, the UI flips from the setup panel to its Live phase. The example wires three small components into the right-hand sidebar and main pane: a status badge that tracks the connection lifecycle, a “now playing” panel that mirrors the state snapshot and exposes transport controls, and the video pane itself.StatusBadge.tsx is the user’s window into the four-state connection machine. Every state,
including the multi-second waiting step where Reactor is provisioning a GPU, gets a visible label
and color.
app/components/StatusBadge.tsx
useHelios() is the only hook needed here: status, connect, disconnect, and lastError all
live on it. The button toggles purely on status === "disconnected"; every other state
(connecting, waiting, ready) renders the Disconnect button.
NowPlaying.tsx is the canonical example of how the rest of the app reads model state: subscribe
once with useHeliosState, hold the latest snapshot in useState, read fields off it. No event
aggregation, no derived booleans, no useReducer over chunk_complete events.
app/components/NowPlaying.tsx
pause, resume, and reset are typed SDK methods on useHelios(), same shape as setPrompt and
start from earlier sections: each returns a Promise that can reject with a command_error if its
preconditions aren’t met.
The video pane itself is one component:
app/components/Video.tsx
<HeliosMainVideoView /> is a typed wrapper around <ReactorView track="main_video"> that handles
<video> element setup, srcObject binding, and browser autoplay policy quirks. Style the outer
container; never reach for the underlying element.
Hot-swapping prompts mid-stream
The most distinctive Helios feature is its ability to change the prompt without restarting. The example’s “evolve the scene” picker matches the active prompt against the prompt library and offers one-click continuations.app/components/EvolveScene.tsx
setPrompt call. No start, no reset, no acknowledgment wait. The model
is already generating, and the next 33-frame chunk picks up the new prompt automatically. From the
user’s perspective the scene just keeps going; from the model’s perspective the prompt schedule was
updated for the next chunk boundary.
Rewinding the scene
Prompt swapping steers the video forward. Rewinding steps it back: save a snapshot of the world state, keep generating, and when the scene goes somewhere you don’t want, return to the snapshot and try a different direction. Helios keeps up to 50 snapshots for the life of the session and evicts the oldest once the buffer is full, so the example’sSnapshots.tsx defers to the model’s own list for which rows to show rather
than remembering every snapshot it has saved.
app/components/Snapshots.tsx
snapshot_list is the only way to read the buffer’s contents. The state snapshot that drives
the rest of this app reports which snapshot the generation descends from, in current_snapshot_id,
but not what the buffer holds, so this is the one panel that can’t be a function of state alone. It
calls listSnapshots() when generation starts, then again on every snapshot_saved.
Rewinding doesn’t restore the prompt. rewind rolls back the latent history, the RNG state, and
the chunk index. The active prompt keeps steering. That is the behavior you want for “rewind and go
somewhere else,” but for “rewind and put it back” you re-send the prompt yourself, which is why
rewindTo pairs the two calls.
One operation at a time. rewind queues the restore for the next chunk boundary instead of
applying it on receipt, so a save issued inside that window captures pre-restore state. The panel
locks every button while either operation is outstanding.
The panel also clears itself when the session ends or resets. The buffer belongs to the session: it
starts empty, reset empties it and restarts snapshot_index at 1, and it is released when the
session ends. Stale rows would collide with new ones rather than merely look out of date.
Snapping a clip
The SDK ships recording primitives so you don’t have to wire upMediaRecorder yourself. The
example’s SnapClip.tsx captures the last 10 seconds of the live stream and opens a modal with the
SDK’s built-in preview player and a download button.
app/components/SnapClip.tsx
@reactor-team/js-sdk, not @reactor-models/helios. Recording
is a base-SDK feature. It works identically for every Reactor model, and the typed model packages
don’t re-export the recording surface. So direct base-SDK imports are idiomatic in this one place,
and you can drop the file into any other Reactor example unchanged.
requestClip(durationSeconds) is the whole capture API. It returns a Clip value that you hand
to <ClipPlayer> to preview and <ClipDownloadButton> to save. Neither component needs a getJwt
prop: both inherit the resolver from <HeliosProvider getJwt={…}> through React context, and it
resolves against the same cached /api/reactor/token route from
Authentication, so repeat captures don’t trigger new token mints. Errors come
back as a RecordingError with a typed code and reason, distinct from the command_error
events covered next.
Pass getJwt explicitly in one case only: when the clip UI renders through a portal outside the
provider’s subtree, such as a toast mounted in app/layout.tsx. Capture the resolver with
reactor.getJwtResolver() at action time and thread it down.
Surfacing command_error
Every Helios command can fail a precondition check (e.g.start before a prompt at chunk 0). The
example never lets these fail silently.
app/components/CommandError.tsx
useHeliosCommandError is the typed wrapper for the command_error message: it fires when Helios
rejects a command, carrying the failing command name and a human-readable reason. The component sits
in the sidebar, renders nothing until an error arrives, and clears itself when the next state
snapshot lands so a stale banner can’t pile up.
What’s intentionally left out
Not every Helios feature is surfaced in this demo. See below for a list of what’s missing and the one-line addition that wires each one in.
For the full design rationale, and the patterns to follow when adding any of the above, read
skill/SKILL.md in
the example repo.