reactor.sendCommand() or
the typed SDK. This page describes the reactor/genie-4 contract. See the
overview for a quick start.
Tracks
There are no input tracks. Attach both outputs before starting a world. Handle blocked audio
playback with a user control. A connected track alone does not establish that the world is live or
that the browser plays its sound.
Session lifecycle
One session holds at most one world at a time. The phases areidle, starting, live, ending,
ended, and failed. Call start_world from idle, ended, or failed. The first generated
picture moves the world from starting to live.
A world lasts up to 60 seconds live. end_world stops it early and leaves the Reactor session open.
Starting again creates a new world. Supply the prompt or image again, because a world cannot resume.
Subscribe to session_state before connecting, then await get_state for a fresh snapshot. Clear
local snapshots when disconnected. After reconnecting, read get_state again before enabling world
controls.
Commands
The JavaScript SDK reports refused commands through
error, or React lastError, with
operation: "sendCommand". Calls resolve undefined on failure and do not reject. For the three
commands without a reply body, success also resolves undefined. The Python SDK raises
ReactorError for a refusal. See Refusals.
The React examples below render inside Genie4Provider. Its provider declares the model and tracks.
See the tutorial for authentication and media playback.
start_world
Start from idle, ended, or failed. Supply a nonblank prompt, an image, or both. The prompt and
image apply only at world start. Use instruct for later scene changes.
Upload the image through
uploadFile() and use the returned reference. Reference images work best
at a 16:9 aspect ratio; other shapes are center-cropped. Do not pass a file path or a URL in
reference_image. For an image-only start, omit prompt or pass prompt: "". Do not set it to
undefined or null.
phase: "live". A later failure appears as phase: "failed" with last_error. A refused
start uses busy, invalid_input, or invalid_image.
set_keys
Replace every key this client holds. Valid keys are w, a, s, d, e, literal space " ",
arrowup, arrowdown, arrowleft, and arrowright.
Send an update whenever a key is pressed or released. Send
[] when the final key is released.
Release held keys when someone switches tabs or apps, or clicks away from the controls. Also release
them when a mouse or touch press ends or is canceled. Ignore keyboard presses while someone types
in a text field.
Keys sent during starting take effect when the world becomes live. Keys sent before starting a
world or after it ends are ignored.
instruct
Change the scene or give a character a line while the world is live. A new instruction replaces
one still being delivered.
not_live outside live, or invalid_input for blank text.
end_world
Stop the current world and await its session_state reply. This command accepts an empty object. It
is valid in every phase and safe to repeat. If no world runs, it returns the settled state. When it
stops a world, the phase becomes ended with end_reason: "ended_by_client".
reply.phase. The base JavaScript result
uses reply.data.phase. Use the awaited reply for this request. Keep subscriptions for broadcasts.
get_state
Read the current session_state snapshot in any phase. This command accepts an empty object.
Refusals
Schema validation can also report
invalid_command before the handler runs, for example for a value
with the wrong type. The four codes above are model-handler refusals.
These refusals use the SDK error path. There is no command_error broadcast. Keep them separate
from connection status and from later world failures in last_error. See
Surfacing refusals for React handling.
Messages
Broadcasts reach every connected client. A command reply goes to the client that sent the command.
Base SDK messages use
{ type: "session_state", data: snapshot }. Typed hooks and typed replies
expose snapshot fields directly, such as state.phase.
session_state payload
The receiving flags describe the model media output. The browser can still block playback. For a
countdown, use
Math.max(0, max_seconds - elapsed_seconds).
end_reason values
last_error fields
World failure codes include
upstream_capacity, upstream_unavailable, upstream_rejected,
upstream_interrupted, upstream_auth, upstream_protocol, content_policy, no_stream,
media_stalled, and platform_failed. Show reason and use retryable when offering another
start.