> ## Documentation Index
> Fetch the complete documentation index at: https://docs.reactor.inc/llms.txt
> Use this file to discover all available pages before exploring further.
# LingBot tutorial
> An end-to-end LingBot walkthrough against the open-source reference frontend.
A guided tour of the open-source
[LingBot Interactive](https://github.com/reactor-team/js-sdk/tree/main/examples/lingbot)
reference app, which demonstrates every important pattern in the LingBot SDK. By the end you'll know
how to start a scene from an image, drive it with WASD, snap clips, and surface model errors.
## Installation and setup
Get the example running locally before reading further. Every section below points back at code in
[the example repo](https://github.com/reactor-team/js-sdk/tree/main/examples/lingbot). You will need:
* Node.js 18+.
* [pnpm](https://pnpm.io/installation) (the example pins lockfiles to pnpm; `npm` or `yarn` will
work but you'll regenerate the lockfile).
* A [Reactor API key](/authentication) (starts with `rk_`).
* Familiarity with the [Next.js App Router](https://nextjs.org/docs/app).
The example lives alongside our other reference apps in
[`reactor-team/js-sdk`](https://github.com/reactor-team/js-sdk) under `examples/`.
```bash theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}}
git clone https://github.com/reactor-team/js-sdk
cd js-sdk/examples/lingbot
```
Your `rk_…` key must never reach the browser; the example reads it server-side and mints a
short-lived JWT for the client. We'll cover the broker pattern below; for now, drop the key
into `.env`:
```bash theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}}
cp .env.example .env
# then edit .env and set REACTOR_API_KEY to your API key
```
See a "Setup Required" screen? Your `REACTOR_API_KEY` isn't loaded. The check lives in
`app/page.tsx` → `app/SetupRequired.tsx`.
```bash theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}}
pnpm install
pnpm dev
```
Open `http://localhost:3000`, click **Connect**, pick a curated scene or upload your own
image, and drive it with WASD.
## How LingBot works
Building with LingBot is different from calling a typical generative API. There's no image-in /
video-out request. You open a long-lived connection, send a seed image plus a paragraph-length
prompt, and the model begins producing a continuous stream of chunks that you steer in real time
with WASD. The image anchors the scene and is locked at start; the prompt and movement axes drive
everything that happens after.
Opening the connection isn't instant. Reactor provisions a GPU for your session, so the client moves
through the same four states as every other Reactor model before media starts flowing
(`disconnected → connecting → waiting → ready`). See
[Sessions](/concepts/sessions#connection-lifecycle) for the full breakdown.
Three properties of the LingBot API are worth internalizing before you read on, since the rest of
this tutorial assumes them:
* **Commands are asynchronous; events are the source of truth.** Calling `setImage` doesn't mean the
next chunk uses it; the model confirms by emitting `image_accepted` when the upload has been
decoded and is ready to use.
* **Errors arrive out-of-band.** A broken precondition like `start` before `setImage` surfaces later
as a `command_error` event, not as a thrown exception.
* **Movement axes are persistent state, not pulses.** `set_movement` and the two look axes hold
their last value forever; every keydown needs a matching keyup that sends `idle`.
## Authentication
LingBot uses the same broker pattern as every browser-side Reactor app: your `rk_…` key stays on the
server and the client receives a short-lived JWT minted from it. The server-side route at
`app/api/reactor/token/route.ts` and the mount-time fetch in `LingbotApp.tsx` are
character-for-character the Helios setup with the provider swapped to ``. See
[Authentication](/authentication) for the full concept page, including the Express equivalent and
the Python path that skips the broker entirely.
## Starting a scene from an image
The canonical LingBot launch flow lives in `ScenePicker.tsx`. Picking a curated scene fires a
five-step sequence: `uploadFile → setImage → await image_accepted → setPrompt → start`. The wait in
the middle is the part that matters. `setImage` carries an upload the runtime has to decode and
VAE-encode, but `start` carries nothing and sails past on the same data channel. Skip the wait and
the first chunk is generated from the prompt alone, with the image landing one chunk later. The
scene visibly "corrects itself" at the first chunk boundary.
LingBot doesn't have Helios's atomic `setConditioning`; the explicit wait is the answer. The example
uses `useLingbotImageAccepted` with a one-shot ref resolver to gate `setPrompt + start` on the right
event:
```tsx app/components/ScenePicker.tsx theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}}
const { uploadFile, setImage, setPrompt, start } = useLingbot();
// Park the resolver BEFORE calling setImage. If we registered it
// after, the model's ack could land first and we'd miss it.
const imageReadyRef = useRef<(() => void) | null>(null);
useLingbotImageAccepted(() => {
if (imageReadyRef.current) {
imageReadyRef.current();
imageReadyRef.current = null;
}
});
async function startScene(scene: Scene) {
const blob = await fetch(scene.imageUrl).then((r) => r.blob());
const ref = await uploadFile(blob, { name: `${scene.id}.jpg` });
const imageReady = new Promise((resolve) => {
imageReadyRef.current = resolve;
});
await setImage({ image: ref });
await imageReady; // ← the load-bearing line
await setPrompt({ prompt: scene.prompt });
await start();
}
```
The Promise wrapper around `imageReadyRef` is the standard pattern for waiting on an event-bus event
from inside an async function. The hook callback is the resolver; the ref makes it one-shot. Without
the ref reset, a second `image_accepted` from a later run would resolve a Promise nobody is
awaiting.
The curated scenes live in `app/lib/scenes.ts`. Each entry pairs a hand-tuned starting prompt with a
reference image in `public/`. The prompts follow the rules in the [Prompt Guide](/model-api-reference/lingbot/prompt-guide)
above: FOV and subject declared up front, near / mid / far object layers filled in, position-only
camera framing, one atmosphere phrase.
Helios SDK 0.9.0+ has [`setConditioning`](/model-api-reference/helios/schema#set_conditioning), an atomic
command that bundles `setImage + setPrompt` so the race in the first paragraph can't happen.
LingBot has no analogue today; the `image_accepted` wait is the recommended pattern.
## Custom uploads
`CustomStart.tsx` handles the second launch path: the user uploads their own image and types their
own prompt. The trick here is to upload as soon as the file is picked, so `image_accepted` lands
while the user is still typing. When they click Start, the example fires `setPrompt + start`
directly; no `await` needed, because the human typing delay is orders of magnitude longer than the
image decode. Contrast with [Starting a scene from an image](#starting-a-scene-from-an-image), where
a one-click launch has to bridge that gap explicitly.
The two halves of the flow are wired to different events:
```tsx app/components/CustomStart.tsx theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}}
const { uploadFile, setImage, setPrompt, start } = useLingbot();
const [text, setText] = useState("");
// File-pick handler: upload immediately, await image_accepted.
async function uploadCustomImage(file: File) {
const imageReady = new Promise((resolve) => {
imageReadyRef.current = resolve;
});
const ref = await uploadFile(file);
await setImage({ image: ref });
await imageReady;
}
// Start button: image is already accepted, so just prompt + start.
async function startCustom() {
if (!hasImage || !text.trim()) return;
await setPrompt({ prompt: text.trim() });
await start();
}
```
The Start button derives its disabled state from the snapshot:
```tsx theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}}
const hasPrompt = snapshot?.has_prompt === true || text.trim().length > 0;
const hasImage = snapshot?.has_image === true;
```
Reading `has_image` off the snapshot rather than tracking a local boolean keeps the UI honest across
edge cases (a `reset()` from elsewhere in the app, a disconnect mid-upload). The
[`state` payload](/model-api-reference/lingbot/schema#state-payload) is the canonical source for what the model thinks is set.
## Going live
Once `snapshot.started === true`, the setup panels (`ScenePicker` and `CustomStart`) hide and the
live UI takes over: a status badge, a now-playing panel with transport controls, and the video pane.
`StatusBadge.tsx` is the user's window into the four-state connection machine. Every state,
including the multi-second `waiting` step where Reactor is provisioning a GPU, gets a visible label
and color:
```tsx app/components/StatusBadge.tsx theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}}
import { useLingbot } from "@reactor-models/lingbot";
const TONE = {
disconnected: { dot: "bg-zinc-500", label: "Disconnected" },
connecting: { dot: "bg-amber-400 animate-pulse", label: "Connecting…" },
waiting: { dot: "bg-amber-400 animate-pulse", label: "Waiting for GPU…" },
ready: { dot: "bg-active", label: "Connected" },
};
export function StatusBadge() {
const { status, lastError, connect, disconnect } = useLingbot();
const idle = status === "disconnected";
return (
);
}
```
`useLingbot()` exposes `status`, `connect`, `disconnect`, and `lastError`. The Connect / Disconnect
toggle is purely on `status === "disconnected"`; every other state renders Disconnect.
`NowPlaying.tsx` is the canonical example of how the rest of the app reads model state: subscribe
once with `useLingbotState`, hold the latest snapshot in `useState`, read fields off it. No event
aggregation, no derived booleans, no `useReducer` over `chunk_complete` events.
```tsx app/components/NowPlaying.tsx theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}}
const { status, pause, resume, reset } = useLingbot();
const [snapshot, setSnapshot] = useState(null);
useLingbotState((msg) => setSnapshot(msg));
// The SDK doesn't emit a final `state` message on disconnect, so we
// clear ourselves. Otherwise the next session inherits the old one.
useEffect(() => {
if (status !== "ready") setSnapshot(null);
}, [status]);
// Phase switch: while not started (or after reset), render null and
// let the setup panels take over.
if (status !== "ready" || !snapshot?.started) return null;
return (
<>
{snapshot.current_prompt}
chunk {snapshot.current_chunk}{snapshot.current_action || "still"}
{snapshot.running ? (
) : (
)}
);
```
`current_action` is a LingBot-specific snapshot field: a `+`-joined composite like `"w+left"` that
reflects what the model is currently moving / looking. It updates per chunk, so it lags the user's
key presses by one chunk; that's fine for a status readout, but as
[Driving the scene with WASD](#driving-the-scene-with-wasd) covers, it's the wrong source for button
highlights.
The video pane itself is one component:
```tsx app/components/Video.tsx theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}}
import { LingbotMainVideoView } from "@reactor-models/lingbot";
export function Video() {
return (
);
}
```
`` is a typed wrapper around `` that handles
`
LingBot doesn't have Helios's atomic `setConditioning`; the explicit wait is the answer. The example
uses `useLingbotImageAccepted` with a one-shot ref resolver to gate `setPrompt + start` on the right
event:
```tsx app/components/ScenePicker.tsx theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}}
const { uploadFile, setImage, setPrompt, start } = useLingbot();
// Park the resolver BEFORE calling setImage. If we registered it
// after, the model's ack could land first and we'd miss it.
const imageReadyRef = useRef<(() => void) | null>(null);
useLingbotImageAccepted(() => {
if (imageReadyRef.current) {
imageReadyRef.current();
imageReadyRef.current = null;
}
});
async function startScene(scene: Scene) {
const blob = await fetch(scene.imageUrl).then((r) => r.blob());
const ref = await uploadFile(blob, { name: `${scene.id}.jpg` });
const imageReady = new Promise