Skip to main content
LingBot World 2 conditions on three signals at once: a reference image (the first frame), a text prompt (updatable at any time), and real-time input (movement, look, and the native camera-pose layer). The model only ever sees a single prose string via set_prompt, but production scenes are not written as one string. They are authored as layers, and the application recomposes the prose whenever the input state changes, so the text always matches the motion the model is being asked to render:
  • base carries the world’s identity: subject, environment, style.
  • camera and movement each have a static and a dynamic variant, selected by whether movement input is held.
  • events are detail clauses bound to hold-keys; the clause is appended while the key is held and drops out on release.
  • vertical is the jump / crouch / stand sentence while those controls are engaged.
This is the harness the reference app wires up (composePrompt in lib/lingbot-world-prompts.ts), and the shape the lingbot-world-2-prompts skill emits. The layering is authoring discipline, not a wire format: if your app sends one fixed prompt, compose the idle form yourself (base + camera.static + movement.static) and write those three fragments with the same care. Two facts drive every rule below:
  1. Every fragment is read in the company of the others. Each layer owns one axis: base owns what the world is, camera owns how it is framed and what look-input means, movement owns what the subject is doing, events own what just happened. A fragment that encodes another layer’s axis (a motion verb in the base, a point of view in an event) contradicts that layer’s other variant the moment the input state flips, and the model splits the difference on screen.
  2. The model only gates motion on real input channels. Anything a fragment describes as moving will move on its own, forever. The camera layer carries the input contract (“look-input is the only source of camera motion”), and the static variants state the negative space: what holds still or idles when nothing is pressed.
Length budget. The encoder truncates somewhere around ~2000 characters of composed prose, and events stack. Targets: base ≤600 characters, each camera variant ≤300, each movement variant ≤350, each event ≤500. Check that your worst case (base + dynamic camera + dynamic movement + the two biggest events) stays under ~2000, and trim events first; the base and the contracts must never be the part that falls off the tail.

​
The base layer

One to three sentences: subject + environment + style/atmosphere. Nothing else. No point of view, no camera behavior, no motion verbs, no input language; those belong to layers that swap under the base, and a base that pre-commits to “stands still” or “third-person view” fights the dynamic variants every time the player moves. A production base, from the reference app’s noir alley scene:
A lone uniformed police officer in dark blue tactical gear in a narrow urban alley at night. The world contains EXACTLY ONE tall street lamp on the right at a fixed position AND EXACTLY ONE glowing neon shop sign on the left at a fixed position AND EXACTLY ONE shop door straight ahead at a fixed position AND EXACTLY ONE green dumpster on the right at a fixed position. Dark brick walls, heavy rain falling, shiny puddles on the wet asphalt, yellow police tape, blue and red ambient light. Cinematic noir night, reflective wet surfaces.
The craft rules, all load-bearing:
  • Concrete visual casting. The base is the only place the subject is introduced; give it enough visual detail to survive the whole session. Every other fragment refers back with a definite reference (“the officer”, then pronouns) and never re-describes.
  • Populate event targets. Mention the secondary objects your events will interact with. An event that asserts an interaction with an entity the scene never established doesn’t fail; it conjures the referent into frame to satisfy the sentence.
  • Pin landmark objects with explicit counts. While the player drives around, the model is happy to hallucinate a second street lamp as the view sweeps. For the two to four objects that define the space, state the count and fix the position in the blunt form above (“The world contains EXACTLY ONE … at a fixed position”). Texture (rain, puddles, debris) can stay unpinned.
  • Use-ready props. If an event will have the subject use an object, establish it in the base (and the seed image) in a pose compatible with the use. “Holding a large curved blade” composes with a blade-swing event; “slung across its back” fights it.
  • Describe what is present, never what is absent. The model renders the nouns you give it: “a street with no traffic” places traffic, and negation also hides in innocent-looking words (“empty”, “nothing”, “no one”). Use positive substitutes (“the street ahead stays quiet”). The earned exceptions are the camera layer’s contract language and the disambiguation guards covered under events.

​
The camera layer

Two variants, selected by movement state. These templates are near-verbatim from production scenes; keep their structure and substitute the bracketed parts. They are the sentences that make the world controllable instead of a drifting video. camera.static (no movement input): the subject is centered and the only camera motion is look-input orbiting it.
Third-person view, the [subject] locked at the exact centre of the frame at constant size and distance. Neither the [subject] nor the camera moves on its own; arrow-key look-input is the only source of camera motion, arcing the camera around the stationary, centred [subject] only while held.
camera.dynamic (movement input held): strict rear-view tracking, with look-input reinterpreted as steering.
Strict third-person rear view, the [subject] locked at the exact centre of the frame as the camera holds a fixed position behind [it] and tracks [it] forward. The camera does not rotate around the [subject]; look-input becomes the [subject] changing heading.
Notes:
  • Never use unqualified motion verbs for the camera. “The camera orbits” reads as continuous self-motion; “look-input … arcing the camera … only while held” ties the motion to the input channel.
  • Keep the template sentences exact. On current weights a paraphrase reads as a different instruction, not the same one said another way. Find the form your world responds to and reuse it verbatim in both variants and across sessions.
  • Weapon and prop framing guard. When the subject handles items the model loves to render in first person (guns, tools), add a guard to both variants: “any weapon in his hands visible ahead — never a first-person view.”
  • First-person scenes (a place, a cockpit, hands on a tool) adapt the same pair around a named foreground anchor instead of a subject: static keeps the anchor centered with look-input orbiting it; dynamic becomes “Strict first-person view, the [anchor] holding steady at the centre of the frame as the viewpoint advances through the scene; look-input becomes the heading changing.”

​
The movement layer

Two variants, same switch. Don’t restate the subject’s appearance or the environment here; the base already said it. Spend the words on behavior. movement.static is the idle. It is the negative space of the motion contract and the rest state every event returns to, so it earns real craft: the subject is stationary but never inert. Name two or three specific micro-motions bound to the subject, and keep the pose interaction-ready:
The officer stands still on the wet asphalt, weight settled, only his shoulders rising and falling with slow breaths as rain streams off his tactical gear and drips from his fingertips into the spreading puddles.
Without a specific idle, the model animates rain, fish, and traffic on its own and the keys stop feeling causal. “Everything is static” doesn’t work; “the horse’s breath steaming faintly and one hoof shifting in the mud” does. movement.dynamic is travel: concrete motion verbs, ground contact, and the environment responding (“boots splashing through the puddles and kicking up fine spray”). When a side-on view of the subject is plausible (animals, mounts, characters), re-assert the rear view inside the movement prose by describing the geometry the camera contract implies: “moving directly away from the camera so the pair stay in strict rear view — the horse’s hindquarters, rump, and streaming tail toward the viewer.” The camera layer states the rule; the movement layer showing the same geometry from the subject’s side is what holds it.

​
Events: the hold keys

Each event is a name (it labels the key chip) plus a detail clause appended to the composed prompt while its key is held. Events stack: any two may be held together, so each detail must make sense next to any other and never depend on another event being absent. An event’s detail can also be a {static, dynamic} pair when the event looks different mid-motion; use the split only then. Recurring shapes, mixable within one scene (group them so the keys read as a kit, or as escalating story beats): Discipline for every event:
  • Definite reference only. “The officer”, then pronouns; never the base’s full introduction and never a second scene-setting sentence. The base casts the subject once; a re-description is a casting call, and the model answers it with a duplicate.
  • Stay in your layer. Don’t restate the camera contract and don’t encode a movement state (“the officer stands still and fires”); the layers own those. The one allowed restatement is the contextual-entrance opener: “The original focal subject remains the main subject, unchanged in pose and position, while …”.
  • Disambiguation guards. When a noun has a known failure mode, spend a clause steering away from it, placed next to the risky noun. From the noir alley’s rocket launcher: “an RPG: a long, heavy cylindrical steel tube resting across his shoulder … not a handheld gun.” One guard per real risk. This is the earned exception to the no-negation rule: a guard against a specific misreading, never scene-painting by absence.
  • Keep every claim frameable under the camera contract. An oversized entrance happens at a distance the framing can contain (down the alley, on the horizon), never “towering above” the subject. Given an impossible framing, the model breaks the camera rather than refuse the sentence.
  • End settled. Action beats settle back toward the idle (“rising smoothly back to his feet”) so releasing the key reverts to a clean idle. Transformations and chains move the world forward by design, but still end each in a new stable state, never in open-ended accelerating motion.
A complete production event (the noir alley’s key 1):
The officer raises his service pistol in both gloved hands, arms extended ahead of him, and fires down the dark alley; the muzzle flash lights the falling rain, the recoil kicks the pistol back in his grip, and a spent casing clatters onto the wet asphalt.

​
Vertical prompts: jump, crouch, stand

Three per-scene sentences, appended as the vertical segment while jump or crouch is engaged. Write the jump for this subject, and shape it as a complete symmetric arc: launch, airborne moment, return to the ground. The runtime’s camera motion is a symmetric up-then-down arc, so prose that never lands fights the camera coming back down.
The officer springs upward off both feet, leaping high off the wet asphalt, his boots lifting clear of the ground before he drops back down and lands in a low crouch.
The crouch and stand sentences are camera-height moves, and the stock strings are reused verbatim across production scenes (“The camera lowers toward the ground as the character crouches down low …” / “The character straightens back up out of the crouch …”). Adapt them only when the subject makes them absurd (a vehicle).

​
Layer versions

Each layer is a registry keyed by version id, and an event can select versions (baseVersion / cameraVersion / movementVersion; omitted means default). Two patterns earn the machinery:
  • Portal world. A key that teleports the session to a different world: register a second full base (base.portal_world) and tag the event with it. While held, the composed prompt swaps the whole base; events authored against other bases are suppressed, since they can’t share coherent prose.
  • Scene-replace event. An event whose detail is a complete self-contained prompt: register empty versions on all three layers and tag the event with all three. Its prose becomes the only content sent. Use this to preserve a known-good standalone prompt verbatim.
A big event may also swap just the camera or movement version to a registered compact variant, freeing budget for its own detail without forcing other held events off the stack.

​
Directed camera moves

The set_camera_pose layer conditions the camera with per-frame motion deltas, but it is a bias, not a rig: LingBot is a world model with no ground-truth camera. If the composed prompt holds a stable centered subject, a pose-only move fights the text conditioning and tends to drag the subject along with the camera. Pair a sustained directed move with a one-sentence prompt hint, composed onto the active prompt for the move’s duration, that says the same thing in the prompt’s own vocabulary:
The camera orbits steadily around the subject, which stays perfectly still at the exact centre of the frame at constant size and distance as the viewpoint circles it.
Keep rotation gentle (peak ≤ ~0.05 rad/frame); rotation deltas are raw per-frame velocities, so a subtle-looking number compounds over a chunk. Translation magnitude is normalized away per chunk (only its direction and within-chunk shape survive; see the schema), so shape translation by duration rather than size. Strip the hint verbatim when the move releases so hints never stack. Short, input-driven pose motion (a mouse-look flick, a jump arc) needs no hint; reserve prompt pairing for sustained directed moves.

​
The seed image

A session starts from a prompt and an image together, and when they disagree the image wins, with the loss showing up as artifacts rather than an error. So derive the image from the layers rather than writing it from scratch:
  • Start from the base: every noun, the pinned landmarks in their stated positions, the atmosphere and style tags carry over unchanged.
  • Take the framing from camera.static, recast as a still (“A third-person still frame of [subject] centred in frame at medium distance …”), dropping every input clause.
  • Take the pose from movement.static, recast as plain description.
  • Make sure every prop the events will use is visible, in the use-ready pose the base describes.
Render it with any text-to-image model at a 16:9 landscape frame (the model runs at 1664 × 960).

​
Keeping conditions aligned

Most bad output is a conflict between the three signals. Before blaming the model, check all three:
  • Text vs image. If the base describes a subject or materials that differ from the reference image, the world drifts mid-generation as the model tries to satisfy both. Describe what the image shows; regenerate the image if you want something else.
  • Text vs input. Motion described in prose runs regardless of input. A motion verb that belongs in movement.dynamic but leaks into the base keeps running after you send idle; a camera verb outside the contract fights your look input.
  • Text vs text. A fragment that contradicts another (an event appending “the sky clears” over a base that says “overcast”) hands the model two worlds and it splits the difference. Fix the fragment; don’t stack a correction on top.

​
Runtime caveats

  • Keep holds brief; keep the world settled between them. The model conditions on its own recent frames, so a long continuous hold accumulates drift (identity morphs, colors smear). Short presses with a settled world between them stay clean, and a released key cannot restore a world that has already drifted; that is why every event ends settled.
  • Restart before judging a prompt fix. Once a bad frame enters the stream, the model can repaint the same artifact chunk after chunk whatever the text now says. Start a fresh session from a clean frame before you evaluate a change.
  • Intent qualifiers do nothing. Words that state your intent rather than the frame (“correctly”, “make sure that”, “without cutting away”) are invisible to a renderer. If a distinction matters, it has to exist as different composed text that your own logic selects.
  • Fresh sessions ignore input at first. A new world spends its first several seconds materializing the scene, and input sent inside that window is dropped. Schedule nothing that matters until the first stable frames.
  • Movement reliability. Forward is the most stable movement axis; lateral strafing is the least, since it displaces the camera off a centered subject. Route sideways motion through turning where you can.
  • Model limitations. A well-formed scene won’t always land. If the alignment checks above pass and it still isn’t there, you may be at a current model limit rather than a prompt problem.

​
See also

  • The lingbot-world-2-prompts skill turns a one-line idea into a complete layered scene JSON (plus a derived seed-image prompt) in the exact StructuredExample format the reference app loads; it also documents the legacy actions.json export format.
  • LingBot World 2 tutorial: the harness that composes and re-sends these layers, wired up in a real app
  • LingBot World 2 schema: every command, event, and state field
  • LingBot World 2 overview: model name, features, lifecycle