Skip to main content
Use Cosmos3 Nano Sim Transfer when you have already recorded a trajectory and want to see it with a different visual appearance. You supply the control video and a scene description; Reactor streams the generated RGB frames back to your application.

​
Run the example

The recorded-control cookbook example includes a synthetic fixture generator, a hosted Python client, bounded readiness checks, and a saved preview snapshot. Start with its depth video, then substitute your own simulator output.

​
Upload and render

  1. Export a supported control video from your simulator or preparation tool.
  2. Connect a Reactor session and upload the complete control clip using the SDK upload API.
  3. Wait for source readiness, then send generate with a scene caption and output settings.
  4. Inspect main_video and wait for done or failed.
  5. While idle, reuse the same source with another caption to compare appearance variants.
These are operations within one session, not separate render-job HTTP endpoints. The API does not return a durable job ID or a downloadable training-video artifact.

​
Prepare the input

Upload a complete video with a video MIME type, up to 128 MiB. It must contain at least the requested number of frames at 30 or 29.97 FPS. This is the input/playback cadence, not a guarantee of generation throughput. Prepare one supported control representation yourself: depth, edge, blur, or seg (segmentation). Start with depth exported from your simulator.

​
What control means

control declares the representation already present in your uploaded video. It does not extract that representation from ordinary RGB footage. Prepare the matching pixels before uploading: These are learned visual constraints, not guarantees of exact geometry or label preservation. Selecting seg does not perform segmentation, and selecting depth does not estimate depth. The incremental episode interface accepts depth only; these four choices apply to generate. Once a fresh state_update after your upload reports has_source: true and generating: false, send the generate command. This example is its JSON payload, not an HTTP request body:
Output sizes are respectively 832×480, 736×544, 640×640, 544×736, and 480×832. Uploaded controls are directly resized. Their aspect ratio must match the selected output within 3%; mismatches are rejected. Account for resizing when comparing labels.

​
Follow generation

The main_video track carries RGB preview frames as they become available. state_update reports source readiness and whether generation is active. done reports the number of generated frames queued for playback; the stream may still be draining. failed reports an input problem discovered during generation. Invalid commands are refused, including busy, missing_control, and invalid_input. Upload only while idle and validate the video type and 128 MiB size limit locally. With Python SDK 1.10.0, upload_file can return a file reference even when the model rejects an upload; upload-hook refusal codes are not reliably delivered to the caller. A file reference confirms transport, not model acceptance. Wait for a fresh source-readiness update with a timeout. Do not reuse a cached has_source: true flag: a rejected replacement can leave the previous source intact. A video MIME type alone does not establish that the clip contains usable frames. Decode, frame-count, FPS, and aspect validation can still emit failed during generation. Handle this separately from a command refusal. The cookbook example starts from a reset state to avoid silently reusing an old source. Wait until the request is no longer generating before starting another run. You can reuse the uploaded control with a different prompt. reset stops generation at a safe boundary, clears the control, and flushes pending playback; upload again before the next request. There is no pause command.

​
Next steps

Connect to a simulator to supply depth incrementally, or generate appearance variants to explore lighting and materials for a recorded trajectory.