Get your first actions

Request one RoboCasa365 action chunk using synthetic observations. This public model needs no separate model-access grant.

Install the reference client

New to Reactor? How the API works explains the session, camera streams, commands, and action messages used below. You need Python 3.12, uv, a Reactor API key, and Reactor’s public cookbook. The client code is public; your Reactor account still needs access to the selected hosted model. No local model weights or GPU are required for this example.

git clone https://github.com/reactor-team/reactor-cookbook.git
cd reactor-cookbook
cd robotics/sim/notebooks
uv sync --frozen --python 3.12
export REACTOR_API_KEY='your-api-key'

Run the example below from robotics/sim/notebooks/. This environment installs the Python SDK (reactor-sdk, imported as reactor_sdk) and the reactor_robotics helper directory in Reactor’s public cookbook. The helpers provide camera publishing and message queues; they are separate from the SDK. Keep your API key in the environment. On Linux, the published glibc wheel requires glibc 2.34 or newer. The default endpoint is https://api.reactor.inc; REACTOR_API_URL overrides it. Examples select a model by slug, not a frozen release; check its release notes when the deployed contract changes. If setup or connection fails, see first-run troubleshooting.

Request a chunk

Run the cookbook example from the current robotics/sim/notebooks/ directory:

uv run --frozen python first_xr1_robocasa_actions.py

import asyncio
import json
import time

import numpy as np
from reactor_robotics.session import ReactorSession

TRACKS = ("left_agentview", "right_agentview", "wrist_view")


async def main():
    session = ReactorSession("reactor/xr1-robocasa365", fps=15, frame_size=(256, 256))
    try:
        await session.connect(TRACKS, subscribe=("action_prediction",))
        await session.send("set_task_description", {
            "task_description": "close the blender lid"
        })
        session.set_frames({
            name: np.full((256, 256, 3), 40 + 60 * i, dtype=np.uint8)
            for i, name in enumerate(TRACKS)
        })
        # A static synthetic observation makes every history slot identical.
        await asyncio.sleep(0.8)
        rows = [[0.0] * 14 for _ in range(4)]
        await session.send("set_state_history_json", {
            "state_history_json": json.dumps({"state_history": rows}, allow_nan=False)
        })
        started = time.perf_counter()
        await session.send("set_executed_step_json", {
            "executed_step_json": json.dumps({"step": 0})
        })
        reply = await session.next_message("action_prediction", timeout_s=90)
        actions = np.asarray(reply["action"], dtype=np.float64)
        assert actions.shape == (16, 60) and np.isfinite(actions).all()
        assert reply["step"] == 0
        print("actions:", actions.shape)
        print("first simulator action:", actions[0, :12])
        print("request-to-reply ms:", round((time.perf_counter() - started) * 1000, 1))
    finally:
        await session.close()


asyncio.run(main())

Expect actions: (16, 60) and 12 active values in the printed simulator action. The first execution echo is required even though no actions have been executed. The model waits for at least four complete camera sets while the helper continues publishing. Constant synthetic images and zero state check connectivity and reply shape; do not execute these outputs on hardware. Repeating frames is appropriate only for this artificial static history. A rollout requires aligned camera and state history. Printed timing starts at the execution echo, excluding connection and frame warmup. Next, use the integration guide and history contract.

Get started

Models

Get your first actions

Install the reference client

Request a chunk

​Install the reference client

​Request a chunk

Install the reference client

Request a chunk