Skip to main content
The public reference examples include a gateway for NVIDIA RoboLab’s DreamZero client. The gateway exposes an OpenPI-compatible WebSocket endpoint; RoboLab runs the simulator and scores episodes, while Reactor runs inference. The client code is public. Contact Reactor if you need hosted model access.

Prepare the gateway

After client setup, move from robotics/sim/notebooks/ to robotics/sim/dreamzero/:
check_wiring.py runs without an API key or simulator and checks the protocol, camera mapping, state handling, and reset behavior. The gateway command requires REACTOR_API_KEY. The example binds to localhost; choose an appropriate --host address if the simulator runs on a separate machine.

Run RoboLab

Install RoboLab and its supported Isaac Sim environment separately. The simulator needs graphics-capable hardware; the gateway itself needs no GPU. From RoboLab’s root in that environment, run:
Use the gateway machine’s address for a remote simulator. Follow RoboLab’s own installation and container launcher instructions for its Python interpreter and assets. The reference configuration executes a full 24-row chunk at 15 Hz. Keep --num-envs 1: one Reactor session keeps the model’s history for one episode. Multiple environments require separate sessions. RoboLab’s default black second exterior view maps to exterior_2; exterior_1 remains the real primary camera. The gateway connects to Reactor when it receives the simulator’s first observation. For each simulator request, it sends one frame per camera rather than repeatedly publishing the same images between requests. It discards replies with an obs_seq it has already seen. This filters repeated replies, but does not prove that a prediction used a particular camera frame. Expect a gateway listener, a connection on the first simulation request, episode_started, then (24, 8) chunks. RoboLab writes evaluation output under its own output/ directory. A passing wiring check verifies transport behavior; task success must be measured in the simulator.