Skip to main content
Return video frames and audio samples on an Output from process_output(), or pass them to await self.emit(...) from a hand-written run(). The runtime encodes the media, sends it over WebRTC, and adjusts bitrate as network conditions change.

​
Transport

WebRTC carries live media between your model and each client. They negotiate compatible codecs and use ICE to establish connectivity, DTLS to negotiate encryption keys, and SRTP to encrypt media. The runtime handles these protocols and congestion control. Delivery time depends on the network, encoding, and client playback. The runtime also answers the WARP opt-ins, which shorten this handshake. A client that asks for them reaches a live connection in fewer round trips. Commands and messages between the client and model travel over a WebRTC data channel alongside the media streams.

​
Video codecs

Reactor supports multiple video codecs, negotiated automatically with each client via SDP: The table is the runtime’s preference order: it takes the first of these that appears in the client’s offer. VP9 leads for its balance of quality and compression, with H.264 close behind for maximum device compatibility. Encoding is done in software.

​
Audio codec

Audio is encoded with Opus at 48 kHz. Opus is the standard WebRTC audio codec and is supported by all modern browsers.

​
Resolution

There are no hardcoded resolution limits. The runtime encodes and streams whatever resolution your model produces. If your model yields 720p frames, the stream is 720p. If it yields 4K, the stream is 4K.

​
Bitrate

The runtime uses adaptive bitrate control based on network conditions. It monitors transport-level feedback and adjusts encoding bitrate dynamically, scaling between 500 kbps and 10 Mbps depending on available bandwidth.

​
Multiple tracks

A model can output any combination of video and audio tracks. Each field on your Output class becomes a separate media track:
Track names (main_video, secondary_video, main_audio) are the identifiers clients use to subscribe to specific streams.

​
When the model falls behind

Each connection plays out at the rate its chunks are tagged with. When a tick comes round and the next frame is still being generated, the client holds the frame it already has and the wire stays quiet, which keeps the bandwidth for real content. A single black frame marks each boundary: the start of a connection, or a flush(). That is what gives a client a clean opening frame. In the other direction, emitting throttles a model that generates faster than the playout rate: the step loop waits before the next step, and a hand-written run() waits inside emit(). See Buffered latency.

​
Packet loss recovery

The runtime enables RTP retransmission (RTX) by default. Lost packets are retransmitted from a short buffer (200 ms), keeping streams clean without adding latency for packets that arrive normally.