> ## Documentation Index
> Fetch the complete documentation index at: https://docs.reactor.inc/llms.txt
> Use this file to discover all available pages before exploring further.

# Audio

> Emit audio tracks and keep them in sync with video.

Audio tracks work like video tracks. Declare them on your `Output`, emit samples alongside frames,
and the runtime handles encoding, transport, and synchronization.

## Declaring an audio track

```python theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}}
from reactor_runtime import Audio, Output, Video


class MyOutput(Output):
    main_video: Video
    main_audio: Audio
```

Emit both together from the run loop:

```python theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}}
async def run(self) -> None:
    while True:
        await self.connected.wait()
        while self.connected.is_set():
            video = self.pipe.forward(prompt=self.prompt)
            audio = self.pipe.generate_audio()
            await self.emit(
                MyOutput(
                    main_video=video,  # (H, W, 3) uint8
                    main_audio=audio,  # (1, N) int16
                )
            )
```

Every track declared on the class must be supplied on every `emit()`. When a tick produces no audio,
send an empty array rather than omitting the field:

```python theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}}
silence = np.zeros((1, 0), dtype=np.int16)
```

## Audio format

| Property    | Value                                     |
| ----------- | ----------------------------------------- |
| dtype       | `int16`                                   |
| Shape       | `(channels, samples)` — `(1, N)` for mono |
| Sample rate | 48,000 Hz                                 |
| Channels    | 1 (mono)                                  |

The runtime encodes to Opus for transport. 48 kHz is the Opus native rate, so audio already at 48
kHz needs no resampling.

### Custom sample rates

If your model generates at a different rate, subclass `Audio` and declare it:

```python theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}}
from reactor_runtime import Audio


class Audio16k(Audio):
    sample_rate = 16_000


class MyOutput(Output):
    main_video: Video
    main_audio: Audio16k
```

The runtime resamples to 48 kHz for transport. Declaring the true rate matters — it is what makes
the resampling correct and keeps playback at the right pitch and duration.

## Keeping audio and video in sync

When you emit an output carrying both tracks, the runtime keeps them aligned. If the video payload
is a batch, the audio is split proportionally across those frames.

The rule that follows: **each emit should carry the audio that belongs to its video**. At 30 fps,
one video frame is `48000 / 30 = 1600` samples, so a batch of 3 frames should carry about 4,800
samples (100 ms).

Emitting a frame's worth of video with a second's worth of audio will drift, however well the
transport behaves.

## Rate-matching two streams

Models whose audio and video come from separate producers need to pair them before emitting. Buffer
whichever stream runs ahead, and emit only when both have material for the same span:

```python theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}}
async def run(self) -> None:
    while True:
        await self.connected.wait()
        audio_pending: dict[int, np.ndarray] = {}
        video_pending: dict[int, np.ndarray] = {}
        next_block = 0

        while self.connected.is_set():
            kind, index, data = await self._next_block()
            if kind == "audio":
                audio_pending[index] = data
            else:
                video_pending[index] = data

            # ✅ Emit only once both tracks have the same block
            while next_block in audio_pending and next_block in video_pending:
                await self.emit(
                    MyOutput(
                        main_video=video_pending.pop(next_block),
                        main_audio=audio_pending.pop(next_block),
                    )
                )
                next_block += 1
```

Pairing by block index rather than arrival order is what keeps the two tracks locked together when
one producer is briefly slower than the other.

## Reading audio from the client

Inbound audio arrives as `(1, samples)` `int16` at 48 kHz, read the same way as video but in FIFO
order so no samples are dropped:

```python theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}}
from reactor_runtime import ReadMode

chunks = self.input.mic.try_read(1, mode=ReadMode.FIFO)
```

See [Media Input](/deploy/development/reactor-model/media-input#newest-frames-or-oldest) for why
audio wants FIFO and video wants the newest frame.

## Next

<CardGroup cols={2}>
  <Card title="Media Input" icon="layers" href="/deploy/development/reactor-model/media-input">
    Read the client's camera and microphone.
  </Card>

  <Card title="Recording" icon="circle-dot" href="/deploy/development/recording">
    Capture the model's output into downloadable clips.
  </Card>
</CardGroup>