> ## Documentation Index > Fetch the complete documentation index at: https://docs.reactor.inc/llms.txt > Use this file to discover all available pages before exploring further. > ## Agent Instructions > Reactor hosts multiple models, each with its own connect slug (modelName) and command/event schema. The catalog of every model — slug, typed SDK package, and links to its schema — is at /model-api-reference/overview. Some models expose one slug per experience (e.g. HappyOyster); always take the slug from the model's own pages, never guess it. > Fastest path to a working app: `npx create-reactor-app my-app --model=` scaffolds a complete app with secure auth wired up. Typed TypeScript SDKs are published as @reactor-models/; Python uses the base reactor-sdk package. > Auth: exchange an API key (rk_...) for a JWT via POST https://api.reactor.inc/tokens from your server. Never put the API key in client-side code. > Append .md to any docs URL for clean Markdown. Search these docs via the MCP server at https://docs.reactor.inc/mcp. # Track > The Swift object form of a named media track A `Track` is a handle onto one named media slot the model declared — not something you construct yourself. Ask for it **by name** with [`reactor.track(name)`](/sdk-reference/swift/reactor#track-_), or find it by filtering [`reactor.tracks`](#tracklist) when you don't know the name: ```swift theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}} let camera = try reactor.track("camera") // sendonly video try await camera.publish() try camera.pushFrame(bgra, width: 1280, height: 720) let output = try reactor.track("main_video") // recvonly video let frames = try output.onFrame { frame in render(frame.pixels, frame.width, frame.height) } ``` One type covers both directions and both kinds, because the operations are the same operations either way: `pushFrame` sends, `onFrame` receives, one name for video and audio alike — the track already knows its kind. A handle, not an owner: it holds the client **weakly**, so a track parked in a view model cannot keep the session alive for its lifetime. Registering a handler after the `Reactor` that owns it has been released throws `ReactorError.invalidState` rather than silently never firing. Calling a method the track's kind or direction does not allow **throws**, on purpose: `pushFrame` on a recvonly track, `onFrame` on a sendonly one. Each of those would otherwise reach the native layer, find nothing to do, and return — so a caller pushing at 30fps would see a model receiving nothing and no reason why. *** ## Properties ### `name` ```swift Signature theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}} public let name: String ``` The declared name. Never changes. *** ### `kind` ```swift Signature theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}} public var kind: TrackKind? { get } ``` `.video` or `.audio`, or `nil` before the session has declared its tracks. *** ### `direction` ```swift Signature theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}} public var direction: TrackDirection? { get } ``` `.sendonly` or `.recvonly`, or `nil` before the session has declared them. `.sendonly` is from this client's point of view: this client sends, the model receives. *** ### `mid` ```swift Signature theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}} public var mid: String? { get } ``` The SDP media id, once the track has been received. Read from the session rather than cached: it is reported as tracks arrive and is renegotiated on a reconnect. *** ### `paused` ```swift Signature theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}} public var paused: Bool { get } ``` Whether this track is paused right now. Read from the session, not cached, so it stays right across a reconnect — recvonly tracks resume automatically once connected, and a cached `true` would go on claiming otherwise. *** ### `published` ```swift Signature theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}} public var published: Bool { get } ``` Whether this sendonly slot is activated. Kept by the SDK rather than read back, because the session does not record it: publish is a request and unpublish a notification, and neither leaves anything to query. **It is cleared whenever the status leaves `.ready`.** A reconnect resumes recvonly tracks and nothing else, so a slot published before one is not published after it — publish again. See [`Reactor/reconnect()`](/sdk-reference/swift/reactor#reconnect). *** ## Sending ### `publish()` Activates this sendonly slot, so the model has something to receive on. ```swift Signature theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}} public func publish() async throws ``` Publishing is what puts a sender behind the slot: pushing before it would drop the frame, and this SDK [refuses](#pushframe-_) rather than letting it. Throws on a recvonly track, and on a session that is not `.ready`. ```swift Example theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}} let camera = try reactor.track("camera") try await camera.publish() ``` *** ### `unpublish()` Deactivates the slot. Synchronous, unlike the other track methods — there is no round trip, only a local state change and a fire-and-forget notification. ```swift Signature theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}} public func unpublish() throws ``` Throws when the notification could not be made; the track then stays published, so a retry is possible. *** ### `pushFrame(_:)` Pushes a frame into this sendonly track. The overload — and what else is needed — follows from the track's [`kind`](#kind). Each tab shows the signature for that track kind.
For `TrackKind.video`. ```swift Signature theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}} public func pushFrame( _ pixels: Data, width: UInt32, height: UInt32, userData: Data? = nil, captureTimeUs: Int64? = nil ) throws ``` | Argument | Type | Description | | --------------- | -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | `pixels` | `Data` | Exactly `width * height * 4` bytes: blue, green, red, alpha per pixel. | | `width` | `UInt32` | Frame width in pixels. | | `height` | `UInt32` | Frame height in pixels. | | `userData` | `Data?` | Bytes the far end reads as this frame's metadata. Sent as-is — JSON, protobuf or anything else is between you and the model — and dropped silently by a peer that did not declare it reads them, so tagging is safe whatever the far end supports. See [Frame Metadata](/concepts/frame-metadata). Default: `nil`. | | `captureTimeUs` | `Int64?` | When this frame was captured, read from [`Reactor.timeMicros()`](/sdk-reference/swift/reactor#reactor-timemicros). Left `nil`, the frame is stamped as it is pushed, so several tracks capturing one moment arrive microseconds apart. Default: `nil`. | A second overload takes an `UnsafeRawBufferPointer` for a caller who already holds the pixels as a buffer and wants to push without copying: ```swift Signature theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}} public func pushFrame( _ pixels: UnsafeRawBufferPointer, width: UInt32, height: UInt32, userData: UnsafeRawBufferPointer? = nil, captureTimeUs: Int64? = nil ) throws ``` ```swift Example theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}} try camera.pushFrame(bgra, width: 1280, height: 720) ``` For `TrackKind.audio`. ```swift Signature theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}} public func pushFrame( _ samples: [Int16], sampleRate: UInt32 = 48000, channels: UInt32 = 1 ) throws ``` | Argument | Type | Description | | ------------ | --------- | -------------------------------------------------------------------------------------------------------- | | `samples` | `[Int16]` | Interleaved signed 16-bit PCM. `samples.count` must divide evenly by `channels`. | | `sampleRate` | `UInt32` | Sample rate in Hz. Must match what the source declared — 48 kHz for every model today. Default: `48000`. | | `channels` | `UInt32` | Channel count. Must match what the source declared — mono for every model today. Default: `1`. | ```swift Example theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}} try micTrack.pushFrame(samples) ```
Throws `ReactorError.invalidState` on a recvonly track, before [`publish()`](#publish), on a session that has left `.ready`, or on an overload that does not match this track's kind (BGRA into an audio track, or PCM into a video one); `ReactorError.badRequest` on a BGRA buffer whose length does not match the dimensions, or PCM whose sample count does not divide by `channels`. *** ## Receiving ### `onFrame(_:)` Receives decoded frames from this track, copied so they can be kept. [`kind`](#kind) determines the frame type the handler takes.
For `TrackKind.video`. ```swift Signature theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}} public func onFrame(_ handler: @escaping @Sendable (VideoFrame) -> Void) throws -> Subscription ``` | Argument | Type | Description | | --------- | -------------------------------- | ---------------------------------------------- | | `handler` | `@Sendable (VideoFrame) -> Void` | Receives a copied [`VideoFrame`](#videoframe). | ```swift Example theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}} let frames = try output.onFrame { frame in render(frame.pixels, frame.width, frame.height) } ``` For `TrackKind.audio`. ```swift Signature theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}} public func onFrame(_ handler: @escaping @Sendable (AudioFrame) -> Void) throws -> Subscription ``` | Argument | Type | Description | | --------- | -------------------------------- | ---------------------------------------------- | | `handler` | `@Sendable (AudioFrame) -> Void` | Receives a copied [`AudioFrame`](#audioframe). | ```swift Example theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}} let audio = try speech.onFrame { (frame: AudioFrame) in play(frame.samples, frame.sampleRate) } ```
Runs **inline on the library's delivery thread**, deliberately: while it runs, the library keeps only the newest video frame and drops what arrives in between — audio is queued instead (see [`AudioFrame`](#audioframe)). Blocking here *is* the backpressure. Handing frames to a queue instead trades a bounded drop for unbounded latency and memory. Throws `ReactorError.invalidState` when this track sends rather than receives, or when the frame type does not match this track's kind — both would otherwise be a handler that never fires. Hold the returned [`Subscription`](/sdk-reference/swift/types#subscription). It cancels — and stops delivering — as soon as nothing references it. *** ### `onRawFrame(_:)` Receives frames without copying them. ```swift Signature theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}} public func onRawFrame( _ handler: @escaping @Sendable (borrowing RawVideoFrame) -> Void ) throws -> Subscription ``` Same thread, same backpressure as [`onFrame(_:)`](#onframe-_) — but the buffers belong to the library and are **gone when the handler returns**. For a renderer that uploads straight to a texture, this is the version that copies nothing it does not need to. *** ### `VideoFrame` ```swift Definition theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}} public struct VideoFrame: Sendable { public let trackName: String public let pixels: Data public let width: UInt32 public let height: UInt32 public let frameID: UInt64 public let captureTimeUs: UInt64 public let userData: Data? } ``` The track this arrived on. Every recvonly video track decodes into one callback, so on a session with several this is what tells them apart. BGRA pixels — blue, green, red, alpha — `width * height * 4` bytes. The sender's frame counter, or 0 when the frame carried no metadata trailer. When the sender says it captured this frame, in microseconds **on the sender's own clock**. 0 when the frame carried no trailer. Differences between stamps from one sender are what this supports; it is not comparable with a local clock. The bytes the sender tagged this frame with, if any. `nil` when the frame carried no trailer, and also when the far end never declared that it writes tags — no published model attaches one today. `onRawFrame(_:)` hands the same fields, borrowed, as `RawVideoFrame`: ```swift Definition theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}} public struct RawVideoFrame: ~Copyable { public let trackName: String public let pixels: UnsafeRawBufferPointer public let width: UInt32 public let height: UInt32 public let frameID: UInt64 public let captureTimeUs: UInt64 public let userData: UnsafeRawBufferPointer? } ``` **`RawVideoFrame`'s buffers are borrowed for the duration of the handler and no longer.** Copy anything you keep — retaining a pointer here is a use-after-free that reproduces under load and not in tests. *** ### `AudioFrame` ```swift Definition theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}} public struct AudioFrame: Sendable { public let trackName: String public let samples: [Int16] public let sampleRate: UInt32 public let channels: UInt32 } ``` Interleaved 16-bit PCM, `samples.count` total across all channels. Copied, unlike video — audio arrives in short buffers, roughly 10 ms each, and the queue behind it keeps its backlog rather than dropping, because there the queue is the jitter buffer and a hole in it is audible. *** ## Pausing ### `pause()` ```swift Signature theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}} public func pause() async throws ``` Stops this track. Nothing is generated while paused, which on a video track is visible only as a frozen frame. *** ### `resume()` ```swift Signature theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}} public func resume() async throws ``` Starts it again. *** ## `TrackList` The tracks a session declares, **in declaration order**, filterable — for discovery, and for a caller who would rather not hardcode a name. Returned by [`reactor.tracks`](/sdk-reference/swift/reactor#tracks). ```swift Signature theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}} public struct TrackList: Sendable, RandomAccessCollection { public func withKind(_ kind: TrackKind) -> TrackList public func withDirection(_ direction: TrackDirection) -> TrackList public func one() throws -> Track } ``` Filters chain in either order, and it iterates like an array: ```swift Example theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}} let output = try reactor.tracks.withDirection(.recvonly).withKind(.video).one() for track in reactor.tracks { print(track.name) } ``` `one()` throws `ReactorError.notFound` when the list is empty and `ReactorError.conflict` when it holds more than one — a filter that matched several and a caller that wanted one is a question with no answer, and picking the first would answer it wrongly and silently. *** ## `TrackKind` and `TrackDirection` ```swift Definition theme={"theme":{"light":"github-light","dark":"github-dark-high-contrast"}} public enum TrackKind: String, Sendable, CaseIterable { case video case audio } public enum TrackDirection: String, Sendable, CaseIterable { case sendonly case recvonly } ``` `String`-backed, so `TrackKind(rawValue: "video")` and `.video.rawValue` round-trip through the wire spelling directly.