Skip to main content

​
What is a track?

A track is a named media stream between your app and the model. Tracks carry video, audio, or any other data the model expects or produces. Every track has a name that both sides use to identify it.

​
Server-driven track configuration

Track configuration is server-driven. Each model declares the tracks it exposes in its schema, which is the same schema documented on each model’s reference page. Your app does not need to declare tracks up front; the SDK reads the schema and sets up the WebRTC connection accordingly. For the authoritative list of tracks a model exposes, see its Schema page in the Model API Reference. For example, Helios or LingBot. Every track has a name, a kind ("video" or "audio"), and a direction:
  • recvonly tracks are outputs from the model to your app (e.g. generated video).
  • sendonly tracks are inputs from your app to the model (e.g. webcam feed).
A model’s schema lists its tracks like this:
Schema (tracks)
{
  "tracks": [
    { "name": "webcam",     "kind": "video", "direction": "sendonly" },
    { "name": "mic",        "kind": "audio", "direction": "sendonly" },
    { "name": "main_video", "kind": "video", "direction": "recvonly" },
    { "name": "main_audio", "kind": "audio", "direction": "recvonly" }
  ]
}

​
Output tracks (model to app)

The SDK receives the model’s output tracks automatically. Read them by name once the track arrives. 
reactor.on("trackReceived", (name, track) => {
  if (name === "main_video") {
    const video = document.querySelector("video");
    if (video) video.srcObject = new MediaStream([track]);
  }
});

import { ReactorProvider, ReactorView } from "@reactor-team/js-sdk";

<ReactorProvider modelName="your-model-name" jwtToken={token}>
  <ReactorView className="w-full aspect-video" />
</ReactorProvider>

@reactor.on_track("main_video")
def on_main_video(track):
    print("Received output track:", track)

# Or read the current set of received tracks at any time:
tracks = reactor.get_remote_tracks()
If the model exposes multiple output tracks (e.g. a depth map or audio), they are all received automatically. The model emits frames to its output tracks the whole time the session runs; pausing and resuming only switches whether your client receives them, like subscribing and unsubscribing. By default every output is subscribed as soon as the connection is ready. Connect with autoResumeTracks: false to start unsubscribed, then call resumeTrack(name) to begin receiving a track and pauseTrack(name) to stop. See connect().

​
Input tracks (app to model)

Some models accept input tracks (e.g. a webcam feed for video-to-video transformation). Input tracks are explicit: the model receives nothing on a sendonly track until your app publishes a media track to it. Connecting alone never starts sending, so publish once the connection reaches ready. The JavaScript variant captures the webcam with getUserMedia; in React the WebcamStream component captures and publishes for you. 
reactor.on("statusChanged", async (status) => {
  if (status === "ready") {
    const stream = await navigator.mediaDevices.getUserMedia({ video: true });
    await reactor.publishTrack("webcam", stream.getVideoTracks()[0]);
  }
});

import { WebcamStream } from "@reactor-team/js-sdk";

<WebcamStream track="webcam" className="w-48 aspect-video" />

from reactor_sdk import ReactorStatus

@reactor.on_status(ReactorStatus.READY)
async def on_ready(status):
    # track is an aiortc MediaStreamTrack
    await reactor.publish_track("webcam", track)
Call unpublishTrack(name) (or unpublish_track(name) in Python) to stop sending a track. Typically you do this on disconnect or when the source media stops.
Only one connection may publish a given input track at a time. In a session shared by multiple connections, publishing a track that another connection already holds is rejected until that connection unpublishes it.

​
Naming convention

Here’s how track names map between the model and your app:
Track naming: model track names must match app track names
For output tracks, the SDK handles naming automatically. For input tracks (e.g. webcam), the track name you publish must match the attribute name defined in the model’s Python class.
If a track is not working, check that the track name you are using matches the model’s declared track name. See the model’s Schema page in the Model API Reference for the authoritative list.
Check the Model API Reference to see what tracks a model exposes.