Fixed
- A model that fails to load now says why. The session ended, and the error stayed in the log alone. The end of the session now carries the error’s own message. That covers a module that does not import, a weights file the model cannot read, and a
runtime.livekitblock that does not match your model. A message past 4096 characters is cut. The whole message and its traceback stay in the log. - A session with a step count ends as soon as it has taken its steps. A client can send more step requests than the session takes. The runtime waited for an answer to each extra request before it ended the session, so your model stayed busy after its last step. It now ends the session on that step. The extra requests run nothing, and no command they carry reaches your model.
- A LiveKit room connection that has left no longer counts as one that is stuck. A room connection stuck for a minute reports the instance unhealthy, and the deployment replaces it. A connection that had already left its room counted toward that, so a healthy instance could be replaced. The runtime also takes the room’s RPC methods down before it leaves the room, so a call that arrives as the connection closes is refused rather than run.
- A burst of track pause and resume requests now settles in one pass. A client pauses and resumes an outbound track as it unsubscribes and subscribes. Each request changed the connection on its own, so a run of them was applied one at a time. The runtime now applies the whole burst at once, and the track follows the last request.
Rolls up 3.11.3 and 3.11.4.
New
- A split model’s
step()can return a future for a request that finishes later. Return aconcurrent.futures.Futurein place of a result, and that request goes on running while the next batch steps. Callset_running_or_notify_cancel()when the work starts, then callset_result()orset_exception()once. The session stays busy until the future is done, so the model server answers a step it sends meanwhile withStepStillRunning. Insession_closed(), stop that generation’s work and resolve its futures. A future that is still open two minutes later makes the model server report itself unhealthy, and the deployment restarts it. Reach for this in a model half whose requests finish on its own threads. reactor.yamlcan raise the largest WebRTC message your model sends. A stepped session answers each request with that step’s media in one data-channel message, and the message carries up to 64 MiB. The runtime refuses a step above the limit withstep_result_too_large. For a model whose steps return more, add aruntime.webrtcblock withmax_message_size, from256Kito1Gi. The limit holds both ways, and a connection takes the smaller of the two sides’ limits. See reactor.yaml reference.
Fixed
- A session no longer waits on a LiveKit room that does not answer. The teardown waited for the room to be left, so a room that never answered held the session closing and kept the model from the next one. It now waits three seconds and goes on. The leave still runs to its end, so your participant leaves the room rather than staying in it. A room connection stuck for a minute reports the instance unhealthy, and the deployment replaces it.
- The sound stops when a LiveKit session ends. The room connection dropped its queued video at the close, and the audio the room already held played on. The connection now drops that audio too.
Rolls up 3.11.1 and 3.11.2. Both releases rename parts of a split model. A model that runs as one process per session is unchanged.
Breaking
reactor.yamlnames the model half inruntime.model_server, notruntime.worker. Rename the block. It takes the sameimport,max_batch, andbatch_windowkeys. With the old name, the model server process stops as it starts, and reports that the file has noruntime.model_serverblock. See reactor.yaml reference.model.resources.workers.shared_memorysizes each worker’s shared memory. A worker is one session runtime, and a split model runs one per session beside its model server. Theshm_input_bytesandshm_output_byteskeys of the old block are nowinputandoutputin this one. Write a size as a quantity, such as1Mior24Mi, or as a whole number of bytes. The defaults are the same as before: 1 MiB for a request, and 32 MiB for a result. Aruntime.model_serverblock that still carries the two old keys stops the model server, which names them.- The exceptions a split model raises now name the model server.
WorkerLostis nowModelServerLost, and every other name that carriedWorkerfor the model server carriesModelServerinstead. They still come fromreactor_runtime.model_server. Rename what you import and what yourexceptclauses catch.
Breaking
crashis now a method name the runtime owns on your app. AReactorAppsubclass that defines its owncrashmethod fails when the class is defined. Give your method another name. A command calledcrashstill works: rename the method and declare it with@event(name="crash").
New
- Several sessions can share one copy of your weights. Split your model across processes. One model server process loads the weights once, and each session runs the application half beside it. Write the model half as a
ModelServersubclass, which you import fromreactor_runtime.model_server. Itsstep()takes a batch of requests, one per session, and returns one result for each. Name the class in the newruntime.workerblock ofreactor.yaml. The application half opens aModelClientinload(). Itsgenerate()callsself.model.step(request), where it called the model half before. Each session starts withself.model.reset()in@session_started, so it never runs on the state of the session before it. A lost model server reaches the application asWorkerLost, and ends that session runtime alone. Reach for this when one copy of the weights fills most of a GPU. See reactor.yaml reference. - End your model from any thread with
self.crash(error). It ends the model the same way a raise out of a step does. The runtime logs the error and ends the session with an error the client sees. The process then exits as a failure. Use it for a failure your application learns of outside a step, where nothing raises. A lost link to a model server is one: passon_lost=self.crashtoModelClient.connect(). Only the first call counts, and a call made after the model loop stops changes nothing. - A split model’s log lines name the process that wrote them. Every record carries
runtime_role, which ismodel_serverorsession. A session runtime’s records also carry itsslot. Filter onruntime_roleto read one half alone, and onsession_idto read both halves of one run. A model that does not split carries neither field.
Rolls up 3.9.1 through 3.10.2.
New
- A step request can carry H.264 video frames. The client that asks for a step sends each input frame raw, as before, or encoded as H.264. The runtime decodes an encoded frame before the step runs.
generate()reads the same raw frames either way, so your model needs no change. Frames decode only for the input video tracks your model declares. A request the runtime cannot decode is refused, and the step does not run.
Fixed
- A step that produced no audio keeps its video. A model with no audio for a step sends an empty
(1, 0)array. In a stepped session, that array failed the whole step. The step now answers with an audio track that has no frames, and it still carries its video. The streamed path always took the empty array this way. - A command sent the moment a session starts now reaches your model. The runtime is ready as soon as the model thread starts, which can be before that thread’s loop runs. A command or a lifecycle event posted in that window was dropped. The runtime now holds those items and delivers them, in order, once the loop runs.
- A LiveKit byte stream now runs to its end when the next one opens on its topic. A new stream on the same topic ended the stream that still fed the track. An utterance then stopped part way through. Up to four streams now feed one topic at once, each to its own end, and their frames interleave in arrival order. Read
stream_idin a frame’s metadata to tell them apart. A stream past that bound is refused, so a stream that already feeds the track is never cut.
New
- A session can take one step per client request. The caller starts it with
generation: "stepped"instead of the default"streamed". Each request carries the step’s state, commands, and input frames. The runtime applies them, then runsprocess_input(),generate(), andprocess_output()once. The step’s media answers that request, so nothing plays on the output tracks and nothing is paced. Your model code is the same in both modes, but a model that overridesrun()orsubmit_command()cannot be stepped: the runtime refuses to start a stepped session for it. - In a stepped session, a refused step answers the request that asked for it. An
ApplicationErrorfromprocess_input()goes back to the client as the reason the step did not run. The runtime does not wait and ask again, as it does while streaming, and the request’s state and frames stay as they are. A command the request carried is dispatched before the step, so a handler that raisesCommandErrorfails the whole request and the step never runs. - A session can join a LiveKit room. The room connection publishes your output tracks, and every other participant in the room is a client of the session. Add
reactor-runtime[livekit]to your requirements file, because the LiveKit SDK is an optional part of the runtime. Without aruntime.livekitblock, participants reach your model with commands and messages, and the room’s tracks feed your input tracks by name. The block adds the room’s own signals on top: RPC methods that run a command or carry a message out, data topics that run a command, and byte streams that feed an input track. A block that names a command, a message, or a track your model does not declare stops the model from loading. See reactor.yaml reference. - Keep the picture on a cut with
flush(hold=True).self.output.flush()drops what is queued and cuts playout to black. Passhold=Trueand the last frame stays on screen until the next one is emitted. Reach for it in a model that drops what it was saying and carries on rendering, such as an avatar that returns to idle. - An audio input track can bound its buffer in seconds. Declare
buffer_secondson yourAudiosubclass and the track holds that length of audio, whatever frame size the transport delivers. A track without it holds 128 frames, as before. The buffer also holds at most one frame per millisecond of the bound, and aread()for more frames than that raisesValueError. Reach for this on a track whose frame size you do not control. - An audio input track receives samples at the rate it declares. Inbound audio always arrived at 48 kHz, so a model that wanted another rate resampled it in
process_input(). The runtime now resamples what arrives to thesample_rateon the track’sAudiosubclass. A track that declares nothing still reads 48 kHz.
Fixed
- A cut now stops the sound as well as the picture.
self.output.flush()dropped the frames queued for each client, but the audio the transport already held for its wire played on. The old sound then ran over the new scene. The runtime drops that audio too. - A new session keeps the frames its client already sent. A session can start before the model has finished the one before it. The reset at the end of a session dropped every buffered frame, including the frames the next session’s client had published. That session’s first steps then ran short of input. The runtime now drops the frames of the ended session only.
Fixed
- A client that disconnects no longer leaves your model’s output in memory. Every frame on its way to a client is a view into the whole array the step produced. When the client disconnected, the runtime kept the frames still queued for it and the last frame it sent. Those arrays stayed in memory after the connection closed. The runtime now drops the queued frames and that last frame when the connection stops, so the memory is free again.
New
- A saved step now records what each of its video tracks holds. The step’s
result.jsoncarries ausagelist, one entry per video track in that step’soutput.mp4. An entry names the track and the file it sits in. It gives the length in seconds, the frame count, the frame rate, and the frame size. Read it to account for what a step delivered, without decoding the file. A step that wrote no video has an empty list. See reactor.yaml reference. - A starting command the runtime cannot apply now ends a session with a step count. That session runs for its caller alone, so no client of yours can correct its setup. An upload whose bytes never arrive, or arguments the contract refuses, ends the session at once, with the refusal as its reason. The runtime does not send the rest of the starting commands. A session without a step count is unchanged: the runtime records that command as an error, and the rest still run.
- A failed starting command now fails the rest of the session’s steps. When the handler of a starting command raises
CommandError, or any other exception, the session cannot do what its caller asked. The runtime records every remaining step with the handler’s message, and the session ends with an error. In a session of four steps that fails on step 3, steps 3 and 4 both carry that message. With step results on, those steps reach the store past thequeuebound, so each one keeps its own folder. Your model stays up and takes the next session. A command from a client of your own is unchanged: its failure answers that client alone.
Rolls up 3.8.0 and 3.8.1. The runtime now takes its own measurements of a live session, and the patch release pins the transport dependency. Neither changes what you write.
Breaking
- A command and a lifecycle event now reach your model in the order they were sent. Each rode its own queue before, so a command could run ahead of the
@connectedhook of the client that sent it, or ahead of@session_started. The runtime now delivers both from one queue, one item at a time. A handler or a hook must therefore not wait for something that arrives after it. That item sits behind the handler in the queue and never runs, and the model stalls. A@session_startedhook that waits for a command to arrive is the shape that breaks. Move that work into the command’s own handler.
New
- Report a finished step with
complete_step(). The runtime counts a session’s steps from these reports. The default loop reports every step it takes, so a model that writesgenerate()needs no change. A model with its ownrun()callsawait self.complete_step(StepCompleted(output=output))each time it finishes a unit of work, such as a built clip. A loop that reports nothing leaves the step count at zero and saves no results. - Return a
StepCompletedfromprocess_output()to write the step’s report yourself. Import it fromreactor_runtime. It carries theoutputto emit, extrafilesto keep with the step, anerrorfor a step you recovered from, and theelapsedseconds. Leaveelapsedunset and the runtime fills in its own measurement.filesmaps a file name tobytes, and the namesresult.jsonandoutput.mp4belong to the runtime. Returning a plainOutputorNonestill works, and the loop still reports that step. - Keep each finished step as a folder of files. Add a
runtime.step_resultsblock toreactor.yamlwithenabled: true. Every track of the step’s output becomes a stream in oneoutput.mp4, beside the step’s extra files and aresult.jsonthat lists them.queuebounds how many steps may wait to be saved, eight by default. A step that finds the queue full is dropped rather than held, so saving never slows the model down. Each step is its own encode, so this suits a model that finishes a step every few seconds rather than every frame. See reactor.yaml reference. - A session can start with its state already set. The caller that opens it sends a starting input: state values and commands. The runtime applies them before any client connects. Your first step then reads those values rather than the field defaults, because the step loop waits for every starting command to land, uploads included.
@session_startedstill runs first, on the defaults. ClientInfo.systemmarks the runtime’s own client. The starting commands arrive from a system client, a connection the runtime opens for itself.client.systemisTruein the handler that receives them. It counts as a connected client:self.connectedis set, and@connectedruns for it. Nothing you send it is delivered. Read the field in a hook that greets a client or keeps a roster.- A large message now reaches a client that can take one. A data channel carried 256 KiB per message. The runtime now answers a client that offers chunking, and a message of up to 64 MiB passes either way on that channel. A client that does not offer it keeps the plain channel and the old limit. The runtime also sends what a chunked channel still queues before it closes the channel, waiting up to five seconds. A notice sent as a session ends therefore still arrives.
Fixed
- A message the wire refused now appears in the log. A message larger than the client accepts, or one past what the channel may queue, was dropped with nothing above debug level to show for it. The runtime now logs a warning that names the channel and the size. It warns once per channel and kind of refusal, and writes the repeats to debug. A model that oversends on every step does not fill the log.
Breaking
- A lifecycle hook binds to one method. Two methods marked with the same hook now fail when the class is defined. Before, one of them ran and the other was skipped without a warning. Keep one method per hook. To extend a hook a base class declares, override that method and call
super(). The override runs whether or not you repeat the decorator. A method cannot be both a command and a hook. - A private state field needs a default.
_first_frame: np.ndarray | Nonenow fails when the state class is defined. Write_first_frame: np.ndarray | None = Noneinstead. The runtime builds the state from the field defaults when a session starts, and noset_command fills a private field, so a field without a default has no value to start from. See Managing State.
New
- A state class can extend another state class. The subclass keeps every field of its parents and adds its own. Every public field on it is a
set_<field>command, inherited fields included. A field it declares again replaces the parent’s, default and bounds included. Declare an inherited field again as aClassVarto remove it. The fields the subclass adds are keyword-only in the constructor, so a required field can follow an inherited field that has a default. Reach for this when several models share a set of controls. - A message class can extend another message class. The subclass keeps its parent’s fields and adds its own, under the same rules. The parent stays a message of its own and keeps its place in the schema. When a shared base is not a message a client should receive, write it as a plain class rather than a
ModelMessagesubclass. - An app subclass gets the setters of the state class it names. The
set_commands follow the state class on the app itself, not the one a base app declared. A field the base’s state declared and the new one leaves out is no longer a command. A field declared again with new bounds is checked against the new ones.
Fixed
- A command a client sends the moment it connects now reaches the model. A client can open its own channels before the runtime counts the wire open. A command sent in that window was dropped. The runtime now holds those frames and delivers them in arrival order, right after
@connectedruns. It holds 64 frames and logs a warning for each one it drops past that. - A model that crashes no longer leaves the process up. A crashed step loop brings the whole process down, and that wind-down now runs under a deadline of ten seconds. Set
EXIT_TIMEOUT_SECONDSto change it. Past the deadline, the runtime writes every thread’s stack to the log and leaves with a failure status. A thread your model started cannot hold a dead container open.
New
- Run a model on several GPUs with
DistributedRunner. Import it fromreactor_runtime.distributed. The runner starts one process per rank and constructs your worker class in each one. Onegenerate(input)call reaches every rank and returns rank 0’s result. A model that splits a step gathers the pieces onto rank 0 insidegenerate(), because the runner assembles nothing itself. Your application starts the runner in its ownload()and calls it fromgenerate(), and keeps the client’s commands and tracks. Giveworld_sizethe GPU count you ask for inreactor.yaml. - Write each worker as a
DistributedWorker. The base class declaresrank,world_size, anddevice, and the runner sets all three before it callsload(). Your subclass writesload(),generate(), andreset(). When every rank raises the same type of error, the call raises rank 0’s exception and the group stays usable. A crash, a timeout, or ranks that disagree raiseWorkerCrashed,WorkerTimeout, orRankDesync. The runner then refuses further calls, so shut it down and construct a new one. See Multi-GPU inference.
Fixed
- A recording keeps the frames from the end of a session. The recorder closed its encoder as soon as the session stopped, while frames it had already taken were still queued for that encoder. Those frames never reached the file, so the recording and any clip over the end of the session both stopped early. The recorder now waits up to a second for the queue to reach the encoder, and closes the encoder after that. An encoder still behind after the wait loses the rest of the queue. The runtime logs a warning with the number of frames left in it.
New
ReactorAppis the class you subclass, andgenerate()is the method you write. The runtime drives the class one step at a time: each step callsprocess_input(), thengenerate(input), thenprocess_output(outcome), and emits theOutputthe last one returns.generate()runs one step of inference, synchronously, and returns the result or raises. The other two have defaults, so a model that returns anOutputfromgenerate()writes nothing else. Overrideprocess_input()to refuse a step withApplicationError("reason")or to shape what the model gets; overrideprocess_output()to map a result onto tracks, send messages, or recover from an error.StepOutcomecarries whatgenerate()did:result,error, andelapsed. A raise out ofprocess_output()ends the session with an error; the default re-raises. See The Step Loop.- Declare what a client can set as
state:. Annotate anInputStatesubclass on the app and every public field becomes aset_<field>command, validated from itsInputFieldconstraints and documented from its description. The runtime buildsself.statefrom the field defaults when a session starts, andprocess_input()reads it there on every step; the hook takes no parameters. A field with a leading underscore is session scratch with no command. A hand-written@eventof the same name replaces a generated setter. See Managing State. - Handlers run between steps. Every
@eventhandler and lifecycle hook waits for the step in flight to finish before it runs, so a step reads one consistent state and a handler can call into the model safely. When the last client leaves or the session ends, the loop stops at the next step boundary and resets the input buffers. - Playout follows the model’s speed unless you pin it. A class that declares
fpsplays out at that rate. One that does not plays each step’s frames at the rate the step was produced, from the runtime’s own measurement ofgenerate(), so a hand-passedcompute_timeis no longer needed. ReactorModelis nowReactorApp, andInputis nowMediaInput. Both old names still import and resolve to the same classes, with aDeprecationWarning. A model with a hand-writtenrun()keeps working unchanged: overridingrun()replaces the step loop and only the step loop, and the three hooks are then never called. The wire, the schema, and the client SDKs do not change.ReactorPipelineandIdleare deprecated. Both still import fromreactor_runtimeand resolve to themselves, so a model on the generator pattern keeps running, but each import now raises aDeprecationWarningand writes one warning to the log. The replacement is a port, not an import edit: subclassReactorAppand writegenerate(), and refuse a step by raisingApplicationErrorfromprocess_input()whereinference()yieldedIdle. All four deprecated names are removed in the next major.- The runtime’s examples are split into an application half and a model half.
examples/starteris the spinning logoreactor initscaffolds, written as two halves; the logo image is its weights, read from the workspace’sweights/directory, and it runs on a CPU.examples/echoloops the client’s webcam and microphone back through one of seven video effects.examples/waypointdrives a world model from an uploaded seed frame and keyboard and mouse controls. In each,<name>.pyis theReactorAppand<name>_model.pyis a plain class that imports nothing from the runtime.examples/brightnessis gone;startertakes its place. See Application and Model.
New
- A connection is live sooner. The runtime answers the two WARP opt-ins, a pair of IETF drafts that shorten the WebRTC handshake. One carries the DTLS handshake inside the ICE checks, and the other lets the data channel skip a round trip of its own setup. Both are on by default, and you do not have to change your model. A client that does not ask for them gets the ordinary handshake, so what you save depends on the SDK at the other end.
runtime_webrtc_connect_secondsreports the client’s wait until the wire is live.
New
GET /metricsnow reports how a live wire carries your media. The runtime samples each WebRTC connection and adds the readings to the Prometheus output it already serves. Per track, it counts the packets, bytes, and video frames that went each way. It also records the loss the receiver reports on what you sent, and the jitter on what you receive. Thetracklabel is the name you declared on yourOutputorInputclass. Per connection, it records the bandwidth congestion control believes the path will carry, and two round trips. The ICE checks measure one and the receiver measures the other on the media itself, so a gap between them is a queue the checks do not pass through.- The runtime also counts the media it dropped or invented before it reached the wire.
runtime_media_dropped_frames_totalandruntime_media_dropped_samples_totalcount the frames and audio samples it discarded.runtime_media_silence_frames_totalcounts the audio frames it sent as silence because your model produced none. None of this appears in the transport readings, so read these counters beside them to tell a slow model from a poor network. Two histograms time the handshake as well.runtime_webrtc_negotiation_secondsmeasures the answer the runtime builds, andruntime_webrtc_connect_secondsmeasures the client’s wait until the wire is live.
Fixed
- A handler that declares a list of uploads now receives the files. The runtime resolved an
UploadedFileparameter, but not one nested inside alist, adict, or a dataclass field. Each of those reached the handler as the client’s raw reference, a mapping with anupload_idkey. The handler then failed when it read.data. The runtime now follows the command’s declared types and fetches every upload it finds, at any depth. It reads the declaration, not the values. A mapping of your own that carries anupload_idkey still arrives as the client sent it.
New
- The runtime fixes a session’s recording id when it accepts the start. A
session_idin the start parameters becomes the session’s own id. The runtime stores and addresses the recording under that id, and every log record the session writes carries it. Clips and logs then match the id the caller knows the session by. A start that names nosession_idgets a fresh id, so two sessions in one runtime process never write into the same recording directory. A rejected start leaves a live session’s id alone. The transport session id does not change. - A caller can supply a connection’s ICE credentials and port range. Beside
ice_servers, an offer to a connection’ssdp_paramstakes two more optional fields.ice_credentialsis aufragandpwdpair, andport_rangeis an inclusive[min, max]. Leave both out and the media engine makes its own credentials and uses the configured port range. That is the usual case. They exist for a deployment that puts a relaying layer in front of the runtime. That layer has to know a connection’s ICE credentials and media address before the connection exists. A single-port range pins the connection to one port.port_rangereplaces the configured range. It does not narrow it, soWEBRTC_PORT_RANGEdoes not bound what a caller asks for. Ask for a port another live connection already holds and the runtime refuses with409, instead of a negotiation that quietly fails later.
Fixed
- A recording keeps every sample of a batch’s audio. The recorder’s audio buffer held one second of samples, whatever the size of the batch handed to it. A batch that carried more than a second lost its oldest samples. The recorder then paired the audio that was left with the batch’s first frames and filled the rest with silence. A clip lost most of its audio, and what remained ran ahead of the picture. Live playout kept every sample throughout. The buffer is now never smaller than the batch it takes, so every sample reaches the recording.
New
- Write structured logs with
get_logger(). Import it fromreactor_runtime. Pass context as keyword arguments:logger.info("scene changed", prompt=self.prompt). A record prints as the message and then itskey=valuetokens. WhenREACTOR_LOG_FORMAT=jsonis set, each record prints as one JSON object per line. - Every record names the session and the phase it was written in. While a session is live, every record carries its id as
session_id. One filter then finds everything that run logged. The phase arrives at two levels of detail:state, the session’s own word for it, andruntime_state, a coarser one (loading,available,serving,terminated). The runtime stamps a record where it is written, not where it is made. A plainlogging.getLogger(__name__)and the libraries your model imports carry the same fields, so you never thread a session id through your own calls.
Fixed
- Video above 960x540 no longer stops at 2.5 Mbps. Each outbound video track ran on the encoder ceiling the transport derives from the frame size, which is 2500 kbps for a frame that large. A 720p or 4K stream held that rate on a fast link, and the picture stayed soft. The runtime now gives each video track a ceiling of 10 Mbps, the limit the connection’s bandwidth estimate already had. Set
WEBRTC_SENDER_MAX_KBPSto choose a different one.
Rolls up 3.2.4 and 3.2.5.
New
- The recorder’s settings nest under
runtime:. The runtime reads therecording:block asruntime.recording, besideruntime.importandruntime.config. A top-levelrecording:block still works, so an existing manifest needs no change. Declare the block in one place. A manifest that carries both takes the nested block whole. It reads nothing from the top-level one, so anenabled: trueleft up there leaves the recorder off. See Session Recording.
Fixed
- The end-of-candidates marker no longer fails a client’s ICE request. A client marks the end of its trickle-ICE candidates with an empty candidate string (RFC 8838). The runtime rejected that marker. The request that carried it failed, and the runtime dropped any candidate sent with it. The connection then had fewer network paths to try. The runtime now accepts the marker and ignores it.
Rolls up 3.2.1 through 3.2.3.
New
- Read when the sender captured a frame.
InputFrame.capture_time_usholds the capture time the sender declared for that frame, in microseconds, orNonefor a frame that carried no stamp. A sender that declares nothing gets a clock reading from its transport, so most frames carry a stamp. A client that stamps several tracks from one clock reading puts that value on all of them, so a multi-camera capture reads as one moment. Unlikepts, the stamp reads another machine’s clock: two stamps from one sender differ by that sender’s own timing. Subtract a stamp from your own clock, and you mostly measure the offset between two clocks that drift apart. See Video & Audio Tracks.
Fixed
- A recording keeps every frame of a batch. The recorder’s feed queue held four frames, whatever the size of the batch handed to it. Anything larger waited on the encoder, and the frames past that wait never reached the recording. You saw the loss in the log as a full feed queue, and live playout kept every frame throughout. The queue is now never smaller than the batch it takes, so a whole batch fits.
Breaking
- Marking a field for moderation is opt-in.
InputField(moderate=True)marks a field whose text a deployment should have moderated, and the default is nowFalse. Earlier releases marked every field unless it opted out, so a model that wants a field moderated has to say so. Mark only the fields a client writes free text into — a mark on an enum, a number, or a file changes nothing, since only free text is eligible — and only on a command that arrives occasionally, like a prompt or a script. A check runs one command at a time and admits anything still waiting after two seconds, so marking a per-frame command neither gets it moderated nor leaves room for the prompts that matter. The runtime moderates nothing itself: the mark is a preference a deployment reads off the published schema, where every field now states it either way.
New
- A model can send more than one audio track. Outbound audio is keyed by track name, the way video always was, so two
Audiofields on oneOutputreach the client as two tracks. Declaring two previously concatenated them into one buffer that played out through whichever track negotiated last, at twice the rate, with nothing rejecting or logging it. - Clients are told why a session ended. A session the platform ends can carry a human-readable reason, delivered to every connected client before its connection closes, in place of a bare disconnect. Nothing in the model changes.
Fixed
- Audio no longer drifts against video. An outbound audio track’s clock advances only with the samples pushed onto it, so a tick the model did not fill was time the stream never accounted for — and because the packets either side of it stayed contiguous, the client read the whole stream as arriving late, grew its jitter buffer, and time-stretched audio to refill it. That stretching was the artefact and the swinging buffer was the drift. The runtime now pushes a frame on every tick, sending silence for one the model did not fill, and warns with the track’s name when it has had to manufacture a meaningful share of the last second.
- Pausing an audio track stops the audio. A paused track is skipped before anything is read from it, so a client that pauses stops receiving packets instead of roughly fifty a second of digital silence — and the pause no longer counts against the under-production warning, which had been blaming the model for audio the client declined.
Rolls up 3.1.0 through 3.1.2. The two patch releases pin the transport dependency and change nothing you write.
New
- Send metadata with a frame. Wrap a track’s payload in
TrackPayloadto tag what it emits:TrackPayload(frame, metadata={"seed": seed}). A mapping travels as JSON and bytes travel as they are, and a batch carries either one value for the whole batch or one per frame. A bare array still works everywhere it did. - Read the metadata a client attached.
InputFrame.metadataholds the bytes the sender sent with that frame, orNonewhen it sent none. Decoding them is the model’s business — the transport treats them as opaque. See Video & Audio Tracks.
Breaking
emit()waits for downstream room. A model that generates faster than its playout rate is throttled to that rate rather than piling up latency, so a hand-rolled rate limiter is now redundant and should come out. The wait runs off the model loop, so commands and lifecycle hooks keep dispatching while it holds. Passdrop=Trueon a producer that would rather skip a frame than wait.outputis the playout handle, not an annotation. Removeoutput: MyOutputfrom your model class — outbound tracks register when theOutputsubclass is defined, and the name now belongs toself.output. A class that re-annotates it contradicts the real attribute.buffer_sizemust be positive when declared. Zero or less fails at startup instead of silently falling back to the runtime default.
New
self.outputcontrols playout.self.output.fps = nre-paces frames that are already queued rather than waiting for the next emit, so a speed command takes effect immediately.self.output.flush()drops what is queued and cuts the client to black, which is what a scene reset wants so none of the old content plays afterwards. Both fan out to every connection, including ones that join later. See The Run Loop.buffer_sizebounds buffered latency. It declares how many frames may sit between the model and each client, and is never applied below one emitted chunk, so a batching model always fits a whole batch.- A quiet wire on underrun. When no frame is ready for a tick nothing is sent and the client holds what it has, instead of the stream spending bandwidth repeating it. One black frame marks each boundary — a connection opening, or a flush.
Breaking
- An
@eventreturn annotation must name one message type, or nothing. A handler annotates a singleModelMessagesubclass for a typed reply, orNonefor a bodyless acknowledgement. A union —Reply | Noneincluded — is now rejected when the class is declared, because the schema publishes one response shape and a client generated from it would expect no body and receive one. Annotate the message and raiseCommandErrorfor the failure case. See Events & Messages.
New
- Command handlers can fail out loud.
raise CommandError(code, message)answers the calling client with a failure it can branch on, correlated with its command, so an awaiting caller rejects with a reason instead of hanging. Any other exception answers withinternal_errorand keeps its detail in the log. UploadedFile.sizereports the byte length of an upload.- Recording encodes in process. No external encoder binary is involved. A finished recording stays fetchable for five minutes after its session ends and is then deleted; the clip endpoints answer
410 Gonepast that.recording_dir(orREACTOR_RECORDINGS_DIR) chooses where chunks are written. See Session Recording. - Render a schema without serving one.
python -m reactor_runtime.schemaprints the OpenAPI contract of the model in a directory, so a build step can publish it without booting the runtime. The schema is titled with the name the model publishes. GET /metricsserves the process’s metrics in Prometheus text format.
A rebuilt runtime with a smaller, sharper authoring surface. Every change below is source-level and mechanical; the shape of a model — declare tracks,
load(), run(), emit — is unchanged.Breaking
- Import from
reactor_runtimedirectly. Everything author-facing is exported from the top-level package. Replacefrom reactor_runtime.interface import ...withfrom reactor_runtime import .... load()receives a path, not a dict. The signature is nowload(self, config_path: Path | None), and the runtime no longer parses the file. Read it however you like:yaml.safe_load(config_path.read_text()) if config_path else {}.output_bufferis nowself.output. There is no single buffer any more — each connection paces its own playback — so the handle is named for what it does.output_buffer.flush()becomesself.output.flush()andoutput_buffer.set_fps(n)becomesself.output.fps = n, both fanning out to every connection.buffer_sizeandemit(drop=...)keep their meaning. See The Run Loop.- Drop the
output: MyOutputclass annotation. Outbound tracks register when theOutputsubclass is defined, so the annotation was never read — and the name now belongs to the playout handle, so leaving it in contradicts the real attribute. Delete the line; nothing replaces it. OutputandInputsubclasses drop@dataclass. Declare the track annotations and nothing else.Output.__init__now validates that you supplied exactly the declared tracks, so a missing track fails at theemit()call rather than silently streaming nothing.@event(dedupe=True)is removed. Every command is delivered. A handler that must collapse a burst should track the latest value itself.runtime.weights_pathis no longer read by the runtime.get_weights_path()resolves$REACTOR_WEIGHTS_PATH, falling back to~/.cache/reactor_registry.reactor runstill readsruntime.weights_pathto decide what to mount and sets the variable for you, so most workspaces need no change. See Weights.servehas no subcommands. The entry point ispython -m reactor_runtime.serve, run from the directory holdingreactor.yaml. Fetch a model’s schema fromGET /schemaon the running server instead ofserve schema.- Removed from the public API:
get_profiler(),ReactorConfig,ReactorCore, and theEvent/Connected/Disconnectedclasses. Use the@event,@connected, and@disconnecteddecorators. - Python 3.12 or newer is required.
- No media libraries on the host. The runtime carries its own WebRTC stack as a wheel, so a model image needs nothing from the system package manager and a workspace Dockerfile can start from a plain Python base.
PREFERRED_TRANSPORTand theGST_-prefixed variables are gone with it.
New
- Session lifecycle hooks.
@session_startedand@session_endedbracket the session as a whole and fire once each, so a model can tell a new viewer joining apart from the session itself beginning. Note that a session end tears its connections down wholesale without firing the per-client@disconnectedhooks. See Sessions & Clients. - Command replies. An
@eventhandler that returns aModelMessagesends it as that command’s correlated reply, so a client awaiting the command resolves with the state that actually took effect — useful when a value is clamped or a default is resolved server-side. - Read order per track. Reads take a
mode:ReadMode.LATESTreturns the newest frames and clears the backlog,ReadMode.FIFOconsumes in arrival order. Video wants the former, audio the latter. See Video & Audio Tracks. - Rate from measured throughput. Passing
compute_timetoemit()plays the chunk back at the rate you actually produced it, so a model that warms up or slows under load stays in sync without touchingfps. - Defaults validated at import. An
InputFielddefault that violates its own constraints now fails when the class is defined rather than on the first request, and mutable defaults are rejected outright.
Rolls up the patch releases since 2.8.0.
New
- Clients are told when the network is struggling. The runtime periodically sends a
mediaStatsmessage on the data channel carrying an aggregate video quality score (0–10), so frontends can show a “connection is unstable” notice when reduced stream quality comes from network conditions rather than the model. Already surfaced by the JS SDK.
Fixed
- H.264 and H.265 now stream above 720p. The senders pinned the level offered by the client onto the encoder, and browser offers advertise a ~720p level — so any higher-resolution model produced zero video on those codecs while the session otherwise looked healthy. The encoder now derives a level that fits the actual resolution (only the negotiated profile is enforced), and 2K+ output flows on both codecs, verified against real Chrome (H.264) and Safari (H.265) clients.
- Video survives low-MTU networks. RTP packets are now capped at 1200 bytes, matching mainstream WebRTC stacks. Sessions on VPNs, WireGuard/Tailscale tunnels, PPPoE ISPs, and similar paths previously connected fine (data channel, audio) but silently black-holed every full-size video packet. Overridable per deployment with
GST_RTP_PAYLOAD_MTU. - Every frame width works over WebRTC. Widths that are not a multiple of 4 no longer come in as a grayscale, sheared smear or get dropped by the encoder on the way out — the transport now honours GStreamer’s row padding in both directions. Widths that already worked keep the exact same fast path.
- Native macOS runs. The GStreamer send pipeline no longer fails to build on Homebrew PyGObject, so running the runtime directly on a Mac (outside
reactor run) works for local development.
Breaking
- The
aiortcWebRTC fallback has been removed. Running the runtime directly on a host now requires a working GStreamer installation. Stay inside the container withreactor run, which bundles GStreamer and every other dependency automatically.
New
- Multiple clients can share one session.
@connectedand@disconnectednow run once per client as each one joins and leaves, andself.connectedstays set while at least one client is connected. Single-client models keep working unchanged:self.send()still reaches the one client connected. - Per-client messaging with
ClientInfo. Any handler (@event,@connected,@disconnected,@file_uploaded) can accept aclient: ClientInfoparameter for the client that triggered it. Callclient.send()to reply to that single client orself.send()to broadcast to everyone, and store the handle to message a client later. Runtime responses (requestClip,requestRecording, schema) now address only the requesting client instead of broadcasting. See Events and Messages.
New
- Session recording, configured from
reactor.yaml. Models can now record every session continuously and let clients request snap clips or full recordings of the live stream. Fliprecording.enabled: trueinreactor.yamland the runtime hooks the same buffer that feeds the wire, encodes fMP4 chunks in the background, and exposes arequestClip/requestRecordingAPI to clients. Already wired in the JS SDK and the Demo Frontend’s Capture panel. See the new Session Recording page for the full configuration reference, encoder knobs (chunk_seconds,crf,target_width, …), and multi-track disambiguation.
Breaking
- Host CLI is now the Go
reactorbinary. The runtime no longer publishes a console script. Replacepip install reactor-runtime+reactor-runtime init|runwith the GoreactorCLI; the runtime itself ships only insidereactor-runtime-base. - Workspace runtime version is now pinned in the
Dockerfile.reactor initsubstitutes a fully-qualifiedFROM reactortechnologies/reactor-runtime-base:<X.Y.Z-N>line. Existing workspaces usingARG RUNTIME_VERSIONneed a one-line edit; everything else keeps working unchanged. reactor runno longer accepts build-time flags. Build-phase concerns (-f,--no-cache,--build-secret) moved toreactor build; run-phase concerns (--port,--gpus,--tty,-e,--env-file) stay onreactor run. Both share the same image tag, soreactor build && reactor runalways boots the freshly built image.
New
- Scaffolded workspaces double as plain Docker projects.
reactor initemitsENTRYPOINT [..., "python", "-m", "reactor_runtime.serve"]andCMD ["run"], sodocker build && docker run -p 8080:8080 .works without thereactorCLI installed.reactor runitself remains opinionated about therunsubcommand.
Breaking
- CLI renamed:
reactor→reactor-runtime. The bundled console script is nowreactor-runtime; updatereactor run|init|schemacall sites accordingly. When the GoreactorCLI is installed,reactor runkeeps working by delegating toreactor-runtime rununder the hood.
New
- Modern nested shape for
reactor.yaml. Identity and runtime entrypoint are now split undermodel:andruntime:sections. The legacy flat shape keeps working but logs a one-shot deprecation warning per process; new scaffolds emit the modern shape. See Model Anatomy. get_weights_path()helper. New import fromreactor_runtimereturns apathlib.Pathto the resolved weights root, with resolution order$REACTOR_WEIGHTS_PATH→runtime.weights_path→~/.cache/reactor_registry. See Weights.runtime.weights_pathinreactor.yaml. Optional field for committing a workspace-relative weights root (the env var still wins, so production deployments override committed values).reactor-runtime initimprovements. Name argument is optional (scaffolds into the current empty folder when omitted), model name is substituted intoreactor.yamlautomatically, a Dockerfile is part of the scaffold (python:3.12-slim+ GStreamer +uv), andrequirements.txtshipsreactor-runtimeso the workspace.venvhas the import target available immediately.
Improved
- Multi-line
ModelMessagedocstrings render correctly.reactor-runtime schemaand downstream SDK / docs generation now preserve the full docstring of eachModelMessagesubclass. Wrapped one-sentence summaries are no longer truncated at the first newline, and undocumented@dataclassmessages no longer leak their constructor signature into the description.