generate(input) method with the same input. The runner waits for every rank and
returns rank 0’s result.
Define your worker by extending DistributedWorker.
Implement load(), generate(), and reset(), then pass the class to the runner.
Development preview. See availability
before selecting a runtime version.
ReactorApp example and
complete LingBot example.
Constructor
With CUDA, each rank binds to its visible device ordinal and the process group uses NCCL. The CPU
path uses Gloo. A single worker still runs in a separate process but skips process-group
initialization. The runner uses the
spawn multiprocessing context.
Methods
None
Construct each worker, assign its rank, world size, and device, then call
load(**load_kwargs).
Block until all workers load. A runner starts once. On startup failure, the runner shuts down its
workers and raises the error.Any
Send the same input to every worker. Block until every worker answers, then return rank 0’s result
if all succeeded. Other ranks’ result values do not return to the caller.
None
Call every worker’s
reset() and wait for every answer. The worker decides which state to
release. Keeping loaded weights is the convention.None
End the workers and release the runner’s shared-memory blocks. Repeated calls are safe. The runner
also registers this method with
atexit. Explicit cleanup is useful in scripts.Properties
int
The configured number of workers.
bool
Whether the runner has started, remains usable, and has not shut down. This is not an idle
watchdog. Worker liveness is checked while waiting for a call.
Call contract
When a runner error escapesReactorApp.generate(), the default step loop passes it to
process_output() as outcome.error. See
Handle a failed step for application
recovery. Direct runner calls raise to their caller.
Use one calling thread and one outstanding call. Concurrent calls are unsupported. Inputs and
results must be picklable. Use contiguous CPU NumPy arrays for large payloads to take advantage of
shared memory. The receiver copies the arrays out before the next call.
If all ranks raise the same exception type, the caller receives rank 0’s exception and the runner
stays healthy. This does not prove the model’s state is recoverable. An exception that cannot cross
the process boundary becomes a RuntimeError containing its type and message.
Mixed success, or different exception types, raises
RankDesync. A dead worker raises
WorkerCrashed. An unanswered call can raise
WorkerTimeout. These failures leave the runner unusable.
Shut it down and construct a new one.
Shared-memory allocation can raise
SharedSlotAllocationFailed.