OrbitTE moves bytes between CPU memory and NVIDIA GPU allocations, within and across machines. It follows Mooncake TENT's engine, topology, context, buffer, endpoint and worker architecture, with resource ownership and transfer state machines implemented in Rust. The runtime has no Mooncake C++ dependency.
TCP, RDMA and opt-in CUDA IPC are separate payload transports. A shared control service handles peer discovery, memory grants and receipts. Applications own cache identity, recovery policy and replica placement.
- Owned memory and explicit completion. Registrations retain their actual allocations; asynchronous READ/WRITE batches, cancellation and notifications share bounded admission. Failed native cleanup retains unresolved memory.
- Persistent host RDMA. Context-owned MRs, reusable QP arrays, shared CQ lanes and a transport worker pool serve concurrent requests. A periodic monitor fences failed routes and resumes drained contexts.
- Topology-aware multirail transfers. Device masks, NUMA priorities and in-flight byte accounting select paths for bounded request slices. Parent completion requires every slice and its source receipt.
- Optional NVIDIA support. Device and pinned-host allocations use the CUDA Driver API. Device transfers select owned staging or explicit GPUDirect RDMA, with allocation-range ownership and consumer visibility before completion. Cooperative same-host GPU IPC adds explicit export ownership and close receipts; local owned pinned host READ/WRITE and bounded cross-submission import reuse are also available. TCP and staged RDMA share bounded, reusable local CUDA staging chunks.
On Linux with Rust 1.85 or later:
git clone https://github.com/OrbitInfra/OrbitTE.git
cd OrbitTE
cargo run --locked --example tcp_roundtripExpected output:
18 bytes: hello from OrbitTE
The CPU/TCP example needs no CUDA or RDMA SDK. Select optional rdma and cuda
features with the build guide, then follow the
Rust API guide and RDMA/CUDA guide.
The bounded host-RDMA RoCE profile is independently accepted. The packaged host API passes the declared two-host RoCE concurrency, resource-cleanup, restart, engine-replacement and in-flight DMA profiles. Native WR/CQE accounting verifies shared QP limits and exact bytes on two physical rails. Installed automatic discovery also matches a separately collected PCI, NUMA, port and GID inventory. Full P4 acceptance still requires physical link/device-loss evidence, provider teardown and InfiniBand. Four fresh independent two-host RoCE repeats pass. P5 CUDA work proceeds with the available hardware; see the fixed acceptance queue.
OrbitTE remains under development. The bounded GPU Direct profile passes 72 installed H20/RoCE cases on stable and Rust 1.85, with an explicit host-wide context inventory limit. Direct mode has provider prerequisites. The cooperative CUDA IPC/NVLink path passes 48 installed H20 cases with explicit exporter-lifetime limits. Bounded subbatch import reuse passes 28 additional installed cases on two H20 hosts; local owned pinned host READ/WRITE and mixed host/GPU groups pass 30 further cases plus four independent repeats. These bounded profiles are independently accepted. Optional cross-submission import caching passes 34 installed cases plus four independent repeats, with bounded mappings and explicit exporter close receipts. It supports a fixed working set until requester shutdown; source allocations stay loaned. Local staging chunks share pinned buffers across TCP and staged RDMA; 24 installed H20 cases plus four independent repeats pass for that bounded healthy profile. Opt-in remote Pin/Delegate/Unpin sessions pass 16 installed H20 TCP/RDMA cases plus four independent repeats, with scoped completion and private MR cleanup. This healthy serial profile is independently accepted. The opt-in bounded remote pipeline supports up to 16 reusable slots and passes 24 fresh installed H20 TCP/RDMA cases plus four independent repeats. Source and this healthy pipeline profile are accepted for that recorded artifact. Requester staging now uses a bounded ring with per-chunk PRE/CROSS/POST progress, nonblocking CUDA event queries, negotiated chunk sizes and a bounded CROSS queue. Up to eight shared staging worker shards progress network operations while local CUDA advances; queued cancellation precedes Pin, and final success still requires remote receipts and full drain. Its v19 hardware qualification is pending; earlier H20 results apply to their recorded artifacts. Concurrent CROSS on the wire and location selection remain open. Broader IPC parity, optional SSD transports and full TENT API/wire parity remain open. Capabilities describes supported paths; qualification records exact artifacts and limits.
The published orbit-te alpha reserves the package identity. Use the source
checkout for the implementation described here.
| Topic | Guides |
|---|---|
| Getting started | Quick Start, Build, Capabilities |
| Using the engine | TCP and memory ownership, Async batches, RDMA and CUDA, CUDA IPC, Local staging |
| Design and API | Architecture, Rust API, Frozen TENT reference |
| Development | Contributing, Qualification, Completion plan |
| Operations | Troubleshooting, Installed host validation, Native cleanup |
Technical documentation lives in docs/. The website uses Mooncake's Sphinx
Book Theme with Orbit branding and deploys automatically from main through
GitHub Pages. See building the website.
Start with the contributor guide and the source reading order. The completion plan is the only execution queue; complete P4 before renewing OrbitKV integration.
OrbitTE is an independent OrbitInfra project, licensed under Apache-2.0. Mooncake TENT is the frozen capability and architecture reference. Rust API, native ABI and wire compatibility are tracked separately in the parity contract.