a community oriented 1:1, vLLM-alike (Continuous batching, paged KV) engine in C++ with additional features (GGUF, RadixAttention, Cache-aware scheduling, ...)
-
Updated
Oct 28, 2026 - C++
#
radixattention
Here are 5 public repositories matching this topic...
A deterministic PyTorch autograd verification trap for catching silent KV-cache routing and block-alignment failures in vLLM and SGLang serving infrastructure.
cli inference pytorch autograd multi-agent fuzzing sympy formal-verification mlops kv-cache llm-serving vllm pagedattention sglang agentic-workflow ml-infra radixattention
-
Updated
Jun 7, 2026 - Python
This is a state‑of‑the‑art infrastructure that solves one of the most critical scalability bottlenecks in production LLM serving: distributed KV‑cache fragmentation. By extending SGLang's RadixAttention concept to a cluster‑aware, shared‑nothing topology, it transforms a set of independent vLLM workers into a cache‑conscious serving fabric.
inference prometheus radix-tree performance-optimization load-balancing distributed-cache opentelemetry kv-cache cache-routing llm-serving vllm pagedattention sglang prefix-caching radixattention gpu-efficiency
-
Updated
Jul 11, 2026 - Python
Sovereign structured LLM generation engine. SGLang RadixAttention + Pydantic schema enforcement + Ollama fallback. Guaranteed JSON output. No hallucinated keys.
-
Updated
Aug 15, 2026 - Python
SGLang vs vLLM playground: FastAPI app benchmarks TTFT/throughput against your own servers, plus 2 notebooks
benchmarking jupyter-notebook fastapi kv-cache llm-serving vllm llm-inference pagedattention sglang radixattention
-
Updated
Jul 9, 2026 - Jupyter Notebook
Add this topic to your repo
To associate your repository with the radixattention topic, visit your repo's landing page and select "manage topics."