#

gpu-efficiency

Here are 3 public repositories matching this topic...

aragit / vllm-context-cache-router

This is a state‑of‑the‑art infrastructure that solves one of the most critical scalability bottlenecks in production LLM serving: distributed KV‑cache fragmentation. By extending SGLang's RadixAttention concept to a cluster‑aware, shared‑nothing topology, it transforms a set of independent vLLM workers into a cache‑conscious serving fabric.

inference prometheus radix-tree performance-optimization load-balancing distributed-cache opentelemetry kv-cache cache-routing llm-serving vllm pagedattention sglang prefix-caching radixattention gpu-efficiency

Updated Jul 11, 2026
Python

ingero-io / scan

See the Model FLOPs Utilization gap behind your GPU's "100% utilization": the GPU spend nvidia-smi and DCGM hide. No root access required.

gpu cuda prometheus nvidia observability mfu gpu-monitoring finops gpu-utilization llm llm-serving vllm llm-inference sglang dcgm model-flops-utilization gpu-efficiency

Updated Aug 15, 2026
Go

pieroevcc / MNISTDigitsToGIF

Video diffusion model for generating MNIST digit animations on a consumer GPU. Implemented from scratch with PyTorch.

pytorch mnist diffusion-models 3dunet gpu-efficiency

Updated Jul 16, 2026
Python

Add this topic to your repo

To associate your repository with the gpu-efficiency topic, visit your repo's landing page and select "manage topics."