This repository is a collection of accelerated platform best practices, reference architectures, example use cases, reference implementations, and various other assets on Google Cloud.
An accelerated platform utilizes specialized hardware components, or accelerators, such as GPUs (Graphics Processing Units) and TPUs (Tensor Processing Units), to significantly increase the speed of computationally intensive tasks. These tasks may include data analysis, machine learning, artificial intelligence, and graphics rendering. By offloading demanding workloads from traditional CPUs to dedicated hardware accelerators, which are capable of much faster parallel calculations, the platform optimizes high-performance computing.
The GKE Base Platform (reference implementation) is an opinionated, foundational platform built on GKE using modular Terraform (Terraservices) that incorporates security, networking, observability, and autoscaling best practices optimized for running accelerated AI/ML workloads.
Standardized, production-grade patterns and automated deployments for serving real-time, streaming, async, and batch AI inference across single-region and multi-region GKE clusters with GPUs and TPUs.
-
Multi-region GKE inference, NVIDIA Dynamo, and NVIDIA TensorRT-LLM
- Multi-region GKE inference foundation and Multi-region Terraform foundation — Hub-and-spoke multi-cluster GKE topology with deterministic VPC subnets, fleet service exports, and global cross-region Envoy Gateway failover.
- NVIDIA Dynamo and TensorRT-LLM reference architecture on multi-region GKE
— KV-cache-aware routing, pinned host-memory KV cache offloading,
disaggregated prefill/decode (P/D), and cross-region failover on NVIDIA
Blackwell
RTX PRO 6000GPUs. - NVIDIA TensorRT-LLM and Nemotron-3 reference architecture on multi-region GKE
— Global open-model inference with
NVFP4quantization, Multi-Token Prediction (MTP), Cloud Storage Anywhere Cache, and GKE Pod snapshots onRTX PRO 6000GPUs.
-
Intelligent inference scheduling with
llm-d -
Rapid PoC, fast-start model streaming, and GKE Pod snapshots
-
- Online inference using vLLM with GPUs on Google Kubernetes Engine (GKE)
- Online inference using vLLM with speculative decoding and GPUs on Google Kubernetes Engine (GKE)
- Online inference using vLLM native KV cache offloading on GPUs on Google Kubernetes Engine (GKE)
- Automated vLLM server parameter tuning on GKE
- Online inference using Diffusers with GPUs on Google Kubernetes Engine (GKE)
-
Async, batch, and inference benchmarking
Defense-in-depth runtime architecture and interactive Red Team vs. Blue Team
testbed for autonomous AI agents on GKE using GKE Sandbox (gVisor),
zero-secret egress credential injection, and millisecond memory snapshot
recovery.
Scalable distributed training and fine-tuning blueprints on GKE with GPUs and Cloud TPUs.
-
Supervised fine-tuning (SFT) on Cloud TPUs
End-to-end infrastructure and workload orchestration for reinforcement learning (RL), reasoning alignment, and post-training on GKE with JobSet, Pathways, and Cloud TPUs.
End-to-end reference implementation for deploying ComfyUI image and video generation workflows on GKE with GPU acceleration and Cloud Storage integration.
Isolated, multi-tenant GKE blueprint for running cross-silo and cross-device federated learning workloads on Google Cloud.
-
Federated learning examples
Note
The Cloud Workstations (CWS) Platform is currently in beta and is still being actively developed.
The Cloud Workstations (CWS) Platform is a core, best practices, fully managed workstation environment built to meet the needs of security-sensitive enterprises. It enhances the security of workstation environments while accelerating onboarding and productivity.
For more information about contributing to this repository, see CONTRIBUTING and the GKE Base Platform developers guide.