The 10 Best GPU Orchestration Tools for Kubernetes in 2027

NVIDIA Run:ai is described as the open source AI workload and GPU orchestration platform for Kubernetes, for training, fine-tuning and inference. It offers dynamic GPU allocation, policy-driven governance with quotas, priorities and fairness, and fractional GPU sharing with GPU memory swap. The code is Apache License 2.0 and includes a control plane, a per-cluster component and an installer. NVIDIA announced a definitive agreement to acquire Run:ai on April 24, 2024, and the Run:ai GitHub organization now says it is part of NVIDIA. Commercially supported versions come through NVIDIA DSX OS, NVIDIA Mission Control, NVIDIA AI Enterprise or standalone, and standalone pricing is not published.

KAI Scheduler is a Kubernetes-native scheduler for GPU-heavy AI and ML workloads, built on kube-batch and described as a Cloud Native Computing Foundation sandbox project. It does gang scheduling, packing and spreading, topology-aware scheduling, hierarchical queues with fair-share including Dominant Resource Fairness, and GPU sharing. It is free under Apache-2.0 and shows 1,598 stars.

Kueue is a Kubernetes-native system that manages quotas and how jobs consume them, deciding when a job waits, is admitted or is preempted. It supports fair sharing among tenants and resource flavor fungibility, and integrates with Batch Jobs, JobSet, Pods, Deployments, StatefulSets and LeaderWorkerSet, with RayCluster, RayJob and RayService listed as Stable. It leaves pod placement to kube-scheduler. It is Apache-2.0 and shows 3,100 stars.

Volcano is a Cloud Native Computing Foundation incubating batch scheduler for Spark, TensorFlow, PyTorch, Flink, Argo and Ray. It offers gang scheduling, binpack, DRF, NUMA-aware and task-topology policies, multi-level queues with borrowing, reclaiming and preemption, and GPU virtualization through dynamic NVIDIA MIG, vCUDA and multi-container GPU sharing. It is Apache-2.0 and shows 6,030 stars.

NVIDIA GPU Operator automates the NVIDIA software needed to provision GPUs in Kubernetes: drivers, the device plugin, the Container Toolkit, GPU Feature Discovery node labeling and DCGM-based monitoring. The operator source code and Helm chart are Apache 2.0, while most component images fall under NVIDIA product-specific terms, and the page states no price. It provisions GPUs and is not a job scheduler. The repository shows 2,933 stars.

KubeRay adds RayCluster, RayJob and RayService custom resources for running Ray on Kubernetes, with optional autoscaling, support for heterogeneous nodes including GPUs, and multiple Ray versions in one cluster. It is Apache-2.0 and the repository shows 2,739 stars.

HAMi, a CNCF Incubating project formerly called k8s-vGPU-scheduler, lets workloads request a fraction of an accelerator by memory, cores or device count, for example one GPU with about 3 GB of GPU memory. It supports binpack, spread, topology-aware, mutex and NUMA-based policies across NVIDIA with MIG, AMD, Huawei Ascend and other vendors, with support varying by vendor. It is Apache License 2.0 and shows 4,912 stars.

SkyPilot runs tasks on an existing Kubernetes cluster (EKS, GKE, on-prem, Kubeadm, Rancher, K3s) without provisioning new clusters or nodes. It shows real-time GPU availability per node, runs multi-node jobs as one pod per node, and fails over to another cluster or cloud if scheduling fails. The README lists 25 supported targets. It is Apache-2.0 and shows 10,726 stars.

Dynamic Resource Allocation lets device drivers and admins define device classes that workloads claim, after which Kubernetes allocates matching devices and places Pods. It has been stable since Kubernetes v1.35, first available in v1.30, supports sharing one device across containers or Pods, and filters devices with CEL. The Kubernetes scheduler does not support preemption for DRA resources. It is built into Kubernetes with no separate price.

Slinky Slurm Operator, developed by SchedMD (now part of NVIDIA), deploys and manages parts of Slurm clusters on Kubernetes. NodeSets scale by replica or one pod per node, LoginSets provide login nodes, scale to zero is supported, and components can run partly outside Kubernetes. It is Apache-2.0, shows 377 stars, and community issues are handled on a best effort basis. NVIDIA says Slurm and Slinky support, training and consultation services are available from NVIDIA, with no price given.