Open Source
KubeRay
The open-source Kubernetes operator for running and managing Ray on Kubernetes.
contributors
A joint effort across the industry
why kuberay
Deploy Ray in production
Runs where you are
Installs the KubeRay operator into an existing Kubernetes cluster. No new infrastructure to stand up.
Built for how Ray runs
RayCluster, RayJob, RayService map onto Ray workload shapes.
Backed by the community
Maintained in the open by Anyscale, Google, and many more contributors.
We are using this stack to bring together the best parts of Kubernetes and KubeRay with Anyscale, as we advance our mission to build scientific superintelligence.”
why not just kubernetes
One distributed system running on another
Ray spreads work across a head node and workers that scale together. Kubernetes treats every pod as interchangeable, so a generic deployment struggles to coordinate the two.
| Hand-rolled on plain Kubernetes | With KubeRay |
|---|---|
| One RayCluster resource reconciles head and workers as a unit | One RayCluster resource reconciles head and workers as a unit |
| HPA scales on CPU/memory, not on pending Ray tasks & actors | Ray autoscaler adds and removes workers based on Ray’s resource demand |
| Finished job leaves clusters running; upgrades drop serving traffic | RayJob cleans up when job ends; RayService upgrades with zero downtime and high availability |
| Batch schedulers see individual pods, but do not start together | Gang scheduling through Kueue, NVIDIA KAI Scheduler, Volcano, or YuniKorn |
| Dashboard, metrics, and ingress stitched together per cluster | Ray Dashboard & Prometheus for jobs, actors, and cluster metrics exposed for every cluster |
| No Ray aware hooks for existing Kubernetes tooling | Works with Kueue, Argo CD, and Kyverno out of the box |
how it works
Key features
RayCluster
Manages a Ray cluster's full lifecycle: creation, deletion, autoscaling, fault tolerance.
RayJob
Creates a cluster, runs one job on it, and tears the cluster down when it finishes.
RayService
A RayCluster plus a Ray Serve deployment, with zero downtime and incremental upgrades.
Ray History Server (Beta)
A UI for viewing and managing KubeRay resources. New as of v1.4.0.
Kubectl Plugin (Beta)
Simplifies common workflows for deploying Ray, a friendlier entry point for teams newer to Kubernetes.
Zero downtime & high availability
A RayCluster plus a RayServe deployment with zero downtime and incremental upgrades.
Streamline day one onboarding to production
A RayCluster CRD bootstraps a head pod and every worker pod as a single unit. They discover each other automatically without any manual networking or service wiring required.
examples
Real KubeRay examples
Serve stable diffusion
Deploy a text-to-image model as a scalable RayService on any Kubernetes cluster
Batch inference
Run a RayJob that processes a large dataset once, then tears the cluster down when it finishes
Priority & gang scheduling with Kueue
Queue RayJobs by priority, and admit every pod in a job together or not at all, through Kueue
Serve LLM
Deploy a large language model with Ray Serve LLM directly on Kubernetes
RLHF with verl
Post-train an LLM with human feedback using verl on a KubeRay cluster
Ray & Agent sandbox
Run untrusted or agent generated code in an isolated sandbox on any RayJob
kuberay + anyscale
Keep your clusters.
Add a control plane.
Anyscale KubeRay Connect adds agentic developer experience & full-stack observability on top without migrating workloads.
KubeRay (Open Source) |
Anyscale KubeRay Connect |
|
|---|---|---|
| Ray cluster lifecycle on Kubernetes | Supported | Supported |
| Autoscaling on Ray Demand | Supported | Supported |
| Fleet scheduling across clusters | Not specified | Supported |
| Unified observability across clusters | Not specified | Supported |
| [Capability to confirm] | Not specified | Supported |
| Support | Community (Slack & Github) | Enterprise Support Tier |