KubeRay: A Kubernetes Ray clustering solution

As a distributed computing framework, Ray works best in clustered mode, where multiple Ray workers can connect to the Ray head and execute tasks in parallel. Managing multiple Ray clusters and workloads in production is challenging, especially when workload patterns are different and may have different computational requirements and dependencies. It becomes more challenging when it requires multi-tenancy and large-scale performance in a cost-efficient way. KubeRay is an open source toolkit to run Ray applications on Kubernetes. KubeRay provides several tools to improve the experience of running Ray workloads on Kubernetes by extending the Kubernetes API and functionality to support the creation of Ray clusters of containers with a single command. In this talk, we will discuss the architectural decisions and show you how KubeRay easily manages heterogeneous resources, job submission lifecycle, application dependencies, and autoscaling.