ML Infra + Apps

A cross-platform, cross-tool Ray on Kubernetes deployment

Ray Summit 2022

Ray recently added M1 ARM wheels, but building a heterogeneous Ray cluster on Kubernetes still has some rough edges, especially when you add the JVM to the mix. This talk will walk through how to cross-build different arch specific containers into combined images, deal with mixed Ubuntu versions, glibc versioning, and other things one generally hopes to avoid. At the end of this talk you will be able to deploy end-to-end Ray + Spark + Dask + Jupyter on a mixed architecture Kubernetes cluster, plus understand why you might not want to.

About Holden

Holden Karau is a transgender Canadian open source developer with a focus on Apache Spark, Airflow, Kubeflow, and related "big data" tools. She is the co-author of Learning Spark, High Performance Spark, and Kubeflow for Machine Learning. She is a committer and PMC on Apache Spark. She was tricked into the world of big data while trying to improve search and recommendation systems and has long since forgotten her original goal.

About Kyle

Kyle Bendickson is passionate about delivering simple solutions to complex problems. He loves open source software and is particularly passionate about distributed systems, Kubernetes, big data processing, streaming data, and especially continuous SQL queries, as well as search and recommendations.

Holden Karau

Engineer, Netflix / Pigs Can Fly Labs

Kyle Bendickson

OSS Engineer, Tabular
Ray Summit 2022 horizontal logo

Ready to Register?

Come connect with the global community of thinkers and disruptors who are building and deploying the next generation of AI and ML applications.

Save your spot

Join the Conversation

Ready to get involved in the Ray community before the conference? Ask a question in the forums. Open a pull request. Or share why you’re excited with the hashtag #RaySummit on Twitter.