It’s time for April's Ray Meetup, a monthly series where we get together to discuss Ray and Ray’s native libraries for scaling machine learning workloads. This month we will discuss Ray Serve, Ray’s ML framework-agnostic, production-ready, operational, and scalable model serving library.
Along with a demo, the talks will cover three functional areas of model serving with Ray Serve:
An overview of Ray Serve features and functionality and roadmap
On building multi-model inference pipelines with Ray Serve and scaling with Ray
Operationalizing Ray Serve
Join us if you are interested in serving and operationalizing ML models at scale using Ray Serve!
6:00 PM Welcome remarks, announcements & agenda by Jules Damji, Anyscale
6:05 PM “Ray Serve: Overview and roadmap,” Edward Oakes, Anyscale
6:15 PM Q&A
6:20 PM “Developing and deploying scalable multi-model inference pipelines,” Jiao Dong, Anyscale
6:45 PM Q&A
7:00 PM “Operationalizing Ray Serve,” Shreyas Krishnaswamy, Anyscale
7:25 PM Q&A
7:30 PM Demo
7:45 PM Q&A
In this introductory session, we’ll discuss the motivation behind Ray Serve, who’s using Ray Serve and why, and recent features and updates, including a look at the future feature roadmap as we approach Ray 2.0.
In this talk, we aim to show how to leverage the programmable and general-purpose distributed computing ability of Ray to facilitate authoring, orchestrating, scaling, and deployment of complex serving pipelines as a DAG under one set of APIs, like a microservice. Learn how you can program multiple models dynamically on your laptop as if you’re writing a local Python script, deploy to production at scale, and upgrade individually.
In this session, we will introduce you to a new declarative REST API for Ray Serve, which allows you to configure and update your Ray Serve applications without modifying application files. Incorporate this API into your existing CI/CD process to manage applications on Ray Serve as part of your MLOps lifecycle.
Edward Oakes is a software engineer and project lead on the Ray Serve team. He works across the stack at Anyscale, from Ray Core to Ray Serve to the Anyscale platform.
Jiao Dong is a software engineer focusing on Ray Serve and Ray infrastructure at Anyscale.
Shreyas Krishnaswamy is a software engineer focusing on Ray Serve and Ray infrastructure at Anyscale.