Registration

Community Meet Up / Happy Hour 

We are delighted to host an exclusive Ray Summit Meetup, hosted by Anyscale with Ray community talks, on the eve of the summit. Invited Ray community speakers will share how they use Ray to scale and solve challenging ML problems. You don't have to be registered for the Ray Summit to attend. The meetup is free for the community. Join us for the Ray Summit Happy Hour from 5:00 - 6:00 p.m., followed immediately by the meetup. Agenda (The times are not strict; they may vary slightly.) - 5:00 - 6:00 p.m. Ray Summit Community Happy Hour (in Seacliff Foyer) - 6:00 p.m. Welcome remarks, announcements, and agenda - Jules Damji, Anyscale - 6:05 p.m. Talk 1: Ray + Arize: Close the ML infrastructure loop - Aparna Dhinakaran, Arize AI - 6:35 p.m. Q & A - 6:40 p.m. Talk 2: Approaching Cluster Multi-tenancy with Ray Job - Jaehyun Sim, Ikigai Labs - 7:20 p.m. Q & A - 7:25 p.m. Talk 3: Large-scale distributed approximate nearest neighbor search with Ray - Daniel Acuna, Syracuse University - 7:50 p.m. Q & A Talk 1: Ray + Arize: Close the ML infrastructure loop Detecting, diagnosing, and resolving ML model performance can be difficult for even the most sophisticated ML engineers. As more machine learning models are deployed into production, it is imperative we have tools to monitor, troubleshoot, and explain model decisions. Join Aparna Dhinakaran, chief product officer at Arize AI, in a discussion on the state of commonly seen ML production monitoring challenges. Learn how to use ML Observability from training through production environments to find upstream model issues faster, monitor your models in real time at scale, and improve model interpretability and explainability. This talk will highlight common challenges seen in models deployed in production, including model drift, data quality issues, distribution changes, outliers, and bias. The talk will also cover best practices to address these challenges, and how to use observability and explainability to help identify model issues before they impact your business. Talk 2: Approaching Cluster Multi-tenancy with Ray Job Maintaining a highly available service with close to zero downtime and optimal performance is both challenging and almost mandatory in the world of data-intensive operations. Our strategy for achieving scalability and reliability without compromising on simplicity of implementation involves utilizing Ray API functionality in simple but clever ways. We'll explore how to incorporate multi-cluster sanity, seamlessly swappable dependencies, and fail-safe cluster transitions with long-running Ray clusters to meet the demands of modern cloud infrastructure. We will also touch on challenges that we faced at Ikigai when implementing this strategy and how we overcame those to reach stability. Talk 3: Large-scale distributed approximate nearest neighbor search with Ray One of the simplest and most reliable learning methods in AI is to use memory: retrieve training data points that are closest to the testing data. However, as the datasets grow, such search gets prohibitively expensive. This talk will describe how we use Ray to develop approximated large-scale, distributed nearest neighbor search. In particular, I will describe applications to fraud detection in images of the scientific literature. Next, I will describe how we harness Ray to process tens of millions of scientific articles and images and billions of keypoints extracted from these images. Then, I will describe how we use Ray to integrate GPUs into this computation, effectively cutting down billion-scale searches from days to seconds. Finally, I will discuss how we use Ray Serve to provide this search as a service, hiding all the complexity behind a simple interface. I will end by discussing how Ray compares to other tools we have attempted to use in the past, including Message Passing Interface (MPI) and Dask, and discuss the advantages and disadvantages of Ray.

Associate Professor

Director of Engineering

Chief Product Officer

Lead Developer Advocate

Ray Summit Meetup Community Talks

Breakfast   Networking 

Day 1 keynote speakers are: Soumith Chintala, Meta AI; Kim Hazelwood, Meta AI; Greg Brockman, OpenAI; Ion Stoica, Anyscale; and Robert Nishihara, Anyscale. Please check back for more details on each keynote, coming soon.

Co-founder, Executive Chairman & President

Co-founder and CEO

Co-founder & CTO

Engineer

Engineering Director

Keynotes, Day 1

Extended Break

Applying AI models to end-to-end data analysis pipelines plays a critical role in today's large-scale, intelligent applications. On the other hand, AI projects usually start with a Python notebook running on a single laptop or workstation, and one needs to go through a mountain of pains to scale it to handle larger datasets with high performance (for both large-scale experimentation and production deployment). These often require data scientists to follow many manual, error-prone steps and even make intrusive code changes so as to fully take advantage of the available hardware resources. To address these challenges, we have open sourced BigDL 2.0 (https:/ /github.com/intel-analytics/BigDL/) under the Apache 2.0 license (combining the original BigDL and Analytics Zoo projects), which allows users to build end-to-end AI pipelines that are transparently accelerated on a single node (with up to 9.6x speedup in our experiments) and seamlessly scaled out to a large cluster (across several hundreds of nodes in real-world use cases). It automatically provisions Big Data and AI systems (such as Ray and Apache Spark) for the distributed execution; on top of the underlying systems, it efficiently implements the distributed, in-memory data pipelines (for Spark Dataframes, TensorFlow Dataset, PyTorch, DataLoader, as well as arbitrary Python libraries), and transparently scales out deep learning (such as TensorFlow and PyTorch) training and inference on the distributed dataset (through scikit-learn style APIs). BigDL 2.0 has already been adopted by many real-world users (such as Mastercard, Burger King, Inspur, etc.) in production. In this session, we will demonstrate how to build an end-to-end AI pipeline using BigDL2.0 on Ray, and showcase real-world BigDL use cases.

AI Framework Engineer

BigDL 2.0: Seamless Scaling of AI Pipelines from Laptops to Distributed Cluster

Ray AI Runtime (AIR) is an open-source toolkit for building end-to-end machine learning applications. By leveraging Ray and its library ecosystem, Ray AIR brings scalability and programmability to ML platforms. In this session, we'll be exploring Ray AIR functionality and features, along with some benchmarks demonstrating its utility.

Engineering Manager

Software Engineer

Introducing Ray AI Runtime

In this talk, we are going to discuss how to run a question-answering system on Ray Serve. To that end, we'll cover the key architectural components used in question-answering systems, such as a data store, indexing pipeline, and querying pipeline. Haystack is an open source framework that allows you to connect a collection of multiple transformer state-of-the-art NLP models into a single pipeline. Using Ray, we'll discuss how to deploy it all for a GPU-empowered inference. Key learnings include how to assemble NLP models into pipelines for empowered inference; how to run Hugging Face models on Ray Serve; how to deploy a NLP model pipeline using Ray Serve; and how to access persistent storage from code deployed on Ray Serve.

Senior Software Engineer

Running a question-answering system on Ray Serve

At Uber, machine learning powers many business decisions to bring about experiences that delight our customers. To empower our product development teams with ML technology, we have built the Michelangelo Machine Learning Platform as an end-to-end solution for standardizing ML application development from exploration to production. In Michelangelo's early years, we centered our technology stack on Apache Spark, a large-scale data processing and classical ML training platform. Recent mass adoption of deep learning by our product teams has driven us to rethink our Spark-based technical architecture. Compared to classical ML techniques such as linear/logistics regression and tree-based methods, deep learning poses significant new infrastructure challenges in compute, network, and storage. In this talk, we will present case studies to highlight some of the specific challenges in large-scale deep learning training, and how we address them by leveraging Ray, a distributed compute platform.

Senior Engineering Manager

Staff Software Engineer

Sr. Software Engineer

Large-scale deep learning training and tuning with Ray at Uber

Building machine/deep learning models is a tough task; it involves multiple sub-tasks such as data preparation, model architecture creation, hyperparameter tuning, and extensive experimentation. Each set of hyperparameters creates a new experiment and if not done in an efficient manner, can be highly compute intensive, resulting in higher compute costs and time. Ray Tune gives us out-of-the-box features to manage compute resources efficiently. Its scheduling algorithms give us the opportunity to cut down the compute required by pruning bad trials and exploit the same resources to schedule new trials. Also, with hyperparameter tuning and extensive experimentation comes the problem of tracking, logging all these experiments, and saving the best model. Integration of Ray Tune with tools like MLflow and Weights & Biases saves the day. We will talk about how we have used Ray Tune to reduce manual efforts, compute costs, and logging hassle while training our models.

Lead AI Engineer

Reducing cost, latency, and manual efforts in hyperparameter tuning

One of the challenges of using deep learning in production is managing the cost of loading models for inference. In this talk, we'll show how you can reduce this cost almost to zero by leveraging features of PyTorch and Ray. We'll introduce the concept of zero-copy model loading, storing the weights of a deep learning model in shared memory so that any process with access to the shared memory segment can load the model nearly instantaneously. We'll show, with code examples, how to implement zero-copy model loading with PyTorch and Ray. Then we'll introduce our open-source library, zerocopy, which lets you apply zero-copy model loading to your PyTorch model by changing two lines of Python code in your Ray application. We'll finish off with a simple benchmark study that shows how zero-copy model loading lets you run NLP models with stateless Ray tasks instead of heavyweight actors, giving you a self-tuning model deployment that delivers dramatically better performance than a naive deployment of the same models to Ray Serve.

Principal Research Staff Member

Zero-copy model loading with Ray and PyTorch

Lunch 

Forecasting of disk utilization is crucial for one of KocDigital's clients, a leading private corporate cloud company that operates three data centers in Turkey. Full disk utilization leads to severe availability issues that undermine the company's reputation, so the company needs to be able to forecast disk utilization for several weeks, observe the risk of exceeding a critical utilization threshold, and take precautions to prevent availability issues. Forecasting a single server's disk utilization is a relatively simple task that requires time-series forecasting capabilities, but producing forecasts for tens of thousands of servers each and every day would be extremely challenging. In this talk, we'll share how Ray helped KocDigital ML engineers accelerate the training speed (finding out the best fitting model for each server) and generate inference almost 10 times to create a robust forecasting solution for the company. Ray Core (task and actor) and Ray Clusters were implemented successfully to overcome the computational complexity that stems from the scale of the problem.

Data & Analytics Director

Ray for large-scale time-series forecasting of disk utilization in a data center

Apache Beam provides a unified model for developing batch and streaming data analytics pipelines, and the Ray Beam Runner Project is an initiative to introduce a new Pythonic Apache Beam Runner for Ray. The project was conceived based on strong community interest in integrating Ray with Beam, and a prototype quickly proved its viability. We will discuss the current state of this initiative, and our long-term vision to provide a unified authoring and execution environment for mixed-purpose batch, streaming, and ML pipelines.

Sr. Software Development Engineer

The Ray Beam Runner Project: Unified batch, streaming, and ML

Hugging Face Transformers is a popular open-source project that features state-of-the-art machine learning for PyTorch, TensorFlow, and JAX. It allows for easy access to state-of-the-art NLP, computer vision, audio, and multimodal models. The computational and memory requirements for fine-tuning and training those models can be significant. To deal with this issue, the Ray team has developed a Hugging Face integration for Ray AI Runtime (AIR), allowing Transformers model training to be easily parallelized across multiple CPUs or GPUs in a Ray cluster, saving time and money, all the while allowing to take advantage of the rich Ray ML ecosystem thanks to a common API. In this session, we explore the integration between Hugging Face and Ray AIR, allowing users to scale their model training and data loading seamlessly. We will dive deep into the implementation and API and explore how we can use Ray AIR to create an end-to-end Hugging Face workflow, from data ingest through fine tuning and HPO to inference and serving.

Hugging Face + Ray AIR

Is your training infrastructure built on Kubernetes? Do you want to enable Ray on Kubernetes? Our ML platform is completely based on Kubernetes because of its scalability and rapid bootstrapping time of resources. In this talk we will demonstrate how we are leveraging Ray on Kubernetes to create an infrastructure to perform distributed training. We will showcase our custom SDKs that let users spawn on-demand Ray clusters to train models from notebooks. Our SDKs abstract and hide the complexities of spawning and bringing down the on-demand cluster from our users so that they can focus on the "what" while the platform takes care of the "how."

Staff Engineer

Lead Engineer

Distributed training with Ray on Kubernetes

In this 30-min talk, we will go over some of RLlib 2.0's biggest improvements, including: - A new way of configuring supported algorithms - Revamped core algorithm logics using plain Python implementations - Restructured algorithm and policy implementations that are easier to extend and customize - A feature that consolidates all user environment interactions to make restoring and serving RLlib policies simple and more robust - Most importantly, the integration between RLlib and the Ray AIR ecosystem Our goal for all this work is to make RLlib intuitive, easy to extend, and performant for various research and production use cases.

Machine Learning Engineer

What's new in RLlib

Machine learning is widely applied across various teams at Instacart to improve the experience for customers, shoppers, brand partners, and retailers. Instacart fulfillment ML helps more than 200,000 shoppers locate the fastest and most efficient delivery route for delivering products from 96,000 stores to hundreds of millions of homes all over the US and Canada. To predict regional-level fulfillment metrics, such as delivery ETA or supply/demand ratio, it is common to train and deploy one model per regional zone. At Instacart, we have around 2,000 unique regional zones for which we want to build an efficient job-queue system to highly parallelize model training workflows so that each re-training iteration can be completed as quickly as possible. In this talk, we'll cover how we previously solved for this using a queueing system built with Celery deployed on AWS ECS, and how by moving to Ray distributed frameworks we've been able to scale up to more parallel workers with less cost compared with the previous service on ECS, decreasing training time from 2-3 days to just a few hours.

Senior Machine Learning Engineer

Scaling Instacart fulfillment ML on Ray

Hamilton (https://github.com/stitchfix/hamilton) is an open source, declarative, general purpose, dataflow micro-framework, written in Python. It was originally created to manage complexities of scaling a team along with a time-series feature engineering code base past thousands of features at Stitch Fix. At a high level, in this talk, we'll cover: what Hamilton is and why it was created, how to use it for feature engineering, and how you can scale computation easily with the out-of-the-box Ray integration. At a low level, through code in the slides and a quick demo, you'll walk away with an understanding how a Data Science team at Stitch Fix scaled their team and code base with Hamilton, what Hamilton is and the declarative API paradigm it prescribes as opposed to traditional approaches, and lastly how the Ray integration with Hamilton works and how you can utilize it.

Mgr. Data Platform

Scalable feature engineering with Hamilton on Ray

In this talk, we aim to show how to leverage the programmable and general-purpose distributed computing ability of Ray to facilitate the authoring, orchestrating, scaling, and deployment of complex serving graphs as a DAG under one set of APIs, like a microservice. Learn how you can program multiple models dynamically on your laptop as if you're writing a local Python script, deploy to production at scale, and upgrade individually.

Multi-model composition with Ray Serve deployment graphs

Graph processing aims to process super large-scale graphs and execute graph data analysis tasks, such as PageRank, TriangleCount, community detection, etc. Most graph processing systems are under in-memory architecture, which makes it hard to process real-world large graphs with a gigantic number of edges (e.g., 100 trillion edges) at a cheap cost. In addition, few graph processing systems really implement serverless fault tolerance capability, hindering wide adoption in production. Hence, at ByteDance, we proposed an enterprise-level Graph Analytics Platform (GAP) for graph computing and graph mining, named ByteGAP, to process super large graphs such as Douyin and TikTok. ByteGAP is built on a serverless engine atop KubeRay to provide flexible cluster resource management, automatic deployment in the cloud, elasticity for scalability, and fault tolerance. It eases the maintenance and deployment effort and lowers the number of machines and memory consumption but supports much larger graphs by a hierarchical out-of-core design of DRAM/PMEM/SSDs on Ray clusters. Thanks to Ray's powerful abstraction of tasks/actors/GCS, we have built a Ray-based control plane for rank management, agent (actor) rendezvous, and stateful fault tolerance as an infrastructure component. It handles failures at node level, agent level, and worker level end-to-end with synergy that MPI can not fully cover. The Ray agents of dynamically assigned ranks manage worker processes of different languages as well as checkpoints in PMEM. It can relaunch agents and workers of specific ranks, load mutable status of vertices from vertex table in PMEM, and ensure that they are automatically recoverable from any iteration for any workers of arbitrary ranks.

Senior Researcher and Software Architect

A serverless resilient graph analysis platform built on Ray

Extended Networking Break

Ray gives you an easy and reliable method to scale your machine learning experiments horizontally. Ray also comes with an out of the box Weights & Biases integration so you can easily monitor and debug your training jobs. In this talk, I�ll show you how a few lines of code can take you from scanning opaque console logs to a transparent, complete, and collaborative view of your complex experiments.

Illuminate your distributed ML workloads with Weights & Biases

Counterfactuals, an active area of research in machine learning explainability, are explanations that produce actionable steps to move a data point from one side of a decision boundary to another. These explanations have a clear use case for several applications ranging from loan decisions to healthcare diagnosis, where they need to advise stakeholders about actions they can take to achieve a different outcome. Individuals not provided loans want steps they can take to achieve a loan, and similarly patients want to know how they can achieve a better diagnosis. This presentation showcases FastCFE, an algorithm and feature that uses reinforcement learning to provide real-time counterfactual explanations. Our presentation is broken down as follows: 1. Overview of counterfactuals and reinforcement learning (RL) 2. Deep distributed reinforcement learning using OpenAI Gym and Ray+RLlib and a demonstration using these frameworks. 3. Benchmarks and results

PhD Student

FastCFE: A counterfactual explainer using reinforcement learning

In collaboration with Microsoft and Vestas, we developed wind farm controllers that boost annual energy production by 1-2%. The controller achieves this by adjusting the yaw of upstream wind turbines in the farm to minimize wake losses at downstream turbines. The training of such deep reinforcement learning-based controllers for wind farms required large-scale and time-consuming computational fluid dynamics simulations. During the training of the controllers we used up to 15,000 CPU cores in parallel, using our platform DeepSim. DeepSim is minds.ai's end-to-end machine learning platform that uses Ray for distributed computing, model serving, and data processing. In this talk, we will discuss the project and how the controllers were efficiently trained in the cloud.

Neural Network Engineer

Wind farm optimization using RL and large-scale simulation

As a distributed computing framework, Ray works best in clustered mode, where multiple Ray workers can connect to the Ray head and execute tasks in parallel. Managing multiple Ray clusters and workloads in production is challenging, especially when workload patterns are different and may have different computational requirements and dependencies. It becomes more challenging when it requires multi-tenancy and large-scale performance in a cost-efficient way. KubeRay is an open source toolkit to run Ray applications on Kubernetes. KubeRay provides several tools to improve the experience of running Ray workloads on Kubernetes by extending the Kubernetes API and functionality to support the creation of Ray clusters of containers with a single command. In this talk, we will discuss the architectural decisions and show you how KubeRay easily manages heterogeneous resources, job submission lifecycle, application dependencies, and autoscaling.

Principal Software Engineer

KubeRay: A Kubernetes Ray clustering solution

In this talk we present highly available (HA) serving, one of the features of Ray 2.0. While Ray users thus far have been able to achieve high availability for their online serving workloads (with Ray Serve) using load-balancing across multiple Ray clusters, there is a material benefit to eliminating all single points of failure even within a single Ray cluster. HA serving within single Ray clusters allows users to experience less disruption during head-node failures, and can improve the efficiency of the cluster. This talk explains how HA serving works in Ray 2.0 at the architecture level, its supported functionality, and how you can deploy it in practice.

Highly available architectures for online serving in Ray

Alpa is a Ray-native library built for automatically training and serving large models (e.g., GPT-3). Alpa automates model-parallel training of large deep learning (DL) models by generating execution plans that unify data, operator, and pipeline parallelism. Existing model-parallel training systems either require users to manually create a parallelization plan or automatically generate one from a limited space of model parallelism configurations, which does not suffice to scale out complex DL models on distributed compute devices. Alpa distributes the training of large DL models by viewing parallelisms as two hierarchical levels: inter-operator and intra-operator parallelisms. Based on this, Alpa constructs a new hierarchical space for massive model-parallel execution plans. Alpa designs a number of compilation passes to automatically derive the optimal parallel execution plan in each independent parallelism level and implements an efficient runtime to orchestrate the two-level parallel execution on distributed compute devices. Our evaluation shows Alpa generates parallelization plans that match or outperform hand-tuned model-parallel training systems even on models they are designed for. Unlike specialized systems, Alpa also generalizes to models with heterogeneous architectures and models without manually designed plans. In this talk, we will focus on both the algorithm side and also the engineering/system implementation side, where Ray is a crucial building block of the Alpa runtime.

Postdoc

Alpa: Simple large model training and inference on Ray

In this introductory session, we'll discuss the motivation behind Ray Serve, who's using Ray Serve and why, and recent features and updates in Ray 2.0.

State of Ray Serve in 2.0

Production schedule design at Dow is a complex process that involves designing production cycles for multiple production units subject to manufacturing and scheduling constraints. Formulating this problem as a single large mixed integer linear program (MILP) optimization is computationally expensive. To solve this complex problem in reasonable time, we introduce a multi-agent decomposition approach that splits the problem into two separate hierarchical agents. In this talk, we will show how Ray has helped our team significantly reduce the solution time needed to solve this complex problem, leading to significant value for our end users, who can now solve and compare problem scenarios much more quickly, in turn leading to faster decision making for the overall business. Finally, we will highlight how our solution method, leveraging Ray, can be generalized and applied to a wide range of use cases in the MILP space.

Data Scientist

Ray for distributed mixed integer optimization at Dow

Modern large language models require distributed training strategies due to their size. The challenges of efficiently and robustly training them are met with rapid developments on both software and hardware frontiers. In this talk, we explore challenges and design decisions associated with developing a scalable training framework, and present a quantitative analysis of efficiency improvements coming from adopting new software and hardware solutions, such as Ray, JAX pjit, and TPUv4.

Scalable training of language models using Ray, JAX, and TPUv4

Traditionally, a workflow consists of a pipeline of tasks, executed and automated according to a set of procedural rules. Workflows enable coordinating and monitoring among distributed people, organizations, and tasks with strong durability, observability, and repeatability. Recently there is a growing trend of workflow-as-code for applications pipelines in favor of these properties, represented by workflow systems like AirFlow, Prefect, Temporal, and so on. However, many workflows today are data workflows: they are application pipelines that may pass and process large amounts of data between steps. Examples include ETL workloads and ML pipelines. These aforementioned workflow systems are less efficient and flexible for data processing, while Ray offers both efficiency and flexibility for data-intensive workloads. Combining the advantages of both Ray and a workflow system, we show that efficiency, durability, and flexibility can be achieved simultaneously on data pipelines with durable Ray tasks via Ray Workflow. This talk gives an introduction to Ray Workflow, how you can use Ray Workflow as durable Ray tasks, and how to program data pipelines with Ray Workflow. Ray Workflow will be available as alpha in Ray 2.0.

PhD Student, UC Berkeley

Ray Workflow: Durable Ray tasks for efficient, flexible data pipelines

Our epic community bash is being held at the San Francisco Exploratorium, Art of Tinkering exhibit. Come explore, play, and celebrate with the Ray Community. This is an exclusive interactive opportunity not to be missed!

Attend Our Welcome Community Bash  

Registration Opens

Breakfast   Networking

Day 2 keynote speakers are Richard Liaw and Bill Chambers, Anyscale; Smitha Shyam, Uber; Jaikumar Ganesh, Anyscale; Anca Dragan, UC Berkeley; Dario Gil, IBM; Phillip Moritz, Anyscale.

SVP & Director of Research

Assistant Professor

Keynotes, Day 2

In the past year, Ray has made so many additions including changes they made to the store cluster metadata, which was originally based on Redis. As the use cases have expanded to ML based worklaods, Ray and Redis can be used together in ML pipelines, and this reviving the relevance of Redis to the Ray architecture. From simply backing up cluster metadata to pulling real-time features from an online feature store, Ray can utilize Redis in several ML contexts. This talk will discuss the return of the Redi(s): how Redis and Ray can be used together in different types of ML pipelines.

Principal Applied AI Engineer

Return of the Redi: Redis and Ray in the ML Ecosystem

Extended Networking 

Shuffle is a key primitive in large-scale data processing applications. The difficulty of large-scale shuffle has inspired a myriad of implementations. While these have greatly improved shuffle performance and reliability over time, it comes at a cost: flexibility. We show that contrary to the popular wisdom, shuffle can be implemented with high performance and reliability on a general-purpose system for distributed computing: Ray. In this talk we present Exoshuffle, an application-level shuffle system that outperforms Spark and achieves 82% of theoretical performance on a 100TB sort on 100 nodes. In Ray 2.0, we have integrated Exoshuffle with the Datasets library to provide high-performance large-scale shuffle for ML users.

Large-scale data shuffle in Ray with Exoshuffle

As Qatar is gearing up to host the FIFA World Cup 2022, the mobility of the many thousands of people who will attend is a major hurdle for transport authorities. This is an unprecedented challenge considering the fact that all eight stadiums hosting the event are within a 20-mile radius of the city center. We at Qatar Computing Research Institute (QCRI) are working with the local transport authorities to model the traffic demands and plan a traffic control policy targeted at the event. In particular, we have developed: (a) a parallel, congestion-optimized, traffic microsimulator to analyze various what-if scenarios such as closure of a road segment, (b) a prediction model based on Graph Convolutional Networks for traffic in-flow estimation at a traffic light junction, and (c) a multi-agent reinforcement learning framework based on RLlib to design a coordinated traffic light control policy. We will present a brief overview of our work and walk through a demonstration of how we incorporated RLlib to speed up multi-agent learning for coordinated traffic light control.

Postdoctoral Researcher

Gearing up for FIFA 2022 using RLlib-powered traffic control

Spotify started evaluating managed Ray infrastructure as a distributed computing platform that enables ML research and experimentation for non-standardized ML workflows. Ray and its ecosystem provide researchers and data scientists at Spotify with a better model development experience, instant access to distributed computing, and a more expressive programming interface. These features greatly complement our existing production ML workflow. In this talk, we will share the story of how Ray started at Spotify, how we built and managed our Ray infrastructure, how Ray has been integrated with Spotify's internal infrastructure, and various best practices we have learned along this journey. Finally, we will also showcase some ML applications our users have built with the help of Ray during our evaluation.

ML Platform Engineer

How Spotify sped up ML research and prototyping with Ray

Are you looking to scale your ML pipeline to multiple machines? Are you encountering an ingest bottleneck, preventing you from saturating your GPUs? This talk will cover how Ray AIR uses Ray Datasets for efficient data loading and preprocessing for both training and batch inference, diving into how AIR uses Datasets to achieve high performance and scalability. We start by giving an overview of creating training and batch inference pipelines using Ray AIR. Next, we dive into the Ray Datasets internals, detailing features such as distributed data sharding, parallel + distributed I/O and transformations, pipelining of CPU and GPU compute, autoscaling pools of inference workers, and efficient per-epoch shuffling. Finally, we present case studies of users that have deployed such AIR workloads to production and have seen the performance + scalability benefits. You can learn more about Ray Datasets here: https://docs.ray.io/en/latest/data/dataset.html

Scaling Training and Batch Inference: A Deep Dive into AIR's Data Processing Engine

The TorchX team has been collaborating with the Ray team on easy ways to launch elastic large-scale distributed training jobs using TorchX entirely from a notebook, and this is now available as an experimental project. There are a few problems that have traditionally prevented anyone from doing this, such as needing an easy way to submit jobs against that infrastructure, devising an easy way to monitor job status and aggregate logs from several machines, and integrating that job with the rest of your infrastructure in a modular way. TorchX components can make it much easier to reuse code and experiment with different training infrastructures while you experiment with taking your work from research to production without needing to write a single new line of code.

Large-scale distributed training with TorchX and Ray

In 2021, Amazon launched an experiment to improve the scalability, performance, cost, and operational sustainability of its S3-based data catalog�s critical change data capture (CDC) workloads using Ray on EC2. This resulted in a promising production prototype ("The Flash Compactor") and a new open source Ray ecosystem project ("DeltaCAT") that brought order-of-magnitude improvements to the latency, cost, and scalability of these workloads. However, bridging the gap from petabyte-scale prototype to the exabyte-scale critical path demands a high-availability service that can manage thousands of concurrent job runs and offers near-real-time operational health insights. It also demands a seamless migration of our existing high-availability CDC workloads to Ray without disrupting critical pipelines. In this talk, we�ll discuss our progress in meeting these demands together with the problems encountered, solutions created, critical insights gained, and anticipated future work.

Paving the Road for Large-Scale Data Processing in Production with Ray

Production RL and decision-making with RLlib

At Cruise, we rely on custom data pre-processing before feeding data into ML models. In many cases ML engineers prefer to develop data pre-processing as part of their ML training code, making quick iterations and debugging much easier. This puts high pressure on performance and reliability of data pre-processing, because we need to make sure that by the time an ML model is ready to accept a next mini-batch of data, it is already available. Otherwise we will be wasting GPU time. Some of these data transformations could have run offline and cached, but there is no existing system available which would make it easy to move the logic between training and offline batch data processing jobs. Moreover, the memory usage of the data processing pipeline is a large consideration for us due to the use of high resolution sensors our cars have. In this talk, we will share our progress on building a new ML data pre-processing framework, Cruise.Data, and how Ray helps us to scale it. Cruise.Data is a novel system which combines best properties of tf.data, the PyTorch ecosystem, and large-scale data processing frameworks.

Sr Staff Software Engineer

Cruise.data: A new dataset processing pipeline for Cruise ML

Merlin, Shopify's new machine learning platform (https://shopify.engineering/merlin-shopify-machine-learning-platform), is based on an open source stack and tooling end-to-end. In this talk I will share a deeper look at the process, architecture, and how Merlin, using Ray, is helping us scale our ML work.

Machine Learning Platform Tech Lead

The Magic of Merlin: Shopify's new machine learning platform

At Riot Games, we leverage large-scale deep reinforcement learning to build bots that play our games at various skill levels to provide additional signals to our designers and ensure that we are releasing the best experiences for our players. As part of this experience we've experimented with multiple methods for controlling and tuning game servers to get the most out of our training time while the bots are learning to play the game. With our game Team Fight Tactics, we were able to take a unique approach that leveraged large neural networks to predict outcomes rather than have to rely on the server. In this talk, we will discuss the supervised learning process that we took and how we simplified and scaled with Ray Data, Ray Train, and Ray Tune.

Head of Tech Research

Large-scale deep learning to augment production RL workloads

Predibase is a new kind of low-code machine learning platform from the creators of Ludwig and Horovod, combining large-scale deep learning with state-of-the-art model architectures across a range of modalities, including NLP, computer vision, tabular, and hybrid data. Unlike existing AutoML solutions, Predibase provides a modular "lego brick" experience that allows data scientists and engineers alike to iteratively build the best model for their data without needing to master a low-level framework like PyTorch or TensorFlow. In this talk, we'll explain how Predibase uses Ludwig and Ray under the hood to deliver an end-to-end, cloud-native solution encompassing data processing, model training, hyperparameter optimization, and model serving, all done at scale. We'll also explore how Predibase provides a unique serverless abstraction over Ray, allowing us to support multi-cloud and multi-tenant deployments capable of handling many concurrent users and requests.

Chief Technology Officer

Predibase: A low-code deep learning platform built for scale

In this session, we will introduce you to a new declarative REST API for Ray Serve, which allows you to configure and update your Ray Serve applications without modifying application files. Incorporate this API into your existing CI/CD process to manage applications on Ray Serve as part of your MLOps lifecycle.

Operationalizing Ray Serve on Kubernetes

In this talk we will introduce SkyML, a project being developed at UC Berkeley's Sky Computing Lab for "infra-less" machine learning (ML) on any cloud. SkyML offers ML practitioners and researchers ease of use, performance, and cost saving. For ease of use, SkyML abstracts away the cloud and infrastructure on which an ML application, such as training or hyperparameter tuning, is run. For performance, SkyML allows the easy provisioning and usage of best-of-breed accelerators across clouds. For cost saving, SkyML offers support for spot instances and automatic cleanup of resources when they become idle. We'll describe the current system and share our experience on its usage with early users.

PhD Candidate

SkyML: Infra-less machine learning on any cloud

In this talk, we'll share case studies of how companies are using Weights & Biases and Ray as their MLOps infrastructure, and then give an overview of how to use Ray Tune with Weights & Biases Sweeps to perform hyperparameter optimization at scale.

CEO/Founder

Ray + Weights & Biases: Build and deploy real-world ML models

Graph models provide the best representation for data in many use cases in manufacturing, continuous and discrete, plus closely related business verticals such as pharma. This drives a growing demand for graph technologies applied in this domain. The core concept of bill of materials (BOM) linked to product data, linked to supplier networks, production planning, inventory data, linked with customer and sales data almost naturally translates into graphs. Meanwhile much the relevant input for AI in manufacturing is stored in documents, spreadsheets, or legacy data silos. NLP and deep learning applications help prepare such data for integration into graph models. The graph space has transformed over the past few years: graph neural networks provide exciting new capabilities, graph visualizations augmented by GPU break through previous barriers, algorithms of mathematical graph theory can be applied at scales hardly doable before. There are popular graph query languages and the W3C standards for ontologies and axiomatic inference and validation, and probabilistic graphs. Unfortunately, these camps within graph space remain largely disjoint. An open source project 'kglab' provides integration paths for different kinds of graph work, while aligning with PyData tools. Manufacturing firms including Siemens, Bosch, and BASF began using this library. A follow up project leverages Ray to provide graphs at billion-node scale, for horizontal scale-out of graph compute in large industrial use cases. This talk explores use cases for AI in Manufacturing, discussing where Ray can address critical bottlenecks at scale, and also helps augment work with NLP and deep learning. We'll consider the roles that Ray could play for more optimal graph technologies. Some points are counter-intuitive: for example, GNNs are quite useful, however the hard problems requiring graph algorithms at scale are often in data preparation (resolving ambiguity, detecting unwanted cycles, handling gaps and errors), which must occur long before training any GNN models. While there are numerous graph database vendors, few can handle scale, and their enterprise licensing costs often exceed the cloud computing costs for large clusters. Ray on K8s can be used to break through crucial limitations, allowing for open source integrations for managing large graphs within secure enterprise environments.

Managing Partner

Graphs at scale with Ray, for AI in Manufacturing

Ray recently added M1 ARM wheels, but building a heterogeneous Ray cluster on Kubernetes still has some rough edges, especially when you add the JVM to the mix. This talk will walk through how to cross-build different arch specific containers into combined images, deal with mixed Ubuntu versions, glibc versioning, and other things one generally hopes to avoid. At the end of this talk you will be able to deploy end-to-end Ray + Spark + Dask + Jupyter on a mixed architecture Kubernetes cluster, plus understand why you might not want to.

OSS Engineer

A cross-platform, cross-tool Ray on Kubernetes deployment

Evaluation of large-scale neural language models is crucial but also challenging. It requires models to be fine-tuned and evaluated on a large number of downstream tasks. Such downstream tasks can be very different in problem domain, type of input data, adoption pipeline, and running environment. Therefore, they are usually handled in different pipelines by different research teams. For example, one team might run a GLUE Benchmarking pipeline with 9 sub-tasks while another team might run a Sentiment Analysis pipeline with 17 sub-tasks. Such multi-task evaluation can be both time consuming (it can take a few days or even more) and hard to manage (across different teams and pipelines). Therefore, a toolkit that can support a unified pipeline for multi-task with easy scaling and independent resource management is highly desirable in this domain. By adopting Ray into our pipeline, we achieved easier auto-scaling, better resource management, and unified workflows for different tasks. In this talk, we will walk through the problem and demonstrate how we run our large-scale language model evaluation pipeline in a hybrid cloud with auto-scaling, and cover some details on how Ray helps unify the workflow pipeline with easy code modifications to achieve auto-scaling, dependency management, and better overall performance. This work is being developed as part of IBM's project CodeFlare.

Research Engineer

Evaluating large language models with Ray in hybrid cloud

BigDL 2.0: Seamlessly scaling end-to-end AI pipelines

ML offline and online serving users leverage Ray in radically different ways, which makes it challenging to provide a common architecture to inspect and debug performance problems. In this talk, we discuss Ray 2.0's current observability architecture, including how to view metrics, logs, and inspect the state of tasks, actors, and other resources in Ray. We discuss features like the task timeline and tracing, and present a roadmap for where Ray observability is going in the future with a unified observability data model. This talk is for you if you're interested in how to debug Ray programs, or are curious how one can implement observability in a general purpose distributed system like Ray.

Ray Observability: Present and future

Centered on Ray, the open-source distributed computing framework, Ray Summit brings together ML practitioners, data scientists, and more to discuss the growing demand for distributed computing.

Ray Summit 2022 Logo Dark Blue

Ray Summit 2022 Logo

Ray Logo

Anyscale

Missed us in San Francisco for Ray Summit 2022? Don't fret, you can watch your favorite sessions on demand! 

Presented by Anyscale

Ray Summit collage of speakers from event

Ray Summit collage of speakers from event, stacked vertically

Ray Summit 2022 - Keynote Speakers

Ray Summit is a great opportunity to level up your Ray skills through hands-on training sessions with the builders and maintainers of Ray. We’re offering several pre-conference training sessions on August 22. Space is limited — [reserve your spot now](https://www.myeventi.events/raysummit22/attendee/)!

Training image desktop

Training image mobile

As the annual user conference for the [open source Ray community](https://www.ray.io/community), Ray Summit is a unique opportunity to enhance your Ray skills, build your professional network, and learn what's on the Ray roadmap.

Extend what you can do with Ray by attending hands-on, expert-led workshops and deep dives

Mingle with the world’s top minds in distributed computing, machine learning, and AI

Learn what’s on the Ray roadmap and connect with the builders and maintainers of Ray and Ray libraries

Get ready for our epic community bash at the Art of Tinkering exhibit at the San Francisco Exploratorium on August 23 from 5:30 to 9:30 p.m. Come explore, play, and celebrate with the Ray Community. This is an exclusive opportunity to experience this amazing interactive exhibition before it closes in September.

Man taking a photo on his phone

Ready to get involved in the Ray community before the conference? Ask a question in the forums. Open a pull request. Or share why you’re excited with the hashtag __#RaySummit__ on Twitter.

Ray Summit 2022

Anyscale logo

Platinum Sponsors

Gold Sponsors

Silver Sponsors

Media Partners

Ray Summit is the ideal place to engage with the Ray community — a thriving group of machine learning, AI, and distributed computing professionals who are building the future of scalable Python applications. If you're interested in becoming a sponsor, please reach out to <a href=mailto:sponsorships@anyscale.com>sponsorships@anyscale.com</a> for more information.

Sponsoring Ray Summit is an opportunity to drive thought leadership for your brand, extend your reach, make meaningful connections, and more.

Showcase your technologies and services to influential Python and machine learning professionals, media, and analysts from around the world

Come away with a better understanding of the latest trends in machine learning, AI, and distributed software systems to inform future technology investments

Network with skilled Python and machine learning engineers, software architects, and managers and make valuable professional connections