Practical Reinforcement Learning

July 8, 2020

The videos and slides for this event are now available.

Reinforcement Learning (RL) trains a system to perform a sequence of actions that maximize the rewards in an environment. RL has achieved fame recently as the approach used to beat the world’s best players of games like Go. You’ll hear from RL experts about the current state of the art in RL algorithms and tools, along with real-world applications of RL.

 

Schedule

10:00 AM: Scalable RL for TensorFlow, PyTorch, and Beyond, Eric Liang (Anyscale)

10:15 AM: Connecting Reinforcement Learning to Simulation Software, Max Pumperla (Pathmind)

10:30 AM: Build, train and deploy RLlib models on Amazon SageMaker RL, Anna Luo (Amazon Web Services)

10:45 AM: Panel discussion moderated by Dean Wampler with audience Q&A


Scalable RL for TensorFlow, PyTorch, and BeyondEric Liang (Anyscale)

Reinforcement learning is emerging as a practical tool for optimizing complex, unpredictable environments that can be simulated. For example, game artificial intelligence, system control, robotics, supply chain management, and finance. RLlib is an open-source library for reinforcement learning that offers both high scalability and a unified API for a variety of applications. In this talk we give an overview of RLlib, its design, and upcoming features.

Connecting Reinforcement Learning to Simulation SoftwareMax Pumperla (Pathmind)

Simulation software like AnyLogic is frequently used across many industries to model complex workflows of real-world applications. You can use these models to solve actual problems, and many of the questions that customers have can be framed as reinforcement learning tasks. However, by design, most simulation software providers do not have an integration with tools like Ray RLlib to apply reinforcement learning to their models directly.
We show how Pathmind leverages the best of both worlds by connecting existing simulation models to modern reinforcement learning tools. In particular, we discuss the special challenges we face and how we use Ray to address them by walking you through a concrete example.

Build, train and deploy RLlib models on Amazon SageMaker RLAnna Luo (Amazon Web Services)

Reinforcement learning jobs are computationally expensive requiring multiple CPU and GPU instances. Researchers and practitioners need to manually set up the instances and manage utilization, especially in production. This process can be time consuming and adds additional operational overhead. Amazon SageMaker RL allows customers to focus on their RL research without the need to manage servers. It provides pre-built containers that supports RLlib with both TensorFlow and PyTorch framework. We will present how you can bring in your RL problem and use Amazon SageMaker RL to perform the training. You can easily manage and reproduce your experiments with either single or multiple instances. For production use, we will showcase how you can deploy the trained model with a single click and monitor the endpoint status.

About Eric Liang

Eric Liang is a PhD candidate at UC Berkeley studying the intersection of systems and machine learning. Before grad school, he spent several years industry working on distributed systems at Google and Databricks. He currently leads Ray core and RLlib development at Anyscale.

Eric Liang headshot

About Anna Luo

Anna Luo is an Applied Scientist in AWS. She works on utilizing reinforcement learning techniques for different domains including supply chain and recommender system. She received her Ph.D. in Statistics from University of California, Santa Barbara. Her current personal goal is to master snowboarding

Anna Luo headshot

About Max Pumperla

Max is a deep learning engineer and prolific open source contributor. He’s maintainer of Hyperopt, DL4J core developer, Keras contributor and author of several Python libraries. He’s the author of “Deep Learning and the Game of Go” and Coursera instructor for “Applied AI with Deep Learning”. Max is co-founder of the deep learning platform aetros.com and holds a PhD in pure mathematics.

Max Pumperla headshot