Lightning Talk

The Ray Beam Runner Project: Unified batch, streaming, and ML

Ray Summit 2022

Apache Beam provides a unified model for developing batch and streaming data analytics pipelines, and the Ray Beam Runner Project is an initiative to introduce a new Pythonic Apache Beam Runner for Ray. The project was conceived based on strong community interest in integrating Ray with Beam, and a prototype quickly proved its viability. We will discuss the current state of this initiative, and our long-term vision to provide a unified authoring and execution environment for mixed-purpose batch, streaming, and ML pipelines.

About Patrick

Patrick Ames is a senior software engineer working on data management and optimization for big data technologies at Amazon.

About Pablo

Pablo Estrada is a software engineer at Google, and a PMC member for Apache Beam. He is a big fan of Ray, and has worked to integrate Beam and Ray. Pablo loves participating in the OSS community.

Patrick Ames

Sr. Software Development Engineer, Amazon

Pablo Estrada

Software Engineer, Google
Ray Summit 2022 horizontal logo

Ready to Register?

Come connect with the global community of thinkers and disruptors who are building and deploying the next generation of AI and ML applications.

Save your spot

Join the Conversation

Ready to get involved in the Ray community before the conference? Ask a question in the forums. Open a pull request. Or share why you’re excited with the hashtag #RaySummit on Twitter.