All Posts
Executing a distributed shuffle without a MapReduce system
A distributed shuffle is a data-intensive operation that usually calls for a system built specifically for that purpose. In this blog post, we’ll show how a distributed shuffle can be expressed in just a few lines of Python using Ray, a general-purpo...
How to Speed Up Pandas with Modin
The pandas library provides easy-to-use data structures like pandas DataFrames as well as tools for data analysis. One issue with pandas is that it can be slow with large amounts of data. It wasn’t designed for analyzing 100 GB or 1 TB datasets. Fort...
Getting Started with Distributed Machine Learning with PyTorch and Ray
Ray is a popular framework for distributed Python that can be paired with PyTorch to rapidly scale machine learning applications.
Data Processing Support in Ray
This blog post highlights two features in the latest Ray 1.2 release: native support for spilling to external storage, and support for libraries from the Python data processing ecosystem, including integrations for PySpark and Dask.
Retrieval Augmented Generation with Huggingface Transformers and Ray
Huggingface Transformers recently added the Retrieval Augmented Generation (RAG) model, a new NLP architecture that leverages external documents (like Wikipedia) to augment its knowledge and achieve state of the art results on knowledge-intensive tas...
How to Speed up Scikit-Learn Model Training
This post gives an overview of different ways to speed up your scikit-learn models and discusses some limitations of each approach.
Configuring and Scaling ML with Hydra + Ray
Hydra, from Facebook AI, is a framework for elegantly configuring complex applications. Since its initial release, Hydra has become a popular framework adopted by researchers and practitioners. We are happy to announce that users can now scale and la...
Reinforcement Learning with RLlib in the Unity Game Engine
Train different agents inside the Unity3D game engine, thereby observing that their initial clumsy behaviors become more and more sophisticated and clever over time. We will use Ray RLlib, a popular open-source reinforcement learning library, in conn...
Ray & MLflow: Taking Distributed Machine Learning Applications to Production
In this blog post, we're announcing two new integrations with Ray and MLflow: Ray Tune+MLflow Tracking and Ray Serve+MLflow Models, which together make it much easier to build ML models and take them to production.
Ray Summit 2021 CFP Now Open!
We are very excited to announce the 2021 Ray Summit, which will be held June 22 - 24 as a fully virtual event. We are now accepting proposals for conference talks and the deadline to submit is February 24, 2021.