All Posts

ThumbnailNo MapReduceSystem
03 . 22 . 2021

Executing a distributed shuffle without a MapReduce system

A distributed shuffle is a data-intensive operation that usually calls for a system built specifically for that purpose. In this blog post, we’ll show how a distributed shuffle can be expressed in just a few lines of Python using Ray, a general-purpo...

How to Speed Up pandas with modin (main image)
03 . 03 . 2021

How to Speed Up Pandas with Modin

The pandas library provides easy-to-use data structures like pandas DataFrames as well as tools for data analysis. One issue with pandas is that it can be slow with large amounts of data. It wasn’t designed for analyzing 100 GB or 1 TB datasets. Fort...

PyTorch + Ray
03 . 02 . 2021

Getting Started with Distributed Machine Learning with PyTorch and Ray

Ray is a popular framework for distributed Python that can be paired with PyTorch to rapidly scale machine learning applications.

02 . 16 . 2021

Data Processing Support in Ray

This blog post highlights two features in the latest Ray 1.2 release: native support for spilling to external storage, and support for libraries from the Python data processing ecosystem, including integrations for PySpark and Dask.

02 . 10 . 2021

Retrieval Augmented Generation with Huggingface Transformers and Ray

Huggingface Transformers recently added the Retrieval Augmented Generation (RAG) model, a new NLP architecture that leverages external documents (like Wikipedia) to augment its knowledge and achieve state of the art results on knowledge-intensive tas...

02 . 03 . 2021

How to Speed up Scikit-Learn Model Training

This post gives an overview of different ways to speed up your scikit-learn models and discusses some limitations of each approach.

Hydra+Ray (Anyscale)
01 . 26 . 2021

Configuring and Scaling ML with Hydra + Ray

Hydra, from Facebook AI, is a framework for elegantly configuring complex applications. Since its initial release, Hydra has become a popular framework adopted by researchers and practitioners. We are happy to announce that users can now scale and la...

Unity 3D game world
01 . 19 . 2021

Reinforcement Learning with RLlib in the Unity Game Engine

Train different agents inside the Unity3D game engine, thereby observing that their initial clumsy behaviors become more and more sophisticated and clever over time. We will use Ray RLlib, a popular open-source reinforcement learning library, in conn...

Ray x mlflow
01 . 13 . 2021

Ray & MLflow: Taking Distributed Machine Learning Applications to Production

In this blog post, we're announcing two new integrations with Ray and MLflow: Ray Tune+MLflow Tracking and Ray Serve+MLflow Models, which together make it much easier to build ML models and take them to production.

01 . 13 . 2021

Ray Summit 2021 CFP Now Open!

We are very excited to announce the 2021 Ray Summit, which will be held June 22 - 24 as a fully virtual event. We are now accepting proposals for conference talks and the deadline to submit is February 24, 2021.