All Posts
Attention Nets and More with RLlib's Trajectory View API
In this post, we’re announcing two new features now stable in RLlib: Support for Attention networks as custom models, and the “trajectory view API”. RLlib is a popular reinforcement learning library that is part of the open-source Ray project.
Online Resource Allocation with Ray at Ant Group
Double 11 has become the largest online shopping event in the world. To support this level of online activity, Ant Group has implemented a flexible, high-performance, stable, and scalable online resource allocation system based on Ray.
Executing a distributed shuffle without a MapReduce system
A distributed shuffle is a data-intensive operation that usually calls for a system built specifically for that purpose. In this blog post, we’ll show how a distributed shuffle can be expressed in just a few lines of Python using Ray, a general-purpo...
How to Speed Up Pandas with Modin
The pandas library provides easy-to-use data structures like pandas DataFrames as well as tools for data analysis. One issue with pandas is that it can be slow with large amounts of data. It wasn’t designed for analyzing 100 GB or 1 TB datasets. Fort...
Getting Started with Distributed Machine Learning with PyTorch and Ray
Ray is a popular framework for distributed Python that can be paired with PyTorch to rapidly scale machine learning applications.
Data Processing Support in Ray
This blog post highlights two features in the latest Ray 1.2 release: native support for spilling to external storage, and support for libraries from the Python data processing ecosystem, including integrations for PySpark and Dask.
Retrieval Augmented Generation with Huggingface Transformers and Ray
Huggingface Transformers recently added the Retrieval Augmented Generation (RAG) model, a new NLP architecture that leverages external documents (like Wikipedia) to augment its knowledge and achieve state of the art results on knowledge-intensive tas...
How to Speed up Scikit-Learn Model Training
This post gives an overview of different ways to speed up your scikit-learn models and discusses some limitations of each approach.
Configuring and Scaling ML with Hydra + Ray
Hydra, from Facebook AI, is a framework for elegantly configuring complex applications. Since its initial release, Hydra has become a popular framework adopted by researchers and practitioners. We are happy to announce that users can now scale and la...
Reinforcement Learning with RLlib in the Unity Game Engine
Train different agents inside the Unity3D game engine, thereby observing that their initial clumsy behaviors become more and more sophisticated and clever over time. We will use Ray RLlib, a popular open-source reinforcement learning library, in conn...