Ray RLlib

Ray Train

Ray Datasets

RayDP

Retail

Gaming

Government

Financial

Ray Serve

Anyscale

Ray Core

Healthcare

Technology

News

User Story

Engineering

Culture

Jiao Dong

Hao Zhang

Lianmin Zheng

Jun Gong

Jules S. Damji

Phi Nguyen

In part 2 of our generative AI blog series, we cover how Ray empowers large language models (LLM) frameworks such as Alpa.

Figure 5: Technical integration layered stack for LLM


Figure-5

High-Performance LLM Training at 1000 GPU Scale With Alpa & Ray

Training 175B Parameter Language Models at 1000 GPU scale with Alpa and Ray

Antoni Baum

Eric Liang

Eric Liang headshot

Kai Fricke

Richard Liaw

Richard Liaw headshot

Learn the infrastructure challenges for supporting workloads production deployments around foundation models, and how Ray tackles these challenges.

layered-stack

How Ray Solves Generative AI & LLM Infrastructure Challenges

How Ray solves common production challenges for generative AI infrastructure

Anyscale Ray Team

How companies use Ray to scale many Python jobs at massive scale. Scaling many parallel Python jobs can come in many different forms. You can use Ray to train one million jobs in record time, scale many simulations or train the number of experiments that are part of a hyperparameter job.


summit-stories-scaling-parallel-python-jobs

How companies are scaling parallel python jobs for many models training, simulation or hyperparameter tuning.

Ray Summit Series - Scaling Parallel Python Jobs

Ray Summit Stories - ML Workloads (1)

How companies are scaling their ML workloads and driving efficiency and productivity

Ray Summit 2022 stories - ML Platforms

Clarence Ng

Learn about Ray's new out of memory (OOM) monitor and detection feature — all part of our efforts to make Ray easy to observe & debug for ML engineers.

Figure 2: OOM monitor high-level architecture and data flow

Figure 2

Automatic Memory Scheduling for ML Workloads in Ray

Automatic and optimistic memory scheduling for ML workloads in Ray

Uday Kanwar

Deb Daipayan

Blue River Tractor

Blue River Technology is revolutionizing agriculture by applying advanced computer vision and robotics to the problem of crop production using the Anyscale fully-managed Ray platform

Blue River Technology Developers Iterate 2.5X Faster with the Anyscale Fully-Managed Ray Platform

Cade Daniel

Zhe Zhang

The Ray 2.3 release features exciting improvements across the Ray ecosystem. In this blog post, we will highlight new features, performance enhancements, and support for new platforms. 

main_release_image

Announcing Ray 2.3: performance improvements, new features and new platforms

Large Language Models

Ray Summit 2022 Stories - Large Language Models

 Max Mergenthaler

Together, Ray, Fugue and Nixtla provide a powerful open-source solution for organizations looking to perform forecasting and anomaly detection at scale.

Nixtla and Ray

Forecasting at Scale With Nixtla and Ray | Anyscale

Forecasting at Scale

Using Anyscale with Weights & Biases simplifies MLOps with a built-in integration that allows for scalability, reproducibility and consistency.

Weights&Biasesflow

Simplify ML Development Cycle with Anyscale and Weights & Biases

Simplify your ML Development Cycle with Anyscale and Weights & Biases

Frank Sifei Luan, UC Berkeley

frank

Exoshuffle is now the most cost-efficient way to sort 100TB of data on the public cloud, using only $0.97 per terabyte. Learn how this is possible with Ray.

CloudSortBenchmark 01

Ray Is the World’s Most Cost-Efficient Sorting System at $1/TB

Ray breaks the $1/TB barrier as the world’s most cost-efficient sorting system

The Ray 2.2 release features improved observability, better performance for data-intensive AI applications, increased stability, and better UX for RLlib.

View the stack trace or CPU flame graph of a Ray worker process from the dashboard.

Dashboard Flame Graph

Ray 2.2 | Improved Experience, Performance & Stability

Ray 2.2: Improved developer experience, performance and stability

Learn how to conduct batch training using only Ray Core and stateless Ray tasks. We take an in-depth look at two approaches to employ Ray tasks to scale.

NYC taxi dirver data sete

How to Conduct Many-Model Batch Training at Scale with Ray

Many Models Batch Training at Scale with Ray Core

CfP Main page

Ray Summit 2023 | Share Your Ray Story

Ray Summit 2023 Call for Proposals is now open

Antoine Galataud

Inouk Bourgon

Adrien Lafond

foobot_architecture

Foobot optimizes building energy efficiency by training fully autonomous control agents for HVAC systems, bringing energy savings to office buildings, hospitals, schools and commercial buildings.

Robert Nishihara

More companies are needing to train & deploy many small ML models, often hundreds or thousands. Learn how Ray and Anyscale make that possible.

Many_model_image

Training 1 Million ML Models in Record Time | Anyscale

Training One Million Machine Learning Models in Record Time with Ray

Learn the 3 key features that will be critical to ensuring that your ML platform is well-positioned for modern AI applications.

Figure3-Shopify-Merlin-platform

Top 3 Key Elements of a Scalable ML Platform | Anyscale

Three Key Elements of a Scalable ML Platform 

Tengwei Cai, Yang Liu, Chengxi Luo, Xiaofeng Yang, Simon Mo

Calling Python Actor from Java Deployment

calling-pyton-deployment

Flexible, cross-language, distributed model inference framework: Ray Serve with Java API

The Ray Summit 2025 CfP is now open for submissions. The premier AI conference will be held November 3-5 in San Francisco, and the due date for CfPs is July 14th, 2025.  

Ray Summit 2025 CFP Open

Ray Summit 2025: Call for Proposals Closes on July 14th, 2025

Google + Anyscale

Google Cloud Integrates Anyscale's RayTurbo with GKE

Simplifying AI Development at Scale: Google Cloud Integrates Anyscale's RayTurbo with GKE

At Ray Summit 2024, we brought together AI experts and practitioners to dive into the future of distributed AI and scalable machine learning.

image3

Ray Summit 2024: Breaking Through the AI Complexity Wall

This section is used to order the "Types" and "Tags" that show up for filters on the Blog Index

Types

Products / Libraries

Blog

The Call for Speakers for Ray Summit 2025 has Been Extended!

Anyscale is the leading AI application platform. With Anyscale, developers can build, run and scale AI applications instantly.

All Posts