Product

News

User Story

Engineering

Culture

Jeffrey Wang

Kourosh Hakhamaneshi

Abrar Sheikh

Seiji Eicher

See how Ray Serve LLM updates in Ray 2.55 enable DP group fault tolerance for vLLM Wide Expert Parallelism (WideEP)

DP Group Fault Tolerance for vLLM

Announcing DP Group Fault Tolerance for vLLM WideEP Deployments with Ray Serve LLM

Alex Yang

Akshay Malik

Spencer Peterson (Google)

Ray Serve upgrades deliver faster online inference with HAProxy and gRPC, enabling scalable LLM inference on Ray with lower latency and higher throughput.

Major upgrades to Ray Serve: Online Inference with 88% lower latency and 11.1x higher throughput

Ana Biazetti (RedHat)

Richard Liaw

Richard Liaw headshot

Cathal O’Connor (RedHat)

Combine Ray Data + Docling to build scalable document parsing pipelines that process 10k+ complex files with heterogeneous (CPU+GPU) execution and high-accuracy document understanding.

Thumbnail for blog post Breaking the RAG Bottleneck: Scalable Document Processing with Ray Data and Docling

Breaking the RAG Bottleneck: Scalable Document Processing with Ray Data and Docling

Accelerate service debugging with new Ray Serve Grafana dashboard on Anyscale. Available with Ray v2.54+

grafana dashboard thumbnail 1500x1000

Ray Serve’s New Grafana Dashboard: Debugging That Wasn’t Possible Before

Omar Shorbaji

OMAR SHORBAJI

Learn how Vision-Language-Action (VLA) models enable robotics and embodied AI, and how Ray on Anyscale helps teams scale VLA training and deployment.


Scaling Vision-Language-Action (VLA) Pipelines for Robotics with Ray on Anyscale

Vision Language Action Models on Anyscale

Julian Forero

Ian Jordan, PhD

Distributed AI training with Ray on Anyscale: Run PyTorch, XGBoost and DeepSpeed across multi-node, multi-GPU clusters with high efficiency and reliability

Scalable Distributed Training: From Single-GPU Limits to Reliable Multi-Node Runs with Ray on Anyscale

David Wang

GPUs are often underutilized in production AI. Learn why CPU-centric architectures waste GPU capacity and how AI-native execution improves efficiency.

gpu inefficiency thumbnail 2x

GPU (In)efficiency in AI Workloads

KubeRay Team

KubeRay v1.5 introduces several enhancements for running Ray on Kubernetes, incorporating over 180 commits contributed by approximately 50 contributors.


Kuberay v1.5

Introducing KubeRay v1.5 

Masahiro Tanaka

Amjad Almahairi

Nithin Chalapathi

Matthew Deng

Praveen Gorthy

Stephanie Wang

multimodal ai training thumbnail 2x

30% Faster Multimodal AI Training with Ray and Disaggregated Hybrid Parallelism

Rui Qiao

We are excited to announce new Ray Serve LLM APIs that make it easy to deploy state of the art serving patterns with vLLM

vLLM Ray Serve Anyscale - 3

Ray Serve LLM on Anyscale: Wide-EP and Disaggregated Serving with vLLM

Ray Serve LLM on Anyscale: APIs for Wide-EP and Disaggregated Serving with vLLM

Heading to AWS re:Invent? Join us at booth #1854, book a 1:1 meeting or learn more across breakouts, lightning talks and executive round tables. 


anyscale re:invent 2025

AI with Ray and Anyscale at AWS re:Invent 2025

Emre Saglam

Every Ray server component now enforces authentication through a token validation middleware. This ensures that only requests with the correct token are processed by Ray’s internal and external services.


ray token based auth

Introducing Token-Based Authentication for Ray

Ray has become the engine for AI-native computing for thousands of AI- and digital-natives. Now Anyscale and Microsoft are partnering to bring the power of Ray to every enterprise.

Announcing first-party service - Anyscale on Azure

Announcing Anyscale on Azure: Build, Run and Scale AI-Native Workloads Securely on Azure Infrastructure

Ray, the open-source distributed compute framework for AI is joining the PyTorch Foundation, part of The Linux Foundation. 

Ray joins PyTorch Foundation

Ray is Joining The PyTorch Foundation

opensource stack blog thumbnail

Components of an Open Source AI Compute Tech Stack

An Open Source Stack for AI Compute: Kubernetes + Ray + PyTorch + vLLM

Explore a technical comparison of leading Reinforcement Learning (RL) libraries for LLMs from Ray. This guide analyzes frameworks like TRL, Verl, and RAGEN to help developers choose the best tools for RLHF, reasoning, and agentic AI.

The architecture of a Reinforcement Learning (RL) library is split into two primary components: Generation and Training. During the generation phase, an LLM Engine performs multi-turn rollouts within an environment to produce data and reward signals. This output is then fed into the training phase to update the model's parameters. This process forms a feedback loop, where the progressively improved model generates the next iteration of data for continuous refinement.

Open Source RL Libraries for LLMs Graph

Open Source RL Libraries for LLMs

This section is used to order the "Types" and "Tags" that show up for filters on the Blog Index

Types

Products / Libraries

Blog

Catch Ray on the roads - Register for your city

Powered by Ray, Anyscale empowers AI builders to run and scale all ML and AI workloads on any cloud and on-prem.

Learn why Anyscale is best place to run Ray

Free, self-paced curriculum for practitioners

One-click to launch. Start fast, make it yours

Blogs