News

User Story

Engineering

Culture

Jules S. Damji

Richard Liaw

Richard Liaw headshot

Ray 2.8 features Ray data extensions to read data from external stores, extensions to Ray Dashboard, and AWS neuron cores support, along with enhancements and bug fixes.

Ray 2.8 blog post

Ray 2.8 features Ray Data extensions, AWS Neuron cores support, and Dashboard improvements

ray 2-7 annoucement

Ray 2.7 features major stability improvements to Ray AI Libraries and KubeRay and introduces RayLLM

2.6 animations

Ray 2.6 | Streaming for Serve & Train | Multi-GPU Learner API

Ray 2.6 features streaming for Serve and Train and new Multi-GPU Learner API

Ray_2.5_main_image

Ray 2.5 | Training & Serving for LLMs, Multi-GPU Training & More

Ray 2.5 features training and serving for LLMs, Multi-GPU training in RLlib, and enhanced Ray Data support

Ben Lorica

Ray_summit_2023_main_image

10 must-attend Ray Summit sessions: Generative AI, Reinforcement learning, scalable ML workloads, and more

10 must-attend Ray Summit sessions: Generative AI, scalable ML workloads, and more

Amog Kamsetty

Eric Liang

Eric Liang headshot

We conduct a comparison of three different solutions for offline batch inference: AWS SageMaker Batch Transform, Apache Spark, and Ray Data.

Figure 1. Throughput difference among three different solutions for 10GB batch inference


Figure 1

Offline Batch Inference | Ray, Apache Spark & SageMaker

Offline Batch Inference: Comparing Ray, Apache Spark, and SageMaker

Jiajun Yao

main_image

Announcing Ray 2.4.0: Infrastructure for LLM training, tuning, inference, and serving

Jiao Dong

Hao Zhang

Lianmin Zheng

Jun Gong

Phi Nguyen

In part 2 of our generative AI blog series, we cover how Ray empowers large language models (LLM) frameworks such as Alpa.

Figure 5: Technical integration layered stack for LLM


Figure-5

High-Performance LLM Training at 1000 GPU Scale With Alpa & Ray

Training 175B Parameter Language Models at 1000 GPU scale with Alpa and Ray

Clarence Ng

Learn about Ray's new out of memory (OOM) monitor and detection feature — all part of our efforts to make Ray easy to observe & debug for ML engineers.

Figure 2: OOM monitor high-level architecture and data flow

Figure 2

Automatic Memory Scheduling for ML Workloads in Ray

Automatic and optimistic memory scheduling for ML workloads in Ray

Cade Daniel

Zhe Zhang

The Ray 2.3 release features exciting improvements across the Ray ecosystem. In this blog post, we will highlight new features, performance enhancements, and support for new platforms. 

main_release_image

Announcing Ray 2.3: performance improvements, new features and new platforms

Antoni Baum

Learn how to conduct batch training using only Ray Core and stateless Ray tasks. We take an in-depth look at two approaches to employ Ray tasks to scale.

NYC taxi dirver data sete

How to Conduct Many-Model Batch Training at Scale with Ray

Many Models Batch Training at Scale with Ray Core

CfP Main page

Ray Summit 2023 | Share Your Ray Story

Ray Summit 2023 Call for Proposals is now open

Ray Summit 2024, the annual Ray community conference, is back in San Francisco, CA this September 30th through October 2nd. It is a two-day event with a third day dedicated to training.

Call for Proposals is open Blog Post

Ray Summit 2024 Call for Proposals is now open

Intel and Anyscale have integrated Ray with Gaudi3. Gaudi3 is AI-optimized for efficient training and other machine learning workloads.

Anyscale - Intel

Accelerating AI: Harnessing Intel(R) Gaudi(R) 3 with Ray 2.10

Anyscale is teaming with NVIDIA to combine the developer productivity of Ray Serve and RayLLM with the cutting-edge optimizations from NVIDIA Triton Inference Server software and the NVIDIA TensorRT-LLM library.

NVIDIA-Anyscale Logo 1280x680.jpg

Low-latency Generative AI Model Serving with Ray, NVIDIA Triton Inference Server, and NVIDIA TensorRT-LLM

Generating embeddings is a critical task for developing successful RAG applications, and Anyscalew with Pinecone is the most cost-efficient solution for this workflow.

blog-anyscale-pincone

RAG at Scale: 10x Cheaper Embedding Computations with Anyscale and Pinecone

This section is used to order the "Types" and "Tags" that show up for filters on the Blog Index

Types

Products / Libraries

Blog

Hosted Anyscale is in Private Preview. Click here to request access!

Anyscale is the leading AI application platform. With Anyscale, developers can build, run and scale AI applications instantly.

Anyscale

Posts by Jules S. Damji

No posts found.