Ray RLlib

Ray Train

Ray Datasets

RayDP

Retail

Gaming

Government

Financial

Ray Serve

Anyscale

Ray Core

Healthcare

Technology

Ray Tune

News

User Story

Engineering

Culture

Jules S. Damji

Richard Liaw

Richard Liaw headshot

ray 2-7 annoucement

Ray 2.7 features major stability improvements to Ray AI Libraries and KubeRay and introduces RayLLM

2.6 animations

Ray 2.6 | Streaming for Serve & Train | Multi-GPU Learner API

Ray 2.6 features streaming for Serve and Train and new Multi-GPU Learner API

Cade Daniel

Chen Shen

Eric Liang

Eric Liang headshot

In this blog, we discuss continuous batching, a critical systems-level optimization that improves both throughput and latency under load for LLMs.

Cover image for 23X batch blog

Achieve 23x LLM Inference Throughput & Reduce p50 Latency

How continuous batching enables 23x throughput in LLM inference while reducing p50 latency

Ray_2.5_main_image

Ray 2.5 | Training & Serving for LLMs, Multi-GPU Training & More

Ray 2.5 features training and serving for LLMs, Multi-GPU training in RLlib, and enhanced Ray Data support

Jiajun Yao

main_image

Announcing Ray 2.4.0: Infrastructure for LLM training, tuning, inference, and serving

Waleed Kadous

Jun Gong

Antoni Baum

In part 4 of our Generative AI series, we share how to build a system for fine-tuning & serving LLMs in 40 minutes or less.

fine-tune-stack

How to Fine-Tune a 6 Billion Parameter LLM for Less Than $7

How to fine tune and serve LLMs simply, quickly and cost effectively using Ray + DeepSpeed + HuggingFace

Kai Fricke

Learn the infrastructure challenges for supporting workloads production deployments around foundation models, and how Ray tackles these challenges.

layered-stack

How Ray Solves Generative AI & LLM Infrastructure Challenges

How Ray solves common production challenges for generative AI infrastructure

Zhe Zhang

The Ray 2.3 release features exciting improvements across the Ray ecosystem. In this blog post, we will highlight new features, performance enhancements, and support for new platforms. 

main_release_image

Announcing Ray 2.3: performance improvements, new features and new platforms

The Ray 2.2 release features improved observability, better performance for data-intensive AI applications, increased stability, and better UX for RLlib.

View the stack trace or CPU flame graph of a Ray worker process from the dashboard.

Dashboard Flame Graph

Ray 2.2 | Improved Experience, Performance & Stability

Ray 2.2: Improved developer experience, performance and stability

Paige Bailey

Jian Xiao

Chandler Gibbons

blog-ray-1-12-thumb

blog-recommended-content-clock-dark

Ray 1.12: Ray AI Runtime (alpha), usage data collection, and more

Juan Navas

blog-distrubuted-hyperparameter-tuning-processing-thumb

How to Distribute Hyperparameter Tuning Using Ray Tune

How to distribute hyperparameter tuning using Ray Tune

hands-on-hyperparameter-tuning-MNIST-samples

How to tune hyperparameters on XGBoost

In this guide, we will learn how to develop and productionize a retrieval augmented generation (RAG) based LLM application, with a focus on scale, evaluation and routing.  

social-rag-based-llm

Building RAG-based LLM Applications for Production (Part 1)

In this blog, we compare full-parameter fine-tuning with LoRA and answer questions around the strengths and weaknesses of the two techniques. 

lora blog 1

Fine-Tuning LLMs: LoRA or Full-Parameter? An in-depth Analysis with Llama 2

factuality 373 examples

Llama 2 is about as factually accurate as GPT-4 for summaries and is 30X cheaper

In this blog, we provide a thorough analysis and a practical guide for fine-tuning. We examine the Llama-2 models under three real-world use cases, and show that fine-tuning yields significant accuracy improvements across the board (in some niche cases, better than GPT-4). 

Llama 2 performance

Fine-Tuning Llama-2: A Comprehensive Case Study for Tailoring Models to Unique Applications

This section is used to order the "Types" and "Tags" that show up for filters on the Blog Index

Types

Products / Libraries

Blog

From the creators of Ray, Anyscale is a framework for building machine learning applications at any scale originating from the UC Berkeley RISELab.

Posts by Richard Liaw

No posts found.