Ray RLlib

Ray Train

Ray Datasets

RayDP

Retail

Gaming

Government

Financial

Ray Serve

Anyscale

Ray Core

Healthcare

Technology

Ray Tune

News

User Story

Engineering

Culture

Akshay Malik

Edward Oakes

Phi Nguyen

ray-serve-ga-blog

Ray Serve: Tackling the cost and complexity of serving AI in production

Jules S. Damji

Richard Liaw

Richard Liaw headshot

ray 2-7 annoucement

Ray 2.7 features major stability improvements to Ray AI Libraries and KubeRay and introduces RayLLM

Stephanie Wang

Scott Lee

Cheng Su

Hao Chen

Eric Liang

Eric Liang headshot

Data loading and preprocessing can easily become the performance bottleneck in actual ML pipelines. Data preprocessing is also becoming more complex. With Ray Data, data loading can be fast, flexible, and scalable.

featured image ml training with ray data

Fast, flexible, and scalable data loading for ML training with Ray Data

Yifei Feng

Sriram Sankar

Siddharth Venkatesh

Ameer Haj Ali

We dive into Anyscale’s evolution to accelerate Gen AI and LLM app development and what enables our Anyscale LLM Endpoints offering. Discover how we've drastically accelerated startup times, optimizing compute availability, enhanced security & access controls, reduced costs, improved spot instance support, provided fast, scalable storage solutions, and more!

cloud infrastructure for llm and generative ai social

Cloud Infrastructure for LLM and Generative AI Applications

Goku Mohandas

Philipp Moritz

In this guide, we will learn how to develop and productionize a retrieval augmented generation (RAG) based LLM application, with a focus on scale, evaluation and routing.  

social-rag-based-llm

Building RAG-based LLM Applications for Production (Part 1)

Robert Nishihara

Joe Spisak

We are excited to announce collaboration between Meta and Anyscale to bolster the Llama ecosystem.

anyscale-meta

Anyscale and Meta Collaborate to Advance the Llama-2 Ecosystem

Artur Niederfahrenhorst

Kourosh Hakhamaneshi

Rehaan Ahmad

In this blog, we compare full-parameter fine-tuning with LoRA and answer questions around the strengths and weaknesses of the two techniques. 

lora blog 1

Fine-Tuning LLMs: LoRA or Full-Parameter? An in-depth Analysis with Llama 2

Vihan Lakshman

Pratik Pranav

Siddharth Jain

Tharun Medini

ThirdAI shares how to employ their BOLT framework along with Ray for distributed training of deep learning and foundational models using only CPUs.

training time vs number of nodes 1b

How ThirdAI uses Ray for Parallel Training of Billion-Parameter Neural Networks on Commodity CPUs

Waleed Kadous

factuality 373 examples

Llama 2 is about as factually accurate as GPT-4 for summaries and is 30X cheaper

Amog Kamsetty

Liguang Xie

At ByteDance, the company behind Tiktok, we leverage multi-modal models to enable many applications, such as text-based image retrieval or object detection. 

bytedance-actor-pool

How ByteDance Scales Offline Inference with multi-modal LLMs to 200 TB Data

In this blog, we provide a thorough analysis and a practical guide for fine-tuning. We examine the Llama-2 models under three real-world use cases, and show that fine-tuning yields significant accuracy improvements across the board (in some niche cases, better than GPT-4). 

Llama 2 performance

Fine-Tuning Llama-2: A Comprehensive Case Study for Tailoring Models to Unique Applications

Robin Singh

Anyscale releases Anyscale Endpoints Preview for LLM developers to run and fine-tune open-source LLMs fast and at scale.

Anyscale Endpoint Thumbnail

Anyscale Endpoints Preview: Fast, Cost-Efficient, and Scalable LLM APIs

This section is used to order the "Types" and "Tags" that show up for filters on the Blog Index

Types

Products / Libraries

Blog

From the creators of Ray, Anyscale is a framework for building machine learning applications at any scale originating from the UC Berkeley RISELab.

Featured Posts and News

Building RAG-based LLM Applications for Production (Part 1)

Fine-Tuning LLMs: LoRA or Full-Parameter? An in-depth Analysis with Llama 2

Llama 2 is about as factually accurate as GPT-4 for summaries and is 30X cheaper

No posts found.