Ray RLlib

Ray Train

Ray Datasets

RayDP

Retail

Gaming

Government

Financial

Ray Serve

Anyscale

Ray Core

Healthcare

Technology

News

User Story

Engineering

Culture

Anyscale Ray Team

In this blog post, we explore Joseph Spisak's insightful talk on "Llama: Scaling Up LLMs in an Open Ecosystem." Read the recap and watch the full session.

top 20 ray session joseph spisak meta

Llama, Scaling Up LLMs in an Open Ecosystem

Yi Cheng

Cade Daniel

Chen Shen

Liguang Xie

In this post, we discuss the importance of speed when loading large language models and what techniques we employed to make it 20x faster. In particular, we use the Llama 2 series of models. We share how you can reduce latency and costs using the Anyscale platform.

loading llms 20x faster

Loading Llama-2 70b 20x faster

Akshay Malik

Edward Oakes

Phi Nguyen

Learn how Ray Serve & Anyscale solves common challenges of serving AI in production - improving time to market, reducing cost, and ensuring reliability.

ray-serve-ga-blog

Tackling the Cost & Complexity of Serving AI in Production

Ray Serve: Tackling the cost and complexity of serving AI in production

Jules S. Damji

Richard Liaw

Richard Liaw headshot

ray 2-7 annoucement

Ray 2.7 features major stability improvements to Ray AI Libraries and KubeRay and introduces RayLLM

Stephanie Wang

Scott Lee

Cheng Su

Hao Chen

Eric Liang

Eric Liang headshot

Data loading & preprocessing can easily become the performance bottleneck in ML pipelines. With Ray Data, data loading can be fast, flexible and scalable.

featured image ml training with ray data

Fast, Flexible and Scalable Data Loading for ML Training

Fast, flexible, and scalable data loading for ML training with Ray Data

Yifei Feng

Sriram Sankar

Siddharth Venkatesh

Ameer Haj Ali

Discover how the Anyscale platform accelerates Gen AI & LLM app development, optimizes compute availability, reduces costs and more.

cloud infrastructure for llm and generative ai social

Cloud Infrastructure for LLM and Generative AI | Anyscale

Cloud Infrastructure for LLM and Generative AI Applications

Robert Nishihara

Joe Spisak

We are excited to announce collaboration between Meta and Anyscale to bolster the Llama ecosystem.

anyscale-meta

Anyscale and Meta Collaborate to Advance the Llama-2 Ecosystem

Artur Niederfahrenhorst

Kourosh Hakhamaneshi

Rehaan Ahmad

In this blog, we compare full-parameter fine-tuning with LoRA and answer questions around the strengths and weaknesses of the two techniques. 

lora blog 1

Fine-Tuning LLMs: In-Depth Analysis with LLAMA-2 | Anyscale

Fine-Tuning LLMs: LoRA or Full-Parameter? An in-depth Analysis with Llama 2

Vihan Lakshman

Pratik Pranav

Siddharth Jain

Tharun Medini

ThirdAI shares how to employ their BOLT framework along with Ray for distributed training of deep learning and foundational models using only CPUs.

training time vs number of nodes 1b

How ThirdAI uses Ray for Parallel Training of Billion-Parameter Neural Networks on Commodity CPUs

Waleed Kadous

Using Anyscale Endpoints, we compared Llama 2 7b, 13b and 70b vs. OpenAI's GPT-3.5-turbo and GPT-4 for accuracy and cost.

factuality 373 examples

Llama 2 vs. GPT-4: Nearly As Accurate and 30X Cheaper

Llama 2 is about as factually accurate as GPT-4 for summaries and is 30X cheaper

Amog Kamsetty

ByteDance, the company behind Tiktok, leverages multi-modal models to enable many applications, such as text-based image retrieval or object detection.

bytedance-actor-pool

How ByteDance Scales Offline Inference with Multi-Modal LLMs

How ByteDance Scales Offline Inference with multi-modal LLMs to 200 TB Data

We examine the Llama-2 models under 3 real-world use cases and show that fine-tuning yields significant accuracy improvements.

Llama 2 performance

Fine-Tuning Llama-2: Tailoring Models to Unique Applications

Fine-Tuning Llama-2: A Comprehensive Case Study for Tailoring Models to Unique Applications

Robin Singh

Anyscale releases Anyscale Endpoints Preview for LLM developers to run and fine-tune open-source LLMs fast and at scale.

Anyscale Endpoint Thumbnail

Anyscale Endpoints Preview: Fast, Cost-Efficient, and Scalable LLM APIs

Goku Mohandas

Made with ML and Anyscale

Machine Learning for Developers

2.6 animations

Ray 2.6 | Streaming for Serve & Train | Multi-GPU Learner API

Ray 2.6 features streaming for Serve and Train and new Multi-GPU Learner API

Eric Greene

The Anyscale Snowflake Connector is a Ray datasets capability that facilitates easy data transfer between Snowflake clusters & Anyscale-hosted Ray clusters.

Anyscale Snowflake Connector Main

Introducing the Anyscale Snowflake Connector | Anyscale

Introducing the Anyscale Snowflake Connector

Fine tuning is one approach to domain-specific model refinement (DSMR), but it’s not a silver bullet for improving domain-specific performance.

fine-tune-img-1

Fine Tuning Is For Form, Not Facts | Anyscale

Fine tuning is for form, not facts

Jerry Liu

In this blog, we showcase how you can use LlamaIndex and Ray to build a query engine to answer questions and generate insights about Ray itself.

llamaindex-img-3

Build and Scale a Powerful Query Engine with LlamaIndex & Ray

Build and Scale a Powerful Query Engine with LlamaIndex and Ray

The Ray Summit 2025 CfP is now open for submissions. The premier AI conference will be held November 3-5 in San Francisco, and the due date for CfPs is July 14th, 2025.  

Ray Summit 2025 CFP Open

Ray Summit 2025: Call for Proposals Closes on July 14th, 2025

Google + Anyscale

Google Cloud Integrates Anyscale's RayTurbo with GKE

Simplifying AI Development at Scale: Google Cloud Integrates Anyscale's RayTurbo with GKE

At Ray Summit 2024, we brought together AI experts and practitioners to dive into the future of distributed AI and scalable machine learning.

image3

Ray Summit 2024: Breaking Through the AI Complexity Wall

This section is used to order the "Types" and "Tags" that show up for filters on the Blog Index

Types

Products / Libraries

Blog

The Call for Speakers for Ray Summit 2025 has Been Extended!

Anyscale is the leading AI application platform. With Anyscale, developers can build, run and scale AI applications instantly.

All Posts