Ray RLlib

Ray Train

Ray Datasets

RayDP

Retail

Gaming

Government

Financial

Ray Serve

Anyscale

Ray Core

Healthcare

Technology

Ray Tune

News

User Story

Engineering

Culture

Waleed Kadous

Kyle Huang

Wendi Ding

Liguang Xie

Avnish Narayan

Ricky Xu

Anyscale is releasing LLMPerf for benchmarking LLMs on current LLM offerings. See benchmarking results for Anyscale Endpoints vs Fireworks.ai.

End-to-end time

Reproducible Performance Metrics for LLM inference

In this post, we discuss the importance of speed when loading large language models and what techniques we employed to make it 20x faster. In particular, we use the Llama 2 series of models. We share how you can reduce latency and costs using the Anyscale platform.

loading llms 20x faster

Loading Llama-2 70b 20x faster with Anyscale Endpoints

In this guide, we will learn how to develop and productionize a retrieval augmented generation (RAG) based LLM application, with a focus on scale and evaluation.  

social-rag-based-llm

Building RAG-based LLM Applications for Production

In this blog, we compare full-parameter fine-tuning with LoRA and answer questions around the strengths and weaknesses of the two techniques. 

lora blog 1

Fine-Tuning LLMs: In-Depth Analysis with LLAMA-2 | Anyscale

Fine-Tuning LLMs: LoRA or Full-Parameter? An in-depth Analysis with Llama 2

Using Anyscale Endpoints, we compared Llama 2 7b, 13b and 70b vs. OpenAI's GPT-3.5-turbo and GPT-4 for accuracy and cost.

factuality 373 examples

Llama 2 vs. GPT-4: Nearly As Accurate and 30X Cheaper

Llama 2 is about as factually accurate as GPT-4 for summaries and is 30X cheaper

This section is used to order the "Types" and "Tags" that show up for filters on the Blog Index

Types

Products / Libraries

Blog

Get started with Serving and Fine Tuning Open Source LLMs with Anyscale Endpoints!

Anyscale is the leading AI application platform. With Anyscale, developers can build, run and scale AI applications instantly.

Posts by Kyle Huang

No posts found.