Ray RLlib

Ray Train

Ray Datasets

RayDP

Retail

Gaming

Government

Financial

Ray Serve

Anyscale

Ray Core

Healthcare

Technology

News

User Story

Engineering

Culture

The Anyscale Team

Ray Meetup Logo

Update on Ray CVEs CVE-2023-6019, CVE-2023-6020, CVE-2023-6021, CVE-2023-48022, CVE-2023-48023

Retrieval-augmented generation, or RAG applications are among the most popular applications built with LLMs. Embedding endpoints enables developers to use open-source embedding models. Today, we are starting with gte-large, and developers can access it at $0.05/MTokens. We plan to add more models in the future, and users can request newer embedding models by filling out this google form. For more info visit here.

anyscale-endpoints-llama-2

Anyscale Endpoints: Embedding endpoint, Llama-2 70B fine-tuning and improved sign-up experience

blog-anyscale-lambda

Anyscale and Lambda - Addressing AI Scarcity with Engineering

Anyscale Ray Team

In this blog post, we explore Waleed Kadous's insightful talk on "Open Source LLMs: Viable for Production or a Low-Quality Toy?" Read the recap and watch the full session.

top 20 bw-12

Open Source LLMs: Viable for Production or a Low-Quality Toy?

Max Pumperla

Max Pumperla headshot

docu-mentor-main-image

Building an LLM-powered GitHub bot to improve your pull requests

In this blog post, we explore David Xia and Keshi Dai's insightful talk on "How Spotify Built a Robust Ray Platform with a Frictionless Developer Experience" Read the recap and watch the full session.

top 20 bw-11

How Spotify Built a Robust Ray Platform with a Frictionless Developer Experience

Justin Olsson

Waleed Kadous

Llama 2 70b matches legislative interns for summaries, but GPT-4 does better. Studying the GPT-4 summaries helped us improve the prompt. It worked.

Human rated performance of different bill summarization

LLM-based summarization: A case study of human, Llama 2 70b and GPT-4 summarization quality

In this blog post, we explore Ali Naqvi and Sasha Ovsankin's insightful talk on "Inference Graphs at LinkedIn Using Ray-Serve" Read the recap and watch the full session.

top 20 bw-10

Inference Graphs at LinkedIn Using Ray-Serve

Jules S. Damji

Richard Liaw

Richard Liaw headshot

Ray 2.8 features Ray data extensions to read data from external stores, extensions to Ray Dashboard, and AWS neuron cores support, along with enhancements and bug fixes.

Ray 2.8 blog post

Ray 2.8 features Ray Data extensions, AWS Neuron cores support, and Dashboard improvements

In this blog post, we explore Percy Liang's insightful talk on "Advances in Foundation Models — Technology, Society, and Applications" Read the recap and watch the full session.

top 20 bw-09

Advances in Foundation Models — Technology, Society, and Applications

Kyle Huang

Wendi Ding

Liguang Xie

Avnish Narayan

Ricky Xu

Anyscale is releasing LLMPerf for benchmarking LLMs on current LLM offerings. See benchmarking results for Anyscale Endpoints vs Fireworks.ai.

End-to-end time

Reproducible Performance Metrics for LLM inference

In this blog post, we explore Shaowei Su's insightful talk on "Optimizing LLM Training with Airbnb's Next-Gen ML Platform at Airbnb" Read the recap and watch the full session.

top 20 bw-08

Optimizing LLM Training with Airbnb's Next-Gen ML Platform

Goku Mohandas

Philipp Moritz

In this guide, we will learn how to develop and productionize a retrieval augmented generation (RAG) based LLM application, with a focus on scale and evaluation.  

social-rag-based-llm

Building RAG-based LLM Applications for Production

Matt Connor

Robin Singh

Start with the best open source LLMs in minutes. Validate your ideas with a familiar API, fine tune LLMs to get the quality you need, deploy your apps, and repeat.

anyscale-endpoints

Announcing Anyscale Private Endpoints and Anyscale Endpoints Fine-tuning

In this blog post, we explore Edward Oakes & Shreyas Krishnaswamy's insightful talk on "Building Production AI Applications with Ray Serve" Read the recap and watch the full session.

top 20 bw-07

Building Production AI Applications with Ray Serve

In this blog post, we explore Lingyi Liu & Pablo Delgado's insightful talk on "Heterogeneous Training Cluster with Ray at Netflix" Read the recap and watch the full session.

top 20 bw-06

Heterogeneous Training Cluster with Ray at Netflix

In this blog post, we explore Jerry Liu's insightful talk on Practical Data Considerations for Building Production-Ready LLM Applications" Read the recap and watch the full session.

top 20 bw-05

Practical Data Considerations for Building Production-Ready LLM Applications

In this blog post, we explore Harrison Chase insightful talk on "Building Context-Aware Reasoning Applications with LangChain and LangSmith" Read the recap and watch the full session.

top 20 bw-03

Building Context-Aware Reasoning Applications with LangChain and LangSmith

The Ray Summit 2025 CfP is now open for submissions. The premier AI conference will be held November 3-5 in San Francisco, and the due date for CfPs is July 14th, 2025.  

Ray Summit 2025 CFP Open

Ray Summit 2025: Call for Proposals Closes on July 14th, 2025

Google + Anyscale

Google Cloud Integrates Anyscale's RayTurbo with GKE

Simplifying AI Development at Scale: Google Cloud Integrates Anyscale's RayTurbo with GKE

At Ray Summit 2024, we brought together AI experts and practitioners to dive into the future of distributed AI and scalable machine learning.

image3

Ray Summit 2024: Breaking Through the AI Complexity Wall

This section is used to order the "Types" and "Tags" that show up for filters on the Blog Index

Types

Products / Libraries

Blog

The Call for Speakers for Ray Summit 2025 has Been Extended!

Anyscale is the leading AI application platform. With Anyscale, developers can build, run and scale AI applications instantly.

All Posts