News

User Story

Engineering

Culture

Waleed Kadous

Kyle Huang

Wendi Ding

Liguang Xie

Avnish Narayan

Ricky Xu

Anyscale is releasing LLMPerf for benchmarking LLMs on current LLM offerings. See benchmarking results for Anyscale Endpoints vs Fireworks.ai.

End-to-end time

Reproducible Performance Metrics for LLM inference

Yi Cheng

Cade Daniel

Chen Shen

In this post, we discuss the importance of speed when loading large language models and what techniques we employed to make it 20x faster. In particular, we use the Llama 2 series of models. We share how you can reduce latency and costs using the Anyscale platform.

loading llms 20x faster

Loading Llama-2 70b 20x faster with Anyscale Endpoints

Amog Kamsetty

Hao Chen

ByteDance, the company behind Tiktok, leverages multi-modal models to enable many applications, such as text-based image retrieval or object detection.

bytedance-actor-pool

How ByteDance Scales Offline Inference with Multi-Modal LLMs

How ByteDance Scales Offline Inference with multi-modal LLMs to 200 TB Data

stable-diffusion-processing-2-billion

Processing 2 Billion Images for Stable Diffusion Model Training - Definitive Guides with Ray Series

In this blog post, we introduce an advanced pre-training solution for Stable Diffusion v2 models, leveraging the power of the Ray and Anyscale Platform to enhance scalability and cost efficiency.  

Scalable and Cost-Efficient Stable Diffusion Pre-training with Ray

Reducing the Cost of Pre-training Stable Diffusion by 3.7x with Anyscale

Ray Summit 2024, the annual Ray community conference, is back in San Francisco, CA this September 30th through October 2nd. It is a two-day event with a third day dedicated to training.

Call for Proposals is open Blog Post

Ray Summit 2024 Call for Proposals is now open

Intel and Anyscale have integrated Ray with Gaudi3. Gaudi3 is AI-optimized for efficient training and other machine learning workloads.

Anyscale - Intel

Accelerating AI: Harnessing Intel(R) Gaudi(R) 3 with Ray 2.10

Anyscale is teaming with NVIDIA to combine the developer productivity of Ray Serve and RayLLM with the cutting-edge optimizations from NVIDIA Triton Inference Server software and the NVIDIA TensorRT-LLM library.

NVIDIA-Anyscale Logo 1280x680.jpg

Low-latency Generative AI Model Serving with Ray, NVIDIA Triton Inference Server, and NVIDIA TensorRT-LLM

This section is used to order the "Types" and "Tags" that show up for filters on the Blog Index

Types

Products / Libraries

Blog

Anyscale Preview is now available! Login today to get free $50 compute credit 🚀

Anyscale is the leading AI application platform. With Anyscale, developers can build, run and scale AI applications instantly.

Posts by Liguang Xie

No posts found.