Pricing designed for your compute needs
Anyscale in our cloud
Build and deploy your AI apps on our infrastructure.
Anyscale in your cloud
Build and deploy your AI apps in your own infrastucture.
Usage CalculatorEstimated cost: $0.00/month
Dedicated computeCompute running in Anyscale's Cloud provisioned for you.
LLM APIsAnyscale hosted LLM endpoints for popular and fine tuned open source models.
- Foundational model inference
- Fine Tuned model inference
- Fine tuning jobs
Supported models and instance types
LLM API
Foundational Model Inference
| Model | Price ($/M tokens) |
|---|---|
| Mistral-7B-Instruct-v0.1 | $0.15 |
| Llama-2-7b-chat-hf | $0.15 |
| gemma-7b-it | $0.15 |
| NeuralHermes-2.5-Mistral-7B | $0.15 |
| Llama-2-13b-chat-hf | $0.25 |
| Mixtral-8x7B-Instruct-v0.1 | $0.50 |
| Llama-2-70b-chat-hf | $1.00 |
| CodeLlama-70b-Instruct-hf | $1.00 |
| thenlper-gte-large | $0.05 |
| BAAI/bge-large-en-v1.5 | $0.05 |
Fine Tuned Model Inference
| Model | Price ($/M tokens) |
|---|---|
| Llama-2-7b-chat-hf* | $0.25 |
| Llama-2-13b-chat-hf | $0.50 |
| Llama-2-70b-chat-hf | $2.00 |
| mistralai/Mistral-7B-Instruct-v0.1 | $0.25 |
| mistralai/Mixtral-8x7B-Instruct-v0.1 | $1.00 |
Fine Tuning Jobs
| Model | Price ($/M tokens) | Fixed Cost/Run |
|---|---|---|
| Llama-2-13b-chat-hf | $2.00 | $5.00 |
| Llama-2-70b-chat-hf | $4.00 | $5.00 |
| mistralai/Mistral-7B-Instruct-v0.1 | $1.00 | $5.00 |
| mistralai/Mixtral-8x7B-Instruct-v0.1 | $4.00 | $5.00 |
Dedicated compute
ProviderThe provider of the underlying infrastructure.
Select any
ConfigurationThe types of resources and GPUs available on the machine type.
Select any
CPU Only
| Provider | Name | vCPUs | Memory | GPUs | AC/hr |
|---|---|---|---|---|---|
| AWS | m5.2xlarge | 8 | 32 | - | 0.5130 |
| m5.4xlarge | 16 | 64 | - | 1.0395 | |
| m5.8xlarge | 32 | 128 | - | 2.0790 | |
| m5.12xlarge | 48 | 192 | - | 3.1050 | |
| m5.16xlarge | 64 | 256 | - | 4.1445 | |
| m5.24xlarge | 96 | 384 | - | 6.2235 | |
| GCP | n2-standard-8 | 8 | 32 | - | 0.6477 |
| n2-standard-16 | 16 | 64 | - | 1.2953 | |
| n2-standard-32 | 32 | 128 | - | 2.5907 | |
| n2-standard-48 | 48 | 192 | - | 3.8860 | |
| n2-standard-64 | 64 | 256 | - | 5.1813 | |
| n2-standard-96 | 96 | 384 | - | 7.7721 |
NVIDIA A100 80GB
| Provider | Name | vCPUs | Memory | GPUs | AC/hr |
|---|---|---|---|---|---|
| GCP | a2-ultragpu-1g-nvidia-a100-80gb-1 | 12 | 170 | 1 | 6.7892 |
| a2-ultragpu-2g-nvidia-a100-80gb-2 | 24 | 340 | 2 | 13.5783 | |
| a2-ultragpu-4g-nvidia-a100-80gb-4 | 48 | 680 | 4 | 27.1566 | |
| a2-ultragpu-8g-nvidia-a100-80gb-8 | 96 | 1360 | 8 | 54.3132 |
NVIDIA A10G
| Provider | Name | vCPUs | Memory | GPUs | AC/hr |
|---|---|---|---|---|---|
| AWS | g5.2xlarge | 8 | 32 | 1 | 1.6335 |
| g5.4xlarge | 16 | 64 | 1 | 2.1870 | |
| g5.8xlarge | 32 | 128 | 1 | 3.3075 | |
| g5.12xlarge | 48 | 192 | 4 | 7.6545 | |
| g5.48xlarge | 192 | 768 | 8 | 21.9915 |
NVIDIA H100 80GB
| Provider | Name | vCPUs | Memory | GPUs | AC/hr |
|---|---|---|---|---|---|
| GCP | a3-highgpu-8g-nvidia-h100-80gb-8 | 208 | 1830 | 8 | 118.6110 |
NVIDIA L4
| Provider | Name | vCPUs | Memory | GPUs | AC/hr |
|---|---|---|---|---|---|
| GCP | g2-standard-8-nvidia-l4-1 | 8 | 32 | 1 | 2.0615 |
| g2-standard-16-nvidia-l4-1 | 16 | 64 | 1 | 2.5105 | |
| g2-standard-32-nvidia-l4-1 | 32 | 128 | 1 | 3.4083 | |
| g2-standard-48-nvidia-l4-4 | 48 | 192 | 4 | 9.1437 | |
| g2-standard-96-nvidia-l4-8 | 96 | 384 | 8 | 18.0820 |
NVIDIA T4
| Provider | Name | vCPUs | Memory | GPUs | AC/hr |
|---|---|---|---|---|---|
| AWS | g4dn.2xlarge | 8 | 32 | 1 | 1.0125 |
| g4dn.4xlarge | 16 | 64 | 1 | 1.6200 | |
| g4dn.8xlarge | 32 | 128 | 1 | 2.9430 | |
| g4dn.16xlarge | 64 | 256 | 1 | 5.8725 | |
| g4dn.12xlarge | 48 | 192 | 4 | 5.2785 | |
| GCP | n1-standard-8-nvidia-t4-16gb-1 | 8 | 30 | 1 | 1.0976 |
| n1-standard-16-nvidia-t4-16gb-1 | 16 | 60 | 1 | 1.7226 | |
| n1-standard-32-nvidia-t4-16gb-1 | 32 | 120 | 1 | 2.9727 | |
| n1-highmem-32-nvidia-t4-16gb-1 | 32 | 208 | 1 | 3.5857 | |
| n1-standard-32-nvidia-t4-16gb-4 | 32 | 120 | 4 | 4.3902 |
NVIDIA Tesla V100 16GB
| Provider | Name | vCPUs | Memory | GPUs | AC/hr |
|---|---|---|---|---|---|
| AWS | p3.2xlarge | 8 | 61 | 1 | 4.1310 |
| p3.8xlarge | 32 | 244 | 4 | 16.5240 | |
| p3.16xlarge | 64 | 488 | 8 | 33.0480 | |
| GCP | n1-highmem-8-nvidia-v100-16gb-1 | 8 | 52 | 1 | 4.1263 |
| n1-highmem-32-nvidia-v100-16gb-4 | 32 | 208 | 4 | 16.5052 | |
| n1-highmem-64-nvidia-v100-16gb-8 | 64 | 416 | 8 | 33.0105 |