LinkBackground: A Missing Layer in Ray’s Security Story

Until now, open-source Ray has not enforced any authentication between a user’s browser and the Ray Dashboard or API endpoints. While users could protect access through network isolation or external proxies, this approach relied on infrastructure controls rather than Ray itself providing built-in safeguards.

This meant that anyone with network access to the Ray Dashboard could interact with the cluster, which was sufficient for trusted environments but not ideal for shared or cloud-hosted deployments.

Anyscale already leverages this type of robust authentication to secure its services, but Ray itself lacked a native, comparable mechanism. To close that gap, the Ray team built a token-based authentication framework directly into Ray, adding a similar layer of protection. This lightweight but powerful mechanism adds a native layer of protection between clients and Ray services, without requiring external identity systems or complex setup.

The goal is simple:

• Give all Ray users a more secure, out-of-the-box experience.

• Maintain compatibility with existing network perimeter defenses.

• Establish a consistent authentication story for the Dashboard and APIs.

LinkHow Token-Based Authentication Works in Ray

Every Ray server component now enforces authentication through a token validation middleware. This ensures that only requests with the correct token are processed by Ray’s internal and external services.

LinkUnified Token Model

A single token secures communication across the cluster:

• Configured once when the cluster starts.

• Shared among all Ray components.

• Verified on every incoming request.

Requests without a valid token are immediately dropped with appropriate error logging and warning.

LinkCoverage Across Ray Components

The middleware covers:

• Public APIs like the Ray Dashboard and the GCS autoscaler service.

• Private APIs such as GCS, Raylets, agents, and other internal control-plane services.

This unified enforcement provides defense-in-depth throughout the Ray architecture.

LinkClient Behavior

All Ray RPC client implementations now send tokens automatically:

• Clients include the token in request headers.

• Internal requests (e.g., core worker gRPC clients) automatically propagate the token downstream.

This keeps authentication transparent for developers while ensuring end-to-end protection.

LinkInternal Communication Exemptions

Some local communications (like those over Unix domain sockets, e.g., Plasma store clients) remain exempt, since they occur in trusted contexts within the same node.

LinkUsing Tokens in Practice

When RAY_AUTH_MODE=token is enabled, Ray RPC clients and servers follow a consistent token lookup order:

1. RAY_AUTH_TOKEN environment variable.

2. RAY_AUTH_TOKEN_PATH environment variable.

3. Default path: ~/.ray/auth_token.

If these aren’t found, behavior depends on the command being used.

LinkLocal Development with ray.init()

If no token exists:

• Ray generates one automatically and writes it to ~/.ray/auth_token.

• A log message informs the user.

• The token is used for all server and client components.

• Subsequent runs reuse the same token.

• Ray CLI commands pick it up automatically.

LinkLocal Development with ray start

If no token is found:

• Ray raises an error and instructs the user to generate one:

1ray get-auth-token --generate

Once generated, the token is used automatically for all processes and reused on future runs.

LinkRemote Cluster with Cluster Launcher

When using ray up config.yaml:

• file_mounts config option mounts the token file in every ray node, and head and worker start commands sets the required env variables.

• Users can also consider hardcoding the tokens and setting RAY_AUTH_TOKEN=<token> but this is not recommended.

• Users can submit jobs to this ray cluster using ray job submit

• When using docker image, users can set env variables by passing them as run_options

• The env variables can be set in initialization_commands but it needs to be added to .bashrc so that the env vars are set across different shell sessions (plain export commands will not work as ray start is called from a different shell, this is the recommended approach for passing env vars).

LinkRemote Cluster with KubeRay

KubeRay automates token management entirely:

• The controller generates a token per RayCluster or RayJob.

• It stores the token as a Kubernetes Secret and mounts it into Ray pods.

• Users can export the secret locally to access the dashboard or CLI securely.

LinkCustom Remote Clusters

Advanced users can manually distribute tokens across nodes:

• ray start will error if no token is found.

• Users can generate one via ray gen-auth-token and copy it to each node under ~/.ray/auth_token.

Token-based auth can be disabled, but only when users understand the security implications.

LinkImpact and What’s Next

Token-based authentication marks a major step forward in Ray’s security posture. For the first time, the Ray Dashboard and internal APIs are protected by a built-in authentication layer, adding another protective layer of security rather than relying solely on external controls.

This change strengthens security for all users, from local development to production clusters and makes secure-by-default the new norm.

Initially, token authentication will ship as an opt-out to ease migration. In future releases, it will become the default for all Ray clusters.

LinkIn Summary

Ray’s new token-based authentication system is a foundational security enhancement. It is easy to adopt, transparent for users, and sets the stage for future innovations in distributed system security.

Secure-by-default is coming to Ray, and this is the first big step.