The Infinite Laptop

LinkProgramming the cloud as easily as your laptop

Think about the last Python program you wrote. Now, imagine you have to make it run 100x faster or process 1000x more data.

Now, imagine you have never built a distributed application or managed a multi-node cluster before.

Then Ray and the Anyscale platform are for you!

LinkThe future of computing is distributed

The future of computing will be distributed. This is the result of two powerful trends. First, the computational requirements of AI applications are growing at a rate that vastly outpaces hardware advances (including accelerators like GPUs and TPUs). Second, AI is penetrating virtually every industry and problem domain.

As a consequence, virtually every industry and problem domain will require distributed computing, making the rise of distributed applications inevitable. These applications are rapidly becoming the norm rather than the exception.

LinkNo good solutions for distributed applications

Unfortunately, building distributed applications is hard. Building them requires developers to reason about how to divide the work between different machines, how to efficiently exchange data between processes, and how to handle machines crashing or being added to the cluster. Building such applications requires deep distributed systems expertise, which few developers have. Think about how many developers could design and build distributed frameworks like MapReduce, Spark, or Kafka!

If distributed applications are to become mainstream, we need to simplify the development of distributed applications and bring them within reach of every developer.

In response to this need, cloud providers and the open source community have released serverless platforms including AWS Lambda, Azure Functions, Google Cloud Functions, and OpenWhisk. These platforms abstract away clusters by allowing users to run only individual functions. Under the hood these platforms scale dynamically and transparently with the number of function invocations, and they charge users only for the resources their functions use. While serverless platforms make distributed programming much easier, they support a narrow set of applications. Several recent papers have extensively documented the limitations of existing severless solutions. For example, most of them are not portable across clouds. They do not support stateful computation, which makes them a poor fit for modern applications involving machine learning or microservices. They do not enable functions to communicate directly with each other, which is prohibitive for communication intensive applications. The current offerings do not support GPUs and TPUs, and some limit computation to 15 minutes.

As such, existing serverless offerings fall short of providing a general platform for distributed applications.

LinkThe “infinite laptop”

Our goal at Anyscale is to enable every developer to build distributed applications and to make developing distributed applications as easy as developing on your laptop.

While your laptop has a familiar and powerful set of development tools, it lacks a few important features. First, it doesn’t have enough resources (CPUs, memory, etc) to run modern workloads. Second, it doesn’t have specialized processors like TPUs or fast GPUs, to speed up large computations. Finally, it isn’t always available. For example, you cannot run a production service on your laptop because you may need to close your laptop.

In a nutshell, as we hear over and over from our users, developers want the development experience of their laptops with the power of the cloud. They want an “infinite laptop.”

We are working to realize the vision of an “infinite laptop” that you can program using Ray. Today, as a first step toward this goal, we are very excited to invite developers to sign up for the private beta of the Anyscale platform.

The Anyscale platform has three desirable properties. First, it provides the experience of a laptop with the power of the cloud. Second, it provides a serverless experience without the limitations of today’s serverless offerings, and finally, it is multi-cloud native.

Laptop experience with the power of the cloud. To provide the laptop experience while running your program on the cloud, Anyscale can automatically sync your code and data from your laptop to the cloud. This allows you to edit programs on your laptop using your favorite editor, execute code on the cluster without any context switching, and check the outputs on your laptop.

Furthermore, Anyscale has built-in dashboards that provide detailed resource utilization and debugging information at your fingertips, just like a laptop.

Serverless experience without serverless limitations. Like existing serverless platforms, Anyscale provides a serverless experience by abstracting away servers and clusters and allowing developers to focus solely on their application logic. Under the hood it will allocate the right instance types and automatically scale to match the application demands.

But Anyscale goes beyond today’s serverless platforms by eliminating most of their limitations. First, the Anyscale platform is based on Ray, which is open source and cloud agnostic. Second, through its actor API, Ray supports stateful applications like serving and training. Third, Ray enables direct communication between actors and tasks. Together, these properties allow developers to build applications that are as performant as applications built by experts from scratch using lower-level RPC frameworks. Lastly, it supports hardware accelerators like GPUs and TPUs, and it has no runtime limits.

Anyscale is cloud native. Anyscale enables you to select a cloud when starting an application and to use multiple clouds at the same time. For example, you can train your model on GPUs and TPUs in one cloud and serve on CPUs in another cloud to save costs and shorten the product development cycle.

LinkSummary

In summary, we believe that distributed applications will become the norm rather than the exception. However, building such applications today is extremely hard as it requires deep distributed systems expertise, which few developers possess.

Our goal is to make developing distributed applications as easy as developing on your laptop. The Anyscale platform aims to achieve this by providing you with the illusion of an “infinite laptop” that you can program with Ray.