How SewerAI Saved 75% in Cloud Operating Costs

TL;DR

SewerAI is a leader in municipal infrastructure maintenance, serving 123 million people in the 12,000 miles of territory under its management. SewerAI partnered with Anyscale to reduce its cloud operating costs by 75%, achieve 3x faster iteration for machine learning (ML) application development, and free up one engineer position to focus on high-value projects. 

Overview

SewerAI uses AI/ML to solve urgent infrastructure challenges. 

Sewer infrastructure is critical to the health and safety of every community. But the EPA estimates that half of that infrastructure in the U.S. is failing, leaving people within those communities vulnerable to public health and environmental risks. 

SewerAI is transforming the way utilities maintain and rehab their underground infrastructure to mitigate these risks. Historically, human inspectors manually assessed sewer conditions using camera trucks. But this often results in errors due to environmental factors and backlogs of weeks or months for inspection reviews. Through the use of artificial intelligence (AI), SewerAI’s AutoCode tools automatically detect conditions in pipeline inspection data from CCTV crawlers, digital slide scanners, drones, GoPros, and jetter cameras. SewerAI uses AI to accelerate the inspection process while making the assessment of sewer conditions more accurate, efficient, and objective. The result is that customers can manage their infrastructure more effectively to prevent and remediate sewage failures — all while lowering their operational costs.

The Challenge

Processing huge volumes of data led to bottlenecks, complexity, and delays.

Using deep learning to process thousands of hours of video footage was critical to the business but came with significant AI and data infrastructure challenges — particularly scaling batch inference in a cost-effective way across hundreds of GPUs. 

At the same time, SewerAI’s approach to training object detection models created a bottleneck in the ingestion process, with expensive and scarce GPUs often achieving only 25% utilization.

Finally, the development process was hindered by the need to onboard engineers with specialized skills, as well as a lengthy testing process.

Together, these challenges reduced SewerAI’s ability to respond quickly and cost-efficiently to the needs of its customers and their communities — and to grow the business.

The Solution

Anyscale Ray accelerates development productivity and lowers the operational cost of AI infrastructure.

SewerAI turned to Anyscale, creators of the Ray open-source distributed computing framework, for help. The SewerAI team was immediately impressed with the level of deployment flexibility that Ray offered. SewerAI’s workloads are large and spikey, and its team loved the fact that Anyscale offered a fast path to deployment without having to develop and maintain AI infrastructure. This meant the business could scale its mission-critical AI development more smoothly, while lowering its cloud operational cost.

The team at SewerAI takes advantage of Anyscale’s Workspaces feature, which enables ML practitioners to build distributed Ray applications and easily advance from research to development to production — all within a single platform. Anyscale Workspaces makes it easier for SewerAI to ramp up new teammates on the application while giving them a centralized environment that integrates with all of their favorite AI/ML tools. The team at SewerAI started prototyping with Anyscale and had fully proven the solution in just over two months — a dramatic time-to-value savings that helped improve the company’s agility.

“Anyscale was exactly what we needed to scale our ML workloads without investing time and money in building our own infrastructure,” says Noah Rubinstein, AI Architect at SewerAI. “It is a platform that we can mold to fit our specific needs, rather than trying to adapt to more opinionated solutions that don’t fit our exact case. As an AI-driven business, that’s a game-changer for us.'"

The Impact

Using Anyscale, SewerAI achieved remarkable gains in the speed at which it can process vital inspection data, and the cost and resources needed to do so:

• Faster processing: 3x faster batch inference, reducing processing time from one hour to 20 minutes

• Resource efficiency: 50% reduction in required machines, as well as 95% GPU utilization — increased from 25% previously

• Accelerated testing: Nearly instantaneous code testing vs. 10 minutes

• Cost savings: 75%+ reduction in total cost of ownership (TCO) compared to AWS Batch

Furthermore, Anyscale's Workspaces feature streamlined development, improved issue diagnosis accuracy, and allowed municipalities to gain actionable insights from unreviewed inspections — all exceeding human performance.

AI is at the very heart of SewerAI’s mission, so it makes sense for the company to maximize its AI development scale, efficiency, and speed. SewerAI now plans to migrate additional workloads to the Anyscale Platform and take advantage of multi-cloud support, making Anyscale its central platform for AI development and deployment.