Sailing to victory with reinforcement learning

Earlier this year, the Emirates Team New Zealand won the America’s Cup for the second time in four years, defending its title as sailing champ against the Luna Rossa Prada Pirelli syndicate from Italy. But unlike their win in 2017, this time they won with the help of a new crew member — an AI agent.

The America’s Cup, the oldest international competition of any sort, is a blend of history and modern technology that pits fine-tuned vessels coupled with the best sailors against each other as they take flight across the seas. With every participant at peak performance, teams are often only as good as their tools. How often they design, evaluate, and iterate can be the difference between victory and defeat. 

Fortunately for Team New Zealand, kiwis can fly — thanks to AI. They partnered with McKinsey’s QuantumBlack, who leveraged Ray and RLlib along with other best-of-breed solutions to train a virtual crew member. Where once there were logistical challenges due to sailors’ scheduled practices, travel, and competitions, now simulations could be run around the clock. RLlib’s multi-agent functionality also meant that countless scenarios could be devised, trained on, and then converged for reliable, consistent results across a myriad of potential real-world situations — sometimes revealing solutions even the Olympic sailors hadn’t thought of. After just eight weeks of training, the AI agent was on par with if not better than its Olympic counterparts. 

“When you start, the AI agent knows nothing and learns by trial and error using countless variables — wind speed, direction, adjustments to the 14 different sail and boat controls — and is refined again and again,” says Nic Hohn, chief data scientist at QuantumBlack. “Since the bot keeps experimenting, if you coach it to learn in the right way, it compresses into hours what would take a human years to understand.” For instance, designers were quickly able to iterate through sailing hydrofoil designs in hours instead of days.

Although they’re not the first team to use a simulator, what Emirates Team New Zealand is doing is differentiated in large part due to the flexibility and scale they were able to achieve with the industry-standard reinforcement learning Python framework built on Ray. It’s not just another reinforcement learning experiment. Sailing is a problem with a game-tree complexity of nearly 2,900 — yet the team was able to successfully speed up the design process by a factor of 10.

Hear from Nic himself how he and his team applied Ray and its libraries to the task:

If you’re ready to get your hands dirty, start out with an overview of Ray and RLlib or check out Reinforcement Learning with Ray’s RLlib on Anyscale Academy.