Siemens brings reinforcement learning to energy, transportation and logistics

Siemens Technology has a long history of using and creating some of the most advanced technologies on the planet, whether in manufacturing, power, or transportation. This has been thanks to smart minds taking advantage of smart tools like neural networks and reinforcement learning (RL) over the last two decades. 

Starting with a smart washing machine in 2003, the application of RL and other machine learning frameworks and technologies has vastly evolved, becoming an integral part of new Siemens products and services. By 2017, Siemens launched its proprietary Gas Turbine Auto Tuner. That same year Siemens launched software to control wind turbines through interpretable RL and also started using uncertainty-aware RL using Bayesian neural networks and Deep Gaussian Processes. By 2022, it was using multi-agent RL to reduce delays and energy consumption in German metro trains. 

To reach these goals Siemens had to bridge the gap between research and industry use of RL. While the research domain treads uncharted territory, is fast moving, and has no limits in terms of resources, the industry domain has existing solutions and domain experts that ensure security and trust amid rigid time and budget constraints. And they did this effectively through experiments, like the gas turbine auto tuner, that proved successful. 

One thing to note in this process is that simulations and digital twins were crucial in ensuring agents were properly trained and models born out without a negative impact on the real world. Offline RL was also Siemens’ preferred approach as real systems would be often prohibited. Additionally, transparency in how the training works and including humans in the loop at least as overseers has been critical in industrial settings. 

To learn more about Siemens Technology’s journey into the world of RL, watch Learning Systems lead Volkmar Sterzing and senior key expert for AI-based control optimization Dr. Marc Weber’s Production RL Summit talk. They share the challenges and opportunities of converting groundbreaking technologies into profitable use cases, scaling and generalizing RL, and developing robust and trustworthy RL solutions.