Safe Reinforcement Learning: A Survey

Reinforcement learning (RL) has proved a prominent success in the game of Go, video games, navigation, recommendation systems and other fields. However, a large number of reinforcement learning algorithms cannot be directly transplanted to real physical environment. This is because in the simulation scenario, the agent is able to interact with the environment in a trial-and-error manner to learn the optimal policy. Considering the safety of systems, many real-world applications require the limitation of random exploration behavior of agents. Hence, safety has become an essential factor for reinforcement learning from simulation to reality. In recent years, many researches have been devoted to develope safe reinforcement learning (SRL) algorithms that satisfy safety constraints while ensuring system performance. This paper presents a comprehensive survey of existing SRL algorithms, which are divided into three categories: Modification of learning process, modification of learning objective, and offline reinforcement learning. Furthermore, five experimental platforms are introduced, including Safety Gym, safe-control-gym, SafeRL-Kit, D4RL, and NeoRL. Lastly, the applications of SRL in the fields of autonomous driving, robot control, industrial process control, power system optimization, and healthcare are summarized, and the conclusion and perspective are briefly drawn. © 2023 Science Press. All rights reserved.