Safe Reinforcement Learning in Simulated Environment of Self-Driving Laboratory

Today we see tremendous potential in applying artificial intelligence (AI), deep reinforcement learning, and agent-based simulation to complex real-world problems. AI helps people support and automate decision-making penetrating almost all daily life aspects and research areas. One of the reasons for this potential is that AI helps us solve problems at a lower cost of resources and time. Materials research acceleration often relies upon AI using and automation of laboratory experiments, bringing significant fruitful results and advances. Self-driving laboratories include closed-loop chemistry experimentation and assist in designing new functional nanomaterials and optimizing their known parameters with AI and machine learning approaches. Due to the possibility of involving in the nanomaterials design process and some hazardous components, routine experimentation under chemists' continuous monitoring is usually required. Shifting to new intelligent technologies in selfdriving laboratories with automated closed-loop experimentation requires excluding risks and accidents because of improper AI applications. This paper discusses safe deep reinforcement learning and its application in a simulated environment in self-driving laboratories experimenting with new functional materials. We proposed an approach to solving the problem of safe reinforcement learning by learning the intelligent agent to find a hidden reward and implemented that approach by constructing and using the heatmap from observation of the hidden reward neighborhood.