Controlling fracture propagation using deep reinforcement learning

Mechanical discontinuity embedded in a material plays an essential role in determining the bulk mechanical, physical, and chemical properties. The ability to control mechanical discontinuity is relevant for industries dependent on natural, synthetic and composite materials, e.g. construction, aerospace, oil and gas, ceramics, metal, and geothermal industries, to name a few. The paper is a proof-of-concept development of a reinforcement learning framework to control the propagation of mechanical discontinuity. The reinforcement learning framework is coupled with an OpenAI-Gym-based environment that uses the mechanistic equation governing the propagation of mechanical discontinuity. Learning agent does not explicitly know about the underlying physics of propagation of discontinuity; nonetheless, the learning agent can infer the control strategy by continuously interacting the environment. The design of Markov decision process, which includes state, action and reward, is crucial for robust control. The deep deterministic policy gradient (DDPG) algorithm is implemented for learning continuous actions. It is also observed that the training efficiency is strongly determined by the formulation of reward function. The reward function that forces the learning agent to stay on the shortest linear path between crack tip and goal point performs much better than the reward function that aims to reach closest to the goal point in minimum number of steps. After close to 500 training episodes, the reinforcement learning framework successfully controlled the propagation of discontinuity in a material despite the complexity of the propagation pathway determined by multiple goal points.