Physics-informed Deep Reinforcement Learning-based Adaptive Generator Out-of-step Protection for Power Systems

This article presents a deep reinforcement learning-based control framework for adaptive generator protection in wide-area power systems. Out-of-step (OOS) generator tripping is an effective emergency control measure for mitigating system-wide black-out risks following any severe disturbance. Traditional protection schemes utilize rule-based mechanisms that fail to adapt to changing operating conditions. With the recent advances in deep reinforcement learning (DRL), the primary objective of our proposed methodology is to learn a DRL agent that: (a) can timely identify and isolate the affected generators after any potential disturbance and thereby maintain the system stability and (b) can adapt in unseen scenarios. But learning to identify an optimal set of generators for bulk power systems under various operating conditions is prohibitive due to: (a) the combinatorial nature of the problem, (b) the exponential increase of action space, and (c) the ultra-selectivity of the generator trip-action. To address these key challenges, we utilized the concept of action masks integrating system physics in the learning process, thereby blocking unnecessary actions in the exploration phase of the policy training, where the action masks are learned in conjunction with the DRL policy. In the policy part, we utilized a derivative-free parallel augmented random search (PARS)-based DRL algorithm, which is fast and highly scalable. Finally, we validated the proposed methodology with IEEE 300-bus systems.