Adaptive Learning Rate Residual Network Based on Physics-Informed for Solving Partial Differential Equations

Recently, Physics-informed neural networks (PINNs) have been widely applied to solving various types of partial differential equations (PDEs) such as Poisson equation, Klein-Gordon equation, and diffusion equation. However, it is difficult to obtain higher accurate solutions, especially at the boundary due to the gradient imbalance of different loss terms for the PINN model. In this work, an adaptive learning rate residual network algorithm based on physics-informed (adaptive-PIRN) is proposed to overcome this limitation of the PINN model. In the adaptive-PIRN model, an adaptive learning rate technique is introduced to adaptively configure appropriate weights to the residual loss of the governing equation and the loss of initial/boundary conditions (I/BCs) by utilizing gradient statistics, which can alleviate gradient imbalance of different loss terms in PINN. Besides, based on the idea of ResNet, the "short connection" technique is used in adaptive-PIRN model, which can ensure that the original information is identically mapped. This structure has stronger expressive capabilities than fully connected neural networks and can avoid gradient disappearance. Finally, three different types of PDE are conducted to demonstrate predictive accuracy of our model. In addition, it is clearly observed from the results that the adaptive-PIRN can balance the gradient of loss items to a great extent, which improves the effectiveness of this network.