Data-Driven MPC for Nonlinear Systems with Reinforcement Learning

Inspired by Willems and the co-authors' idea that continuously excited system trajectories can be used to represent the input-output behavior of discrete-time linear time-invariant (DT LTI) systems. We extend this idea to nonlinear systems. In this paper, we propose a data-driven model predictive control (MPC) scheme with reinforcement learning (RL) for unknown nonlinear systems. We utilize the input-output data of the system to form Hankel matrices to represent the system model implicitly. The accuracy of the prediction is improved by updating the data online. Another core idea of this scheme is to combine the standard MPC with RL to approximate the terminal cost function by TD-learning to ensure the closed-loop stability of the system. Simulation experiments on the cart-damper-spring system were used to demonstrate the feasibility of the proposed algorithm.