Reinforcement learning-based neural control for discrete-time nonlinear systems via deterministic learning

Ensuring the stability of a closed-loop system and the exponential convergence of neural weights to their optimal values with rigorous analysis are challenging and significant problems in reinforcement learning (RL)-based control tasks, since precisely converged weights imply the acquisition of some accurate knowledge of the controlled object, which enables us to design a knowledge-based controller, observer, or planner. To address these issues, this paper combines recent advances in deterministic learning theory with a classical RL method, direct heuristic dynamic programming (HDP), to develop a novel direct HDP-based neural controller, and accurate modelling of unknown nonlinear dynamics in the control process can be achieved. The implementation involves the design of an actor-critic structure that transforms the state tracking and neural weight estimation problems into the stability problem of a class of linear time-varying systems. The exponential stability of the error systems is rigorously proved using Lyapunov's direct method. During the learning process, RL is used to design an NN structure parameter adaptation scheme to achieve better tracking performance. Finally, the validity of the proposed scheme is verified by a series of simulation results.