Data-Based Optimal Tracking Control of Nonaffine Nonlinear Discrete-Time Systems

The optimal tracking control problem of nonaffine nonlinear discrete-time systems is considered in this paper. The problem relies on the solution of the so-called tracking Hamilton-Jacobi-Bellman equation, which is extremely difficult to be solved even for simple systems. To overcome this difficulty, the data-based Q-learning algorithm is proposed by learning the optimal tracking control policy from data of the practical system. For its implementation purpose, the critic-only neural network structure is developed, where only critic neural network is required to estimate the Q-function and the least-square scheme is employed to update the weight of neural network.