Optimal Tracking Control of Affine Nonlinear Discrete-time Systems with Unknown Internal Dynamics

In this paper, direct dynamic programming techniques are utilized to solve the Hamilton Jacobi-Bellman equation forward-in-time for the optimal tracking control of general affine nonlinear discrete-time systems using online approximators (OLA's). The proposed approach, referred as adaptive dynamic programming (ADP), is utilized to solve the infinite horizon optimal tracking control of affine nonlinear discrete-time systems in the presence of unknown internal dynamics and a known control coefficient matrix. The design is implemented using OLA's to realize the optimal feedback control signal and the associated cost function. The feedforward portion of the control input is derived and approximated using an additional OLA for steady state conditions. Novel tuning laws for the OLA's are derived, and all parameters are tuned online. Lyapunov techniques are used to show that all signals are uniformly ultimately bounded (UUB) and that the approximated control signal approaches the optimal control input with small bounded error. In the ideal case when there are no approximation errors, the approximated control converges to the optimal value asymptotically. Simulation results are included to show the effectiveness of the approach.