Optimal Trajectory Output Tracking Control with a Q-learning Algorithm

In this paper a novel Q-learning algorithm is proposed to solve the Linear Quadratic Output Tracking (LQOT) control problem of a linear time invariant system with completely unknown system and reference dynamics. We first define an action-dependent value function for the LQOT problem after we augment the system and the reference states and pick appropriately the user-defined matrices in the performance index of the augmented state. An integral reinforcement learning approach is used to develop a reinforcement learning structure to estimate the parameters of the Q-function online while also guaranteeing closed-loop stability, trajectory tracking and convergence to the optimal tracking solution. A simulation result of an unknown spring-mass-damper linear system is presented to show the efficacy of the proposed approach.