Data-Driven Learning and Control with Multiple Critic Networks

In this paper, we extend our previous work of a three-network adaptive dynamic programming design [1] to be a multiple critic networks design for online learning and control. The key idea of this approach is to develop a hierarchical internal goal representation to facilitate the online learning with detailed and informative internal value signal representations. We present our learning architecture in detail, and also demonstrate its performance on the popular cart-pole balancing benchmark. Simulation results demonstrate the effectiveness of our approach. We also present discussions of further research directions along this topic.