Natural heuristic dynamic programming for dynamic systems control

Heuristic Dynamic Programming (HDP) is the simplest kind of Adaptive Critic (Werbos, 1992). It can be used to maximize or minimize any utility function, such as total energy or trajectory error, of a system over time in a noisy environment. This article proposes a new version of HDP, called NHDP (Natural Heuristic Dynamic Programming). This new version incorporates basic HDP algorithm with the following features:(i) use of Trajectory Pattern Method to guarantee smoothness of trajectory and control signals; (ii) use multiple critic networks to localize effect of each parameter mimicking the natural biological model of human brain; and (iii) allow the controller to learn from slow to fast motion, analogous to the natural learning behavior of humans. A simple dynamic system is used to illustrate NHDP. Copyright (C) 1998 IFAC.