Optimal control of nonlinear systems with dynamic programming

In this paper, a generalized technique for solving a class of nonlinear optimal control problems is proposed. The optimization problem is formulated based on the cost-to-go functional approach and the optimal solution can be obtained by Bellman's technique. Specifically, a continuous nonlinear system is first discretized and a set of equality constraints can be obtained from the discretization. We show that, under a certain condition, the optimal solution of a problem in this class can be approximated by a solution of the set of equality constraints within any precision and the system is guaranteed to be stable under a control signal obtained from the solution. An iterative approach is then applied to numerically solve the set of equality constraints. The technique is tested on a nonlinear control problem from the class and simulation results show that the approach is not only effective but also leads to a fast convergence and accurate optimal solution.