Real-Time Optimistic Planning with Action Sequences

Optimistic planning (OP) is a promising approach for receding-horizon optimal control of general nonlinear systems. This generality comes however at large computational costs, which so far have prevented the application of OP to the control of nonlinear physical systems in real-time. We therefore introduce an extension of OP to real-time control, which applies open-loop sequences of actions in parallel with finding the next sequence from the predicted state at the end of the current sequence. Exploiting OP guarantees, we provide conditions under which the algorithm is provably feasible in real-time, and we analyze its performance. We report successful real-time experiments for the swingup of an inverted pendulum, as well as simulation results for an acrobot, where the impact of model errors is studied.