Coordination of hydraulic manipulators by reinforcement learning

In this paper, a reinforcement learning method is applied to coordinate a pair of horizontal hydraulic actuators engaged in the cooperative positioning of an object. The goal is to enable the actuators to discover how to intelligently select control actions that tend to reduce the interaction forces directed along the axis of motion, while maintaining the desired trajectory. First, a detailed and realistic dynamic model of the entire system is derived. A multi-layer reinforcement learning neural network control architecture is designed next to regulate the interaction force during positioning. To regulate the interaction force, the neural network measures the interaction force and proposes a modification to the a priori prescribed formation constrained position trajectory. Each actuator system is outfitted with such a neural controller so that a decentralized reinforcement learning control system results. Simulations demonstrate the efficacy of the approach towards reducing the interaction forces and minimizing the associated object internal force in a single degree of freedom.