Pushing with Soft Robotic Arms via Deep Reinforcement Learning

Soft robots can adaptively interact with unstructured environments. However, nonlinear soft material properties challenge modeling and control. Learning-based controllers that leverage efficient mechanical models are promising for solving complex interaction tasks. This article develops a closed-loop pose/force controller for a dexterous soft manipulator enabling dynamic pushing tasks using deep reinforcement learning. Force tests investigate the mechanical properties of a soft robot module, resulting in orthogonal forces of 9-13$9 - 13$ N. Then, the policy is trained in simulation leveraging a dynamic Cosserat rod model of the soft robot. Domain randomization mitigate the sim-to-real gap while careful reward engineering induced pose and force control even without explicit force inputs. Despite the approximate simulation, the sim-to-real transfer achieved an average reaching distance of 34 +/- 14$34 \pm 14$ mm (8.1%L +/- 3.4%L$ L \pm L$), an average orientation error of 0.40 +/- 0.29$0.40 \pm 0.29$ rad (23 degrees +/- 17 degrees$\left(23\right)<^>{\circ} \pm \left(17\right)<^>{\circ}$) and applied pushing forces up to 3$3$ N. Such performance is reasonable for the intended assistive tasks of the manipulator. The experiments uncovered that the soft robot interacting with the environment exhibited torsional and counter-balancing movements. Although not explicitly enforced, they emerged from the mechanical intelligence of the manipulator. The results demonstrate the potential of soft robotic manipulation via reinforcement learning. Soft robots can adaptively interact with unstructured environments. Reinforcement learning-based controllers that leverage continuum mechanics are promising for solving complex interaction tasks. A closed-loop pose and force control policy for a dexterous soft manipulator enabling dynamic pushing tasks is trained in simulation. The sim-to-real transfer of the control policy showcases the potential of soft robotic manipulation via reinforcement learning.image (c) 2024 WILEY-VCH GmbH