Autonomous Blood Suction for Robot-Assisted Surgery: A Sim-to-Real Reinforcement Learning Approach

Recent applications of deep reinforcement learning (DRL) in surgical autonomy have shown promising results in automating various surgical sub-tasks. While most of these studies consider the rigid and soft body dynamics in the surgery such as tissue deformation, only a few have investigated the situation where fluid is present. However, the presence of fluids, particularly blood, is common in surgeries, and interacting with them adds additional challenges to task automation. In this work, we investigate the use of DRL in automating blood suction, a common surgical sub-task where blood is removed from the surgical field. We build a blood suction simulation environment based on position-based fluids (PBF), train an agent with domain-randomized environment parameters through curriculum learning, and obtain a generalizable policy that can be applied to various shapes of tissue and types of liquid. Real-world experiments show that the agent can perform autonomous suction in different tissue models with different amounts and types of liquid, and only one of the 50 trials resulted in more than 3 ml of blood remaining.