PnuTac: A vision-based pneumatic tactile sensor for slip detection and object classification

Soft optical tactile sensors allow robots to capture important information, such as contact geometry, estimations of object compliance, and slip detection. However, most optical tactile sensors utilize gel-filled elastic membranes with non-variable stiffness. To overcome this limitation, this paper presents the development of a pneumatic tactile sensor with tunable pressure (PnuTac). The sensor comprises a pneumatic system, an elastic membrane, and a sealed chamber with a camera inside. The inner side of the membrane layer has dot markers on its surface that are used for slip detection. Slippage is prevented by controlling a Robotiq 2-finger gripper that closes according to the slip detection signal. Additionally, objects held by the gripper appear as contours in sensor images. A dataset of 10,000 such images from 10 tools was utilized for training a VGG-19 convolutional neural network for tool classification. Our results show that increasing the pressure of the PnuTac sensor reduces the time it takes for the gripper to stabilize a slipping object. The overall success rate of slip detection was determined to be 87%. The trained neural network, fed from the PnuTac's sensor live data, successfully classified 8 out of the 10 tools.