A Tomato Recognition Method for Harvesting with Robots using Point Clouds

This paper proposes a method to detect the peduncles of tomatoes for a harvesting robot using point clouds. The main objective of this study is to develop automated harvesting methods for harvesting robots that can work as temporary workers. The harvesting robot is equipped with an RGBD camera to detect the peduncles and an end effector to harvest the tomatoes at the detected cutting point. The proposed method detects tomato peduncles based on a point cloud taken by an RGBD camera. First, classification and clustering of the tomato regions are performed to narrow the searching regions. Two resolution voxelization is used to reduce the computational time for classification and clustering. Next, a maturity estimation selects ripe tomato segments based on their color. Finally, an energy function is defined based on the condition of a peduncle and this function is minimized to identify the cutting point on each peduncle. To experimentally demonstrate the effectiveness of our approach, a robot was used to harvest the tomatoes on a real farm. The proposed method detected the tomato peduncles accurately if peduncles are short in length. The harvesting robot could harvest the tomatoes successfully by cutting the peduncles.