Detecting and Localizing Strawberry Centers for Robotic Harvesting in Field Environment

Automated or robotic harvesting methods are being investigated worldwide and have shown promising alternatives to manual harvesting in strawberry production. In robotic strawberry harvesting, the critical task of its machine vision system is to detect the presence and maturity of strawberries and estimate their precise location in the canopies. This study focused on the estimation and localization of strawberry centers in field environment to provide the 3D location of strawberry centers. It first applied a YOLOv4 approach to detect strawberries of different maturity levels (flower, immature, nearly mature, mature, and overripen) from an acquired RGB image. Matured strawberries detected by YOLOv4 were then used as inputs to a YOLOv4-tiny model to estimate berry centers in field conditions. A strawberry canopy dataset including 1300 selected RGB images was used for training the YOLOv4 model. Validation tests using 100 RGB images showed that the trained YOLOv4 model achieved an average precision (AP) of 91.73% in detecting mature strawberries at a reasonably high processing speed of 55.19ms. A dataset containing 750 images of single-strawberry was used in training the YOLOv4-tiny model. The trained model could detect the strawberry center in a processing time of 4.16ms per strawberry and achieved a mean average precision (mAP) of 86.45%. The average errors in estimating strawberry center locations were 1.65 cm on the x-axis, 1.53 cm on the y-axis, and 0.81 cm on the z-axis when the ZED camera was installed at similar to 100 cm. With precise detection of centers of strawberries by combining YOLOv4 and YOLOv4-tiny, the manipulator could receive accurate location information of strawberries to avoid inaccurate or failed picking during harvesting. Copyright (C) 2022 The Authors.