A hierarchical approach of apple identification for robotic harvesting

Identifying fruits with limited visibility and fruits in-clusters have been a challenging problem in machine vision systems used in robotic tree fruit harvesting. Issues with clusters and partial to full occlusion of fruit could be minimized by strategically harvesting most visible fruits first. This work presents a hierarchical method for apple identification suitable for robotic harvesting. Test images were acquired from six overlapping sections of tree canopies trained in tall spindle architecture. An image processing method consisting of Circular Hough Transformation (CHT) and Blob Analysis was applied in an iterative fashion. Clearly visible fruit identified by CHT were preferred to partially visible apples for initial harvesting. These prioritized apples were then manually picked to prove the concept of hierarchical fruit identification approach. As images were taken again after harvesting well expose apples, partially or fully occluded apples were better exposed in successive iterations. This iterative process was applied over every image acquired from both sides of tree canopies until no apples were identified. In total, 980 images were taken from 240 canopy sections of 20 trees where 1807 apples were identified out of 1844 manual counts. On average, this method achieved 98% of identification accuracy. It was also found that 80% of apples were detected and harvested with images taken from one side of the canopy and remaining were harvested from the opposite side. Although this process is intuitive, the work provided a unique and novel insight into the fruit identification and harvesting accuracy achievable with such an approach in field environment, and showed huge potential of this machine vision system for robotic apple harvesting.