A wearable thumb device for fruit firmness estimation with vision-based tactile sensing

Abstract

Recent advancements in non-destructive technologies have enabled precise firmness measurement for various fruits, including kiwifruit. However, existing methods remain limited by high costs, environmental sensitivity, and field application impracticality. This work introduces a novel wearable device for estimating non-destructive fruit firmness, combining human tactile interaction with vision-based tactile sensing and edge computing. Worn on the thumb, the device leverages embodied intelligence, merging intuitive human touch with the precision of a vision-based tactile sensor. A single-board computer processes tactile images locally, enabling reliable operation even in remote environments. The device employs our proposed deep learning model for real-time firmness predictions from a single palpation, minimizing repetitive handling and reducing fruit bruising. Its ergonomic, symmetrical design supports comfortable use on either hand, enhancing usability. Compact and portable, the device integrates essential components within a housing measuring 40 mm × 25 mm × 72 mm and weighing only 135 g. Validated through non-destructive ripeness assessments on ’Hayward’ Kiwifruit, the device demonstrated a strong correlation between tactile images and firmness values when paired with our proposed model, achieving a coefficient of determination (R2) of 0.89. This study created a dedicated dataset on Kiwi firmness to support model development and validation.

Keywords Agricultural device, Deep learning, Firmness, Vision-based tactile sensing, Wearable