Vision–Language–Action Model for Agricultural Fresh Produce Handling with a Humanoid Robot
Kapalik Khanal, Biswash Khatiwada, Stephen Afrifa, Sawyer Exum, Fan Wu, Ramesh Bahadur Bist
Tribhuvan University North Carolina State University
阅读操作
确认中在文库中上传 PDF 后可生成中文音频讲解。
摘要与影响
Reaching, grasping, and placing delicate fresh produce are among the most labor-intensive operations in agricultural production, yet most robotic solutions remain task-specific, single-crop systems that generalize poorly across the morphological diversity of fresh produce. This study investigates whether a large, pre-trained Vision–Language–Action (VLA) model can be adapted with only modest demonstration data to serve as a flexible language-conditioned produce-handling policy on a humanoid robotic platform. The NVIDIA GR00T N1.7-3B foundation model is fine-tuned for a standard 23-degree-of-freedom Unitree G1 humanoid robot fitted with two Dex3-1 three-finger dexterous hands, whose arm and hand joint streams form a 24-dimensional combined manipulation command space. This study also compared the π0.5 VLA model. Demonstrations were collected through a Meta Quest 3 extended-reality teleoperation pipeline augmented with a multi-label recording mode that embeds a natural-language goal in each episode, yielding 605 language-conditioned demonstrations (261,724 synchronized frames) across four produce categories (lemon, peach, pear, and tomato). A dual-camera design paired a wide-angle head camera for global scene context with a wrist-mounted camera for close-range grasp feedback. The results show that the fine-tuned policy achieved a 96.25% individual-trial success rate and 85% on mixed-scene trials requiring retrieval of the language-specified item. However, adapting the π0.5 VLA model to the same data achieved 68.75% individual-trial success. Furthermore, a controlled ablation isolated the role of close-range perception. Without wrist-camera input, the policy reached every target in single-item trials but failed to secure a single grasp, resulting in 0% pick-and-place success despite comparable training loss. Therefore, close-range wrist perception was a precondition for contact in this setting. Together, these results demonstrate that VLA fine-tuning can transform a pre-trained humanoid foundation model into a reliable, data-efficient policy for multi-category fresh produce handling. This work provides a practical step toward flexible humanoid robotic systems for agricultural pick-and-place, sorting, and postharvest automation.
逐年被引趋势
关键指标
同类平均 = 1
同领域 · 同年份 · 同类型
Google Scholar 与 OpenAlex 的被引统计范围不同,数值存在差异属正常。
AI 辅助阅读
依据:摘要
可就本文提问;依据不足时会说明。
学术脉络
学科主题
工程Robot Manipulation and Learning
Smart Agriculture and AI · Reinforcement Learning in Robotics