Efficient Six-Degrees of Freedom (6-DoF) Grasp Pose Detection in Cluttered Scenes via Multimodal Fusion and Object-Centric Receptive Fields

Efficient 6-DoF grasp pose detection is a fundamental and challenging task in robotic manipulation. Existing methods mainly rely on the geometric features of point clouds to synthesise 6-DoF grasp poses, as point clouds contain rich geometric information. However, the rich visual cues contained in images, such as object edges, textures, and other fine-grained details, can also serve as important references. In this paper, we propose a simple yet efficient multimodal 6-DoF grasp pose detection algorithm that also integrates object-centric receptive fields. Specifically, we design an early fusion strategy to integrate point cloud and image information, providing more discriminative multimodal features for grasp pose detection. Furthermore, an object-centric receptive fields module based on grounded segment anything model (Grounded-SAM) is introduced, which provides global references for each point during the grasp parameter decoding stage. Experimental results show that the proposed model achieves competitive performance on the GraspNet-1Billion dataset. Compared with baseline, the AP improves by 7.44, 8.42, and 2.37 in Seen, Similar, and Novel scenes, respectively. To test the model's robustness to image noise, we designed a series of noise experiments, demonstrating the model's strong anti-interference ability. Finally, we deployed the algorithm on a Franka robot and conducted extensive grasping experiments, achieving a high grasp success rate.