SAM-BoMu: Boundary-Aware Multi-Channel Fusion Network for Precise Polyp Segmentation

Accurate localization and segmentation of colorectal polyps are of critical importance for the diagnosis and treatment of colorectal cancer. Given the high demand for precise polyp segmentation in this context, the Segment Anything Model (SAM) has recently demonstrated remarkable potential in the field of medical imaging. However, due to the domain differences between natural and medical images, it faces segmentation challenges arising from the variability in polyp size and shape, coupled with boundary ambiguity. To address these challenges, we propose a SAM-based boundary-aware multi-channel fusion network named SAM-BoMu, which consists of a Multi-Channel Encoder (MCE) and a Multi-Fusion Decoder (MFD). The MCE consists of a ViT encoder, a CNN encoder, and a boundary extractor to extract low-frequency, high-frequency, and boundary features. To reduce the training cost of the ViT encoder, we design an Optimization-Aid Framework for Transformer (OAFT), which only introduces trainable modules to learn spatial and frequency domain details while freezing other parameters. Subsequently, we introduce a Boundary-Constrained Hybrid Fusion (BCHF) module into the MFD to utilize these features for enhancing the segmentation performance. We systematically evaluated the performance of SAM-BoMu on five public datasets. Experimental results show that SAM-BoMu achieves competitive SOTA-level performance across all benchmarks. It obtains the best Dice of 0.954 on CVC-ClinicDB. On a private dataset (AHJU-DB) provided by the Affiliated Hospital of Jiangnan University, it still attained a Dice coefficient of 0.943, further demonstrating the outstanding generalization capability of our approach.