UTB-Net: a UNet-like transformer with hierarchical feature fusion for high-resolution remote sensing urban image segmentation

Semantic segmentation of high-resolution remote sensing sensor urban imagery is essential for land monitoring and environmental change detection. However, accurate segmentation remains challenging due to the inherent difficulty in simultaneously modeling long-range global semantics and fine-grained local details—a dilemma that existing CNN-based or Transformer-based architectures fail to resolve effectively. To overcome this, we propose UTB-Net, a UNet-like Transformer that introduces a hierarchical complementary feature fusion paradigm. Unlike previous methods that treat multi-scale features independently or fuse them via simple concatenation, our approach achieves deep bidirectional alignment between structure-aware and detail-sensitive representations through three novel components: (1) a complementary feature attention module that enables cross-gating interaction between global context and local texture; (2) a differential attention gated fusion module that explicitly quantifies and aligns semantic gaps between encoder and decoder features; and (3) a multi-scale context-aware refinement head that adaptively enhances boundary details during resolution recovery. This design fundamentally addresses the core limitations in sensor-acquired urban scene segmentation, enabling precise delineation of objects across scales and orientations. Evaluated on Vaihingen, Potsdam, and LoveDA datasets, UTB-Net achieves mIoU scores of 84.6%, 87.6%, and 53.5%, demonstrating its effectiveness for sensor-acquired urban imagery.