RcFormer: Reconfigurable Self-Attention Transformer for Image Restoration

Adverse weather and imaging environments may degrade image quality and pose a significant challenge to the visual perception systems of multimedia. Various image restoration tasks necessitate the modeling of multiscale features, which is highly demanding on networks. To date, Vision Transformer has exhibited impressive image restoration performance. However, in this model, global self-attention is computationally expensive and local self-attention typically limits the interaction domain of each token. To solve this problem, we propose a novel reconfigurable self-attention transformer called RcFormer, which is designed to adequately model multiscale image features. This is achieved through a cross-grouped transformer (CGTransformer) block that uses convolution, area self-attention, and row-column self-attention for different head groups. CGTransformer is combined with an intragroup operation interaction structure. Moreover, an intergroup reconfigurable mechanism is implemented based on CGTransformer and channel circulation. The combination of multiple operations effectively enhances the modeling capability in the spatial and channel dimensions for various image recovery tasks. The performance of the proposed RcFormer is compared with low-level vision modules in a unified framework. Extensive experiments demonstrated that RcFormer exhibited a superior performance for the following image restoration tasks: image dehazing, rain streak removal, raindrop removal, snow removal, and single image deblurring. The source code is publicly available athttps://github.com/dehazing/RcFormer.