QGMNet: Question-Type Guided Multiview Network for Remote Sensing Image Question Answering

Remote Sensing Image Question Answering (RSIQA) is a task that searches remote sensing images (RSIs) based on questions to generate accurate answers. These questions are categorized into different types, with each question type highlighting distinct semantics of the RSIs. Answering different types of questions helps non-expert users better understand the rich semantic content within RSIs. However, existing methods have not fully exploited question type to guide cross-modal interaction. To overcome this limitation, a Question-type Guided Multi-view Network (QGMNet) is proposed. The proposed network: 1) employs a dual-branch structure to extract global and local image features for comprehensive representations; 2) applies a question-type multi-view attention method to guide image-question interaction; and 3) adopts a question-type stage-wise training strategy with a joint loss integrating cross-entropy, focal, and contrastive losses for cross-modal learning. Overall, the QGMNet effectively leverages question type to achieve more fine-grained cross-modal interaction. Experiments on RSVQA-LR, RSVQA-HR, and RSIVQA datasets demonstrate that the proposed network achieves state-of-the-art performance.