SkyNative: A Native Multimodal Architecture for Remote Sensing Vision-Language Understanding
Xiao Yang, Ronghao Fu, Zhiwen Lin, Zhuoran Duan, Lang Sun, Jiasen Hu, Jiashun Zhu, Jiasen Hu 等 10 位
Jilin University
阅读操作
确认中在文库中上传 PDF 后可生成中文音频讲解。
摘要与影响
Remote sensing vision-language models (RS-VLMs) commonly employ a pretrained vision encoder and a projection module to map image features into the token space of a large language model. Although effective, this modular RS-VLMs separates visual representation from language reasoning, potentially limiting the direct involvement of fine-grained visual evidence in complex spatial inference. This challenge is particularly relevant to remote sensing imagery, which often covers broad geographic areas and contains multi-scale objects, dense target distributions, and intricate spatial layouts. In this paper, we propose SkyNative, the first study to explore a native multimodal architecture for remote sensing vision-language tasks. SkyNative converts remote sensing images into visual tokens through a lightweight patch embedding module and places them together with text tokens in a shared autoregressive sequence, allowing textual tokens to directly access the preceding visual context. To accommodate the heterogeneous characteristics of the two modalities, we further adopt a modality-aware decoupling mechanism that applies modality-specific projections, normalization, and feed-forward transformations while processing both modalities through shared causal self-attention. Extensive experiments demonstrate SkyNative's strong capabilities in dense small-object perception, large-format contextual understanding, complex reasoning, and robustness, with scores of 68.93%, 47.40%, and 63.43% on HRRSD, RSHR reasoning, and OmniEarth, respectively. These results suggest that the native VLM architecture explored in SkyNative represents a promising approach to RS vision-language modeling.
逐年被引趋势
暂无年度引用数据
关键指标
同类平均 = 1
同领域 · 同年份 · 同类型
Google Scholar 与 OpenAlex 的被引统计范围不同,数值存在差异属正常。
AI 辅助阅读
依据:摘要
可就本文提问;依据不足时会说明。
学术脉络
学科主题
计算机 / AIMultimodal Machine Learning Applications
Advanced Neural Network Applications · Remote-Sensing Image Classification