Fidelity-driven data augmentation for multimodal large language model on architectural heritage interpretation
Rong Huang, Haichuan Lin, Wei Zeng
The Hong Kong University of Science and Technology (Guangzhou) Hong Kong University of Science and Technology
阅读操作
确认中在文库中上传 PDF 后可生成中文音频讲解。
摘要与影响
Multimodal large language models (MLLMs) offer transformative potential for architectural heritage interpretation but struggle with nuanced spatial and semantic analysis. This limitation stems from a critical shortage of large-scale, high-fidelity training data. We address this gap with a fidelity-driven data augmentation framework incorporating Structural- and Semantic-aware Augmentation modules into a diffusion model. The framework generates 1672 high-fidelity synthetic images paired with 59,884 VQA samples. Quantitative evaluations demonstrate that these synthetic images preserve spatial-structural and semantic fidelity comparable to real-world data. Furthermore, MLLMs fine-tuned on this dataset show significantly improved reasoning performance without overfitting or interference. By addressing fundamental data constraints, our framework facilitates the transition from traditional, task-specific tools to general-purpose, instruction-following MLLMs capable of supporting diverse heritage preservation and urban analysis tasks.
逐年被引趋势
关键指标
同类平均 = 1
同领域 · 同年份 · 同类型
Google Scholar 与 OpenAlex 的被引统计范围不同,数值存在差异属正常。
AI 辅助阅读
依据:摘要
可就本文提问;依据不足时会说明。
学术脉络
学科主题
计算机 / AIMultimodal Machine Learning Applications
Generative Adversarial Networks and Image Synthesis · Advanced Neural Network Applications