Evaluating Object Hallucination in Large Vision-Language Models
Yifan Li, Yifan Du, Kun Zhou, Jinpeng Wang, Zhao Xin, Ji-Rong Wen
Renmin University of China Meituan (China)
阅读操作
确认中在文库中上传 PDF 后可生成中文音频讲解。
摘要与影响
Inspired by the superior language abilities of large language models (LLM), large visionlanguage models (LVLM) have been recently proposed by integrating powerful LLMs for improving the performance on complex multimodal tasks.Despite the promising progress on LVLMs, we find that they suffer from object hallucinations, i.e., they tend to generate objects inconsistent with the target images in the descriptions.To investigate it, this work presents the first systematic study on object hallucination of LVLMs.We conduct the evaluation experiments on several representative LVLMs, and show that they mostly suffer from severe object hallucination issues.We further discuss that the visual instructions may influence the hallucination, and find that: objects that frequently appear in the visual instructions or co-occur with the image objects are obviously prone to be hallucinated by LVLMs.Besides, we further design a polling-based query method called POPE for better evaluation of object hallucination.Experiment results show that our POPE can evaluate object hallucination in a more stable and flexible way.
逐年被引趋势
关键指标
同类平均 = 1
同领域 · 同年份 · 同类型
Google Scholar 与 OpenAlex 的被引统计范围不同,数值存在差异属正常。
AI 辅助阅读
依据:摘要
可就本文提问;依据不足时会说明。
学术脉络
学科主题
计算机 / AIMultimodal Machine Learning Applications
Advanced Neural Network Applications · Generative Adversarial Networks and Image Synthesis
参考文献 38
此处列出前 3 条
引用本文 3,117
按被引量排序,此处列出前 3 条