VideoStir: Understanding Long Videos via Spatio-Temporally Structured and Intent-Aware RAG

Scaling multimodal large language models (MLLMs) to long videos is constrained by limited context windows.While retrievalaugmented generation (RAG) is a promising remedy by organizing query-relevant visual evidence into a compact context, most existing methods (i) flatten videos into independent segments, breaking their inherent spatio-temporal structure, and (ii) depend on explicit semantic matching, which can miss cues that are implicitly relevant to the query's intent.To overcome these limitations, we propose VideoStir, a structured and intent-aware long-video RAG framework.It firstly structures a video as a spatiotemporal graph at clip level, and then performs multi-hop retrieval to aggregate evidence across distant yet contextually related events.Furthermore, it introduces an MLLM-backed intentrelevance scorer that retrieves frames based on their alignment with the query's reasoning intent.To support this capability, we curate IR-600K, a large-scale dataset tailored for learning frame-query intent alignment.Experiments show that VideoStir is competitive with stateof-the-art baselines without relying on auxiliary information, highlighting the promise of shifting long-video RAG from flattened semantic matching to structured, intent-aware reasoning.Codes and checkpoints are available at https: //github.com/RomGai/VideoStir.