Query-Sparsified Transformer with Adapter Tuning for Log-Based Anomaly Detection

Log anomaly detection is critical for ensuring the reliability and security of large-scale computer systems. Existing approaches, including rule-based, statistical, and deep learning methods, struggle to capture complex dependencies in semi-structured logs and face scalability bottlenecks when handling long sequences. To address these limitations, we propose a Transformer-based framework enhanced with Query-Sparsified Attention and Adapter Tuning. The Query-Sparsified mechanism dynamically selects salient queries, reducing the quadratic cost of full attention while retaining key contextual information. Meanwhile, the lightweight Adapter modules enable parameter-efficient fine-tuning, improving adaptability to evolving log formats with minimal additional training. Experiments on three real-world datasets (HDFS, BGL, Thunderbird) demonstrate that our method consistently outperforms state-of-the-art baselines, achieving higher precision, recall, and F1-scores. These results highlight the effectiveness and efficiency of the proposed framework, making it a practical solution for scalable and robust log anomaly detection in modern distributed systems.