CosFormer: A Code Semantic-Aware Transformer for Vulnerability Detection

Deep learning-based vulnerability detection has made significant strides, surpassing traditional static and dynamic analysis methods. However, existing approaches, including Graph Neural Networks (GNNs) and Transformer-based models, still struggle to fully capture complex code semantics. In this paper, we proposeCosFormer, a novel Code Semantic-aware Transformer tailored for vulnerability detection.CosFormerintroduces two key components: Code Semantic-aware Embedding, which enhances semantic representation at both the token and line levels, and Spatial Dependency-aware Encoding, which integrates structural dependencies from Control Flow Graphs (CFGs) and Program Dependency Graphs (PDGs) to guide attention toward vulnerability-relevant code. We evaluateCosFormeron four benchmark datasets, including a real-world dataset, and demonstrate its superior performance.CosFormerachieves the highest F1 scores across all tasks, outperforming state-of-the-art GNN-based and Transformer-based models, as well as large language models (LLMs). Notably,CosFormerachieves a Cross F1 of 69.37 and a Mixed F1 of 58.42 in generalization evaluation, surpassing all baselines. These results highlightCosFormer’s effectiveness and robustness in detecting vulnerabilities across diverse and previously unseen codebases.