Lexicon-enhanced transformer with spatial-aware integration for Chinese named entity recognition

Chinese Named Entity Recognition (CNER) is a fundamental and crucial task in information extraction. In recent years, pre-trained language and lexicon-based models have proven more powerful than the previous character-based models in CNER tasks. However, existing lexicon-enhanced BERT models neither integrate lexical knowledge into the fundamental layers of the bidirectional transformer model nor explicitly align character features with lexicon features. In this paper, we propose a spatial-aware lexicon adapter (SALA), a neural adapter capable of dynamically integrating character and lexical representations through spatial-aware attention. SALA is incorporated between the layers of BERT to inject lexical information into the deep contextual representations of corresponding character sequences. The resulting fused vectors are further trained in SALA-BERT to enhance CNER. We evaluate SALA-BERT on various Chinese NER tasks. Compared to previous state-of-the-art models, it achieves comparable or better performance.