Does Multi-Modal Fusion Always Work—A Multi-Modal Decoupled Learning Neural Network (MDLNN) for the Detection of Fake News

The detection of fake news on social media is becoming progressively essential. Current multi-modal detection methods emphasize the learning of complementarity across heterogeneous modalities or the extraction of uncertainty within them to enhance integration, which neglects the coupling and dynamic characteristics of multi-modal content and the interactive mechanisms between them. Inspired by human logical inference perception, we propose a novel multi-modal decoupled learning neural network (MDLNN) to enhance comprehension of coupled multi-modal content. MDLNN primarily comprises three modules. The multi-modal feature extraction module with decoupled learning (MDL) aims at extracting detailed informative features from multi-modal content through a decoupled learning process and facilitates trusted integration at the feature level. The multi-modal feature fusion module with gating units (MGU) seeks to enhance the comprehension of multi-modal content by modeling three intricate interactive mechanisms of informative features and facilitates trusted integration at the decision level. The fake news detection module (FND) focuses on generating detection results and ensures event invariance for multi-modal joint features. Comprehensive qualitative and quantitative experiments on two datasets demonstrate the superior performance of MDLNN.