Dynamic Gated Cross-Modal Fusion with Sarcastic-aware Contrastive Regularization for Multimodal Sarcasm Detection - Research paper - propose SaCR and dynamic gated fusion

Sep 15, 2026

Multimodal sarcasm detection identifies sarcastic intent from paired text and images, where irony often arises from inconsistencies between literal meaning and contextual cues. Existing approaches struggle because modality contributions are instance-dependent and semantic alignment can be misleading, masking contradictory intent. The proposed MSD framework combines Dynamic Gated Cross-Modal Fusion with Sarcastic-aware Contrastive Regularization (SaCR). Bidirectional gated interaction filters cross-modal features and calibrates textual and visual evidence per instance, while a dynamic fusion gate balances modality importance. SaCR applies label-aware contrastive constraints to encourage consistency for non-sarcastic samples and suppress misleading consistency for sarcastic cases. End-to-end multi-objective training jointly optimizes multimodal classification and auxiliary unimodal supervision, achieving improved results on MMSD and MMSD2.0.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages6

Reading Time15 min

AccessFree to download

Introduction

Challenges in multimodal sarcasm detection

Proposed framework: Dynamic Gated Cross-Modal Fusion and SaCR

Dynamic Gated Cross-Modal Fusion with Sarcastic-aware Contrastive Regularization for Multimodal Sarcasm Detection 1st Hao Guo School of Computer and Information Anhui Polytechnic University Wuhu, China [sifan10077@gmail.com](sifan10077@gmail.com) 2nd Subin Huang∗† School of Computer and Information Anhui Polytechnic University Wuhu, China [subinhuang@ahpu.edu.cn](subinhuang@ahpu.edu.cn) 3rd Junjie Chen School of Computer and Information Anhui Polytechnic University Wuhu, China [jorji.chen@gmail.com](jorji.chen@gmail.com) arXiv : 2608 . 19942v 1 [cs .CL] 20 Aug 2026 4th Zhifa Geng School of Computer and Information Anhui Polytechnic University Wuhu, China [zhifageng77@gmail.com](zhifageng77@gmail.com) 5th Sanmin Liu School of Computer and Information Anhui Polytechnic University Wuhu, China [sanmin.liu@ahpu.edu.cn](sanmin.liu@ahpu.edu.cn) 6th Chao Kong† School of Computer and Information Anhui Polytechnic University Wuhu, China [kongchao@ahpu.edu.cn](kongchao@ahpu.edu.cn) Abstract—Multimodal sarcasm detection aims to identify sarcastic intent from multimodal content, where inconsistencies between literal meaning and contextual cues often signal irony. This task has attracted increasing research attention. However, accurate detection remains challenging due to instance-dependent modality contributions and misleading semantic consistency, where surface-level alignment masks underlying contradictory intent. Existing methods often rely on fixed fusion strategies and treat sarcasm as generic cross-modal mismatch, limiting their ability to capture subtle sarcasm cues and instance-specific modality interactions. To address these challenges, we propose a novel MSD framework that integrates Dynamic Gated CrossModal Fusion with Sarcastic-aware Contrastive Regularization (SaCR). Specifically, a bidirectional gated interaction module performs cross-modal feature filtering and adaptively calibrates textual and visual contributions at the instance level. A dynamic fusion gate further balances modality importance to generate more robust multimodal representations. Furthermore, SaCR is introduced as a label-aware contrastive regularization objective that encourages semantic consistency for non-sarcastic samples while suppressing misleading consistency in sarcastic cases. The proposed framework is trained end-to-end with a multi-objective learning strategy that jointly optimizes multimodal classification and auxiliary unimodal supervision. Extensive experiments on MMSD and MMSD2.0 demonstrate that the proposed method consistently outperforms strong baselines. Index Terms—multi-modal sarcasm detection, sarcasm detection, incongruity perception I. INTRODUCTION Multimodal sarcasm detection (MSD) aims to identify sarcastic intent from paired textual and visual content [1]–[4] . †Subin Huang and Chao Kong are with Bigdata and Responsible Artificial Intelligence for National Governance, Renmin University of China, Beijing 100872, China. This work was also supported in part by the Anhui Provincial Natural Science Foundation under Grant No. 2308085MF220 . * Subin Huang is the corresponding author. (a) just got over the flu (b) no jesus on this easter cup now i 'm in the hospital fun ? ! ? ! time to throw a fit ! Fig. 1. Illustrative examples of surface-level consistency masking underlying incongruity in multimodal sarcasm. (a) The text ”fun” and the hospital setting are semantically compatible but sentimentally contradictory. (b) The text ”no Jesus” and the cup image share a literal subject, yet convey ironic intent. Our method aims to capture such hidden incongruity beyond literal alignment. Unlike general multimodal classification, MSD is particularly challenging because sarcastic evidence is often both instancedependent and deceptively aligned across modalities [5]–[7] . In some cases, sarcasm is mainly triggered by visual contradiction, whereas in others it relies more on textual framing or subtle cross-modal

FAQs

What makes multimodal sarcasm detection challenging in this work?

The task is difficult because modality contributions vary by instance and semantic alignment can be misleading, masking underlying intent contradiction between text and images.

How does Dynamic Gated Cross-Modal Fusion work?

It uses bidirectional gated interaction to filter cross-modal features and adaptively calibrates textual and visual contributions at the instance level, then applies a dynamic fusion gate to balance modality importance.

What is SaCR and what does it regularize?

SaCR is a label-aware contrastive regularization objective that encourages semantic consistency for non-sarcastic samples and suppresses misleading consistency in sarcastic cases.

Sep 15, 2026

Uploaded by River Wang