CHARM Charge Calibration and Acoustic Rescue for LLM-based Multimodal Sarcasm Detection

Q: What problem does CHARM address in LLM-based sarcasm detection?

It addresses LLMs’ tendency to over-predict the sarcastic class in zero-shot settings and the limited, uneven transfer of prosodic cues across languages.

Q: How does Bidirectional Charge Calibration (BiCAL) reduce bias without fine-tuning?

BiCAL steers the LLM using charged prompts along a symmetric axis for opposing sarcastic and literal verdicts, then aggregates the induced directional biases so they cancel while an unbiased pragmatic signal remains.

Q: What role does Acoustic Late-Fusion Rescue (ALFR) play?

ALFR fuses BiCAL-calibrated text votes with prosodic descriptors and LLM-generated auditory-perception probes using a shallow classifier, actively down-weighting saturated text evidence and emphasizing acoustic cues.

Jul 16, 2026

Upload by Quinn

Sarcasm detection resolves conflicts between literal wording and intended pragmatic meaning, yet zero-shot instruction-tuned LLMs often over-predict the sarcastic (positive) class and underuse prosodic cues, with inconsistent transfer across languages. CHARM introduces a training-free multimodal framework combining BiCAL bidirectional charge calibration and ALFR acoustic late-fusion rescue. Bidirectional prompt charge symmetry cancels bias while recovering an unbiased pragmatic signal. ALFR integrates calibrated votes with prosodic descriptors and auditory-perception probes, suppressing saturated text evidence in favor of acoustic cues. Experiments report improved zero-shot Macro-F1 on MUStARD and stronger gains on CMMA, supported by statistical significance and cross-cultural prosodic decoupling findings.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages18

Reading Time45 min

Introduction

Motivation and problem setup

Sycophancy bias and charge calibration intuition

CHARM: Charge Calibration and Acoustic Rescue for LLM-based Multimodal Sarcasm Detection Qiyang Sun Student Member, IEEE, Yi Chang*, Yupei Li, Xi Shao Member, IEEE, Zixing Zhang* Senior Member, IEEE, and Bjrn W. Schuller Fellow, IEEE arXiv :2607 . 1 1 102v 1 [ cs . SD] 13 Jul 2026 Abstract—Sarcasm detection, the identification of discrepancies between literal and intended meaning, is a fundamental task in affective computing. However, zero-shot instruction-tuned Large Language Models (LLMs) systematically over-predict the positive (sarcastic) class across the entire capability spectrum, while the prosodic cues humans rely on remain underexploited and transfer unevenly across languages. We introduce CHARM (Charge Calibration and Acoustic Rescue for Multimodal Sarcasm Detection), a training-free framework that couples two modules. Bidirectional Charge Calibration (BiCAL) steers the LLM toward opposing sarcastic and literal verdicts along a symmetric axis of charged prompts; the induced directional biases cancel by construction, and a simple aggregation recovers an unbiased pragmatic signal. Acoustic Late-Fusion Rescue (ALFR) then fuses the calibrated votes with prosodic descriptors and LLM-generated auditory-perception probes through a shallow classifier, actively down-weighting saturated text votes in favour of acoustic evidence. Without fine-tuning any backbone, BiCAL attains the highest reported zero-shot text-only Macro-F1 of 0.787 on MUStARD, while ALFR lifts weak backbones by up to +0.382 Macro-F1 on CMMA. A Stouffer meta-analysis confirms statistical significance on MUStARD and CMMA (Z = 13 .89 and Z = 34 .64, respectively; p < 10 −43). Our analysis further uncovers a cross-cultural prosodic decoupling: low-level acoustics fail to transfer across languages, whereas high-level perceptual abstractions remain robust. Together, these components yield an explainable, cross-lingual multimodal detector. Index Terms—Multimodal Sarcasm Detection, Large Language Models (LLMs), Zero-Shot Learning, Charge Calibration, Acoustic Late Fusion, Cross-Cultural Analysis, Prosodic Decoupling, Explainability I. INTRODUCTION HUMANS recognise sarcasm almost instantly from a dry tone or a knowing pause. However, contemporary instruction-tuned Large Language Models (LLMs) systematically err in the opposite direction. Sarcasm detection involves Qiyang Sun, Yi Chang, and Yupei Li are with GLAM – the Group on Language, Audio, & Music, Imperial College London, UK. e-mail: [q.sun23@imperial.ac.uk](q.sun23@imperial.ac.uk); [yichang312@gmail.com](yichang312@gmail.com); [yupei.li22@imperial.ac.uk](yupei.li22@imperial.ac.uk) Xi Shao is with the College of Telecommunications and Information Engineering, Nanjing University of Posts and Telecommunications, China, e-mail: [shaoxi@njupt.edu.cn](shaoxi@njupt.edu.cn) Zixing Zhang is with the College of Computer Science and Electronic Engineering, Hunan University, China; Zixing Zhang is also with the Shenzhen Research Institute, Hunan University, China. e-mail: [zixingzhang@hnu.edu.cn](zixingzhang@hnu.edu.cn) Bjrn W. Schuller is with GLAM – the Group on Language, Audio, & Music, Imperial College London, UK; CHI – Chair of Health Informatics, TUM University Hospital, Germany; relAI – the Konrad Zuse School of Excellence in Reliable AI, Germany; MDSI – Munich Data Science Institute, Germany; and MCML – Munich Center for Machine Learning, Germany. [e-mail: bjoern.schuller@imperial.ac.uk](e-mail: bjoern.schuller@imperial.ac.uk)[ ](e-mail: bjoern.schuller@imperial.ac.uk)Corresponding author: Yi Chang, Zixing Zhang Manuscript received April 19, 2021; revised August 16, 2021 . identifying the deliberate mismatch between the literal and intended meaning of a speaker [1] . This operation serves asa cornerstone task within affective computing. This field aims to enable machines to perceive, understand, interpret, and express human emotions [2], [3] . Unlike standard sentiment analysis [

FAQs

What problem does CHARM address in LLM-based sarcasm detection?

It addresses LLMs’ tendency to over-predict the sarcastic class in zero-shot settings and the limited, uneven transfer of prosodic cues across languages.

How does Bidirectional Charge Calibration (BiCAL) reduce bias without fine-tuning?

BiCAL steers the LLM using charged prompts along a symmetric axis for opposing sarcastic and literal verdicts, then aggregates the induced directional biases so they cancel while an unbiased pragmatic signal remains.

What role does Acoustic Late-Fusion Rescue (ALFR) play?

ALFR fuses BiCAL-calibrated text votes with prosodic descriptors and LLM-generated auditory-perception probes using a shallow classifier, actively down-weighting saturated text evidence and emphasizing acoustic cues.

Jul 16, 2026

Upload by Quinn