See the Emotion: A Facial Emoji Proxy Modeling for EEG Emotion Recognition

Jul 16, 2026

Upload by Aurora

Despite the high accuracy of EEG-based emotion recognition, existing models remain opaque “black boxes,” lacking semantic grounding between abstract neural features and human-interpretable states. The work reframes explainability as a cross-modal generation task, replacing feature attribution with behavioral visualization. It introduces Facial Emoji Proxy Modeling, translating high-dimensional EEG signals into identity-anonymized facial emojis grounded in neural-facial association. The framework combines FMENet and a Facial Emoji Learning Branch and achieves state-of-the-art EEG-only results while producing semantically faithful, privacy-preserving facial animations of emotional evolution.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages25

Reading Time1 h 3 min

Introduction

Interpretable EEG and the need for semantic bridges

Prior approaches to explainability

Proposed paradigm: Facial Emoji Proxy Modeling

Method overview and key components

See the Emotion: A Facial Emoji Proxy Modeling for EEG Emotion Recognition Jingjing Hu 1 Dan Guo∗ 1 2 3 Haofan Cheng 1 Ying Zeng 4 Zhan Si 5 Jinxing Zhou 6 Meng Wang 1 2 3 arXiv :2607 .029 12v 1 [ cs .CV] 3 Jul 2026 Abstract Despite the high accuracy of EEG-based emotion recognition, existing models remain opaque“black boxes”, lacking semantic grounding between abstract neural features and humaninterpretable states. In this paper, we reframe EEG explainability as a cross-modal generation task, shifting the paradigm from feature attribution to behavioral visualization. We introduce Facial Emoji Proxy Modeling, a novel framework that translates high-dimensional EEG signals into identity-anonymized facial emojis. Guided by the neuroscientific inspiration of neural-facial association, this approach grounds neural representations in the manifold of observable facial dynamics. Technically, our framework integrates FMENet, a specialized backbone modeling expression-relevant spatial synergies, and the Facial Emoji Learning Branch (FELB), which treats emoji reconstruction as a structured semantic regularizer. Extensive experiments on EAVand MMER benchmarks demonstrate that our method achieves state-of-the-art accuracy among EEG-only models. Crucially, it generates semantically faithful facial animations that provide a transparent, privacy-preserving window into the brain’s emotional evolution, effectively allowing users to “see the emotion” directly from neural signals. Code is available at [https://github](https://github) . com/xian-sh/SeeEmotion 1. Introduction Decoding emotional states from electroencephalography (EEG) signals is a foundational challenge in affective com- 1Hefei University of Technology 2Institute of Artificial Intelligence, Hefei Comprehensive National Science Center 3The Key Laboratory of Knowledge Engineering with Big Data, Hefei University of Technology 4PLA Information Engineering University 5University of Science and Technology of China 6MBZUAI. Correspondence to: Dan Guo <[guodan@hfut.edu.cn](guodan@hfut.edu.cn) >. Proceedings of the 43 rd International Conference on Machine Learning, Seoul, South Korea. PMLR 306, 2026 . Copyright 2026 by the author(s) . puting and human-centered AI (Pillalamarri & Shanmugam, 2025) . While deep learning has pushed classification accuracy to impressive levels(Zhong et al., 2022 ; Altaheriet al., 2023 ; Lawhern et al., 2018 ; Ding et al., 2022 ; Eldeleet al., 2024), a profound dichotomy persists between model performance and human interpretability. In essence, prevailing models behave as inscrutable “black boxes”, mapping noisy, high-dimensional EEG sequences to discrete labels without offering an intelligible trace of how the emotional experience unfolds in the neural signal. This opacity fundamentally limits their deployment in high-stakes domains, such as clinical neurofeedback and trustworthy BCI, where verifiable understanding is as critical as prediction. The quest for interpretability has traditionally followed two paths. Post-hoc attribution methods (e.g., Grad-CAM) analyze trained models to highlight influential EEG channels or time points (Miao et al., 2025 ; Bouazizi & Ltifi, 2025 ; Vakala Rani et al., 2025) . However, these saliency maps remain abstract, pointing to where the model attends rather than explaining what it perceives. Alternatively, built-inneuroscientific constraints incorporate priors like spectral dynamics into network design (Zhang et al., 2021 ; Ning et al., 2023 ; Liu et al., 2024) . Yet, their explanations remain couched in domain-specific jargon (e.g.,“alpha-band power in frontal electrodes”), failing to bridge the “last mile” to externally observable behavioral semantics. The core bottleneck lies in the absence of a semantic bridge capable of translating internal neural dynamics into a intuitively understandable vocabulary. To address this, we propose a paradigm shift: moving from the question of“which features matter?”

FAQs

Why are existing EEG emotion recognition models considered “black boxes”?

They map noisy, high-dimensional EEG sequences to discrete emotion labels without providing an intelligible trace of how emotional experience unfolds in neural signals.

How does the paper improve EEG explainability compared with attribution methods?

It shifts from highlighting influential channels or time points to generating behaviorally grounded visualizations, using identity-anonymized facial emojis as semantic proxies.

What does Facial Emoji Proxy Modeling generate from EEG signals?

The model translates raw EEG into dynamically evolving facial emojis, producing semantically faithful facial animations that reveal the emotional trajectory while preserving identity privacy.

Jul 16, 2026

Upload by Aurora