Dual-BEATs: Unlocking Zero-Shot Stereo Audio Perception in Audio Large Language Models via Dithering

Jul 16, 2026

Upload by Levi

Multimodal Large Language Models provide strong semantic audio understanding but remain “spatially agnostic” because common pipelines flatten stereo to monochannel representations. Spatial perception approaches typically depend on complex room simulations or geometry-aware, custom-trained stereo encoders, reducing accessibility. Dual-BEATs routes left and right channels independently through two identical semantic encoders, while adding static uncorrelated dithering noise before encoding to preserve inter-channel variance through normalization layers. On ternary Left/Center/Right localization, dithered models reach up to 97.2% accuracy on subtle panning.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages14

Reading Time35 min

Abstract

1 Introduction

arXiv :2607 .08800v 1 [ cs . SD] 9 Jul 2026 Dual-BEATs: Unlocking Zero-Shot Stereo Audio Perception in Audio Large Language Models via Dithering Shuo-Chun Lin Institute of Information Science, Academia Sinica, Taiwan [shuochunlin@as.edu.tw](shuochunlin@as.edu.tw) Hen-Hsen Huang Institute of Information Science, Academia Sinica, Taiwan [hhhuang@iis.sinica.edu.tw](hhhuang@iis.sinica.edu.tw) Abstract Multimodal Large Language Models (LLMs) have remarkable semantic audio understanding, yet they remain “spatially agnostic” due to their reliance on monochannel audio representations. Currently, spatial audio perception methods mainly focus on complex room simulations and custom-trained, geometry-aware stereo encoders, which limits their accessibility and generalizability. In this paper, we introduce the Dual-BEATs architecture, in which the left and right audio channels are routed independently through two identical semantic encoders as an alternative to specialized spatial modules. To circumvent the architectural bottleneck where internal normalization otherwise erases the inter-channel variance of stereo audio, we inject a static, uncorrelated dithering noise floor prior to encoding. This dithering intervention establishes a macro-variance floor that “smuggles” spatial geometry across the normalization layers. Evaluated on a ternary directional classification task (Left, Center, Right), we demonstrate that dithered models achieve exceptional spatial resolution—reaching up to 97.2% localization accuracy even on subtle 0.5 panning amplitudes—and demonstrates robust, zero-shot generalization to entirely unseen spatial configurations. Our results suggest that with the appropriate acoustic regularization, standard multimodal models are natively capable of generalized stereo audio understanding. 1 Introduction The rapid evolution of multimodal Large Language Models (LLMs) has fundamentally transformed computational audio perception. Foundational open-weight architectures such as Audio Flamingo [1], SALMONN [2], and established end-to-end systems such as Qwen2-Audio [3] have demonstrated remarkable zero-shot capabilities across a wide range of audio-linguistic tasks. By mapping continuous audio waveforms to the discrete textual embedding space of an LLM, these models can successfully reason about complex speech, acoustic environments, and semantic events, exhibiting profound semantic depth. However, despite these advanced semantic capabilities—exemplified by recent models such as Music Flamingo [4] which perform complex chain-of-thought reasoning over music theory—current foundational audio models operate under a restrictive architectural assumption: they process sound with no concept of physical space. The standard preprocessing pipeline for general audio LLMs routinely downmixes stereo or multi-channel audio into a single mono audio channel prior to Preprint. encoding [5; 6] . This spatial deficit extends far beyond open-weight architectures—even state-ofthe-art proprietary multimodal systems, such as the GPT-4 family [7; 8] and Gemini [9], rely on preprocessing pipelines that inherently flatten audio into mono mel-spectrograms. Consequently, because these models are architecturally deaf to inter-channel geometry, any spatial localization they output is purely an artifact of semantic hallucination rather than true acoustic perception. Recent efforts to address this spatial blindness often rely on complex 3D room simulations or customtrained spatial encoders that are tightly coupled with their LLM backbones [10; 11] . While effective for specialized applications, these purpose-built architectures are highly sensitive to acoustic domain shifts. For instance, adapting an encoder optimized for meticulously simulated binaural environments to process standard, unsimulated stereo mixes can aggressively disrupt its pre-trained semantic alignment. A more intuitive and modular approach to generalized stereo percept

FAQs

What problem does Dual-BEATs address in audio multimodal LLMs?

It addresses the spatial agnosticism caused by converting stereo audio into monochannel representations, which removes inter-channel geometry needed for localization.

How does Dual-BEATs preserve stereo spatial information through the model?

It independently encodes left and right channels using two identical semantic encoders, and injects static uncorrelated dithering noise before encoding to maintain inter-channel variance across normalization layers.

What localization results does the dithering approach achieve?

Evaluations on a ternary Left/Center/Right classification show dithered models achieve up to 97.2% localization accuracy, including on subtle 0.5 panning amplitudes, with robust zero-shot generalization to unseen spatial configurations.

Jul 16, 2026

Upload by Levi