OWL - Spatial-Acoustic Geometry Encoder (SAGE) - Audio Large Language Models spatial reasoning

Aug 29, 2026

Spatial reasoning is essential for auditory perception, yet audio large language models (ALLMs) often depend on unstructured binaural cues and one-step inference. This restricts accuracy in direction and distance estimation and limits interpretable, multi-step reasoning. The work proposes SAGE, a geometry-aware audio encoder that aligns binaural acoustic features with 3D spatial structure using panoramic depth images and simulated room impulse responses during training, requiring only audio at inference. Building on SAGE, OWL integrates a spatially grounded chain-of-thought to rationalize direction-of-arrival (DoA) and distance estimates. A new BiDepth dataset with one million QA pairs improves robustness, reducing mean DoA error by 11° and boosting spatial reasoning QA accuracy by up to 25% versus BAT.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages25

Reading Time1 h 3 min

AccessFree to download

Introduction

Key limitations in existing ALLMs

Proposed solution: OWL framework

Geometry-aware representation via SAGE

Dataset and benchmarking

arXiv :2509 .26140v1 [ cs . SD] 30 Sep 2025 OWL: GEOMETRY-AWARE SPATIAL REASONING FOR AUDIO LARGE LANGUAGE MODELS Subrata Biswas, Mohammad Nur Hosssain Khan & Bashima Islam Department of Electrical & Computer Engineering Worcester Polytechnic Institute Worcester, MA 01609, USA {sbiswas, mkhan, [bislam](bislam}@wpi.edu)[}](bislam}@wpi.edu)[@wpi.edu](bislam}@wpi.edu) ABSTRACT Spatial reasoning is fundamental to auditory perception, yet current audio large language models (ALLMs) largely rely on unstructured binaural cues and singlestep inference. This limits both perceptual accuracy in direction and distance estimation and the capacity for interpretable reasoning. Recent work such as BAT demonstrates spatial QA with binaural audio, but its reliance on coarse categorical labels (left, right, up, down) and the absence of explicit geometric supervision constrain resolution and robustness. We introduce the Spatial-Acoustic Geometry Encoder (SAGE), a geometry-aware audio encoder that aligns binaural acoustic features with 3D spatial structure using panoramic depth images and simulated room-impulse responses at training time, while requiring only audio at inference. Building on this representation, we present OWL, an ALLM that integrates SAGE with a spatially grounded chain-of-thought to rationalize over direction-of-arrivals (DoA) and distance estimates. Through curriculum learning from perceptual QAto multi-step reasoning, OWL supports o’clock-level azimuth and DoA estimation. To enable large-scale training and evaluation, we construct and release BiDepth, a dataset of over one million QA pairs combining binaural audio with panoramic depth images and room impulse responses across both in-room and out-of-room scenarios. Across two benchmark datasets, our new BiDepth and the public SpatialSoundQA, OWL reduces mean DoA error by 11◦ through SAGE and improves spatial reasoning QA accuracy by up to 25% over BAT. 1 INTRODUCTION Large language models (LLMs) Achiam et al. (2023); Team et al. (2023); Touvron et al. (2023) have catalyzed rapid progress beyond pure text, inspiring multimodal systems Biswas et al. (2025) that pair an LLM backbone with modality encoders for vision Liu et al. (2023); Li et al. (2023), audio Gong et al. (2023); Ghosh et al. (2025); Goel et al. (2025), and other sensors Imran et al.(2024); Leng et al. (2024); Ouyang & Srivastava (2024) . Through projection layers for cross-modal alignment, these systems can process heterogeneous inputs, reason over joint representations, and follow instructions in context by training on paired (modality, text) data. Within audio, such models learn to align acoustic features with language to parse complex sound scenes, recognize events and speaker attributes, and carry out dialogue conditioned on what they “hear.” Early results show strong zero-shot generalization and retrieval capabilities when trained at scale, and instruction tuning further enables conversational audio-conditioned queries. Despite this momentum, audio-augmented multimodal LLMs lag behind their vision-language counterparts Liu et al. (2024); Bai et al. (2023); Li et al. (2024) due to the unique challenges of sound: long-range temporal dependencies, nonstationary noise, and the need to capture geometric cues of the acoustic environment that shape auditory perception. Even advanced models such as Gemini-2 .5-flash Comanici et al. (2025) struggle with composite acoustic tasks that require fine-grained spatial reasoning, highlighting persistent gaps in audio-language understanding. Recent advancements, such as BAT Zheng et al. (2024b), demonstrates spatial reasoning capabilities, but its localization remains coarse, subdividing the scene into only four broad regions (e.g., front-left, Figure 1: SAGE encodes binaural audio into spatially grounded representations. OWL detects events, localizes by direction and distance, and applies multi-step reasoning, yielding interpretable rationales for quer

FAQs

What problem does OWL address in current audio large language models?

Existing ALLMs rely on unstructured binaural cues and single-pass answers, limiting geometric grounding and interpretable multi-step reasoning for spatial tasks like direction and distance estimation.

How does SAGE enable geometry-aware spatial representations?

SAGE aligns binaural acoustic features with 3D spatial structure using panoramic depth images and simulated room impulse responses during training, while inference needs only binaural audio.

How is OWL different from prior work such as BAT?

OWL combines SAGE with a spatially grounded chain-of-thought that anchors intermediate reasoning to source locations, enabling finer localization and multi-step rationales beyond coarse categorical labeling.

Aug 29, 2026

Uploaded by Caleb Sterling