PROBING LOW-LEVEL ACOUSTIC ATTRIBUTE ENCODING IN CLAP AUDIO EMBEDDINGS

Sep 21, 2026

Uploaded by Asher

Audio foundation models are widely adopted as general-purpose feature extractors, yet the internal structure of their learned representations remains insufficiently understood. This work probes CLAP audio embeddings to study how three perceptual dimensions—reverberation (RT60), loudness (LUFS), and spectral content (SC, via spectral centroid and relative pitch)—are encoded. Probes of increasing complexity predict attributes from frozen embeddings across five datasets spanning noise, speech, monophonic notes, and music mixtures.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages8

Reading Time20 min

AccessFree to download

Abstract

Introduction

Methodology

Target Acoustic Attributes

Proceedings of the 29th International Conference on Digital Audio Effects (DAFx26), Cambridge, MA, USA, 1 –4 September 2026 PROBING LOW-LEVEL ACOUSTIC ATTRIBUTE ENCODING IN CLAP AUDIO EMBEDDINGS Héctor Martel, Joe Hennessy-Priest and Taemin Cho BandLab Technologies, Singapore [hector.martel@bandlab.com](hector.martel@bandlab.com) arXiv :2607 .03806v1 [ ee ss .AS] 4 Jul 2026 ABSTRACT Audio foundation models are widely adopted as general-purpose feature extractors, yet the internal structure of their learned representations remains insufficiently understood. In this work, we analyze CLAP audio embeddings through a probing framework, studying the encoding of three fundamental perceptual dimensions: reverberation (RT60), loudness (LUFS), and spectral content, measured via spectral centroid (SC) and relative pitch (RP) . Probes of increasing complexity are trained to predict each attribute from frozen embeddings across five datasets spanning noise, speech, monophonic musical notes, and music mixtures. Our primary finding is that all of these attributes are reliably recoverable from the CLAP embedding space across the examined datasets. Within this global picture, two encoding regimes emerge: RT60, LUFS, and RP are approximately linearly encoded, while SC requires nonlinear probes. Both regimes generalize across eight additional audio foundation models, with the notable exception that amplitudeinvariant architectures discard loudness entirely by construction. The identified linear feature directions are geometrically consistent across datasets for RT60 and LUFS, while highly domainspecific for RP. Finally, we provide a qualitative demonstration of cross-modal consistency, showing that text embeddings of acoustic descriptors align geometrically with the identified RT60 feature direction. 1. INTRODUCTION CLAP [1, 2] is a prominent audio-language foundation model that leverages contrastive learning to align audio and text in a shared embedding space. Trained on large collections of audio-text pairs, CLAP learns embeddings that capture both acoustic characteristics and semantic content, enabling applications such as audio retrieval, captioning [3, 4], audio quality assessment [5, 6], effect parameter estimation [7], room impulse response synthesis [8], and conditional audio generation [9, 10, 11] . The model has since been extended to incorporate temporal [12] and stereo [13] information. Despite this widespread adoption, the internal representation structure of CLAP remains insufficiently understood. Probing studies of audio-language models have largely examined specific domains or model classes: early work probed speech embeddings for speaker and phonetic information [14], while more recent analyses of Large Audio-Language Models (LALMs), a model class that does not encompass CLAP, probe high-level perceptual attributes such as language, gender, and emotion of speech [15, 16] . Copyright: © 2026 Héctor Martel et al. This is an open-access article distributed under the terms of the Creative Commons Attribution 4 .0 International License, which permits unrestricted use, distribution, adaptation, and reproduction in any medium, provided the original author and source are credited. Figure 1: Overview of the probing methodology. Interpretability approaches for acoustic models have favored qualitative concept alignment [17] over quantitative prediction of physical descriptors. Within the CLAP literature, studies have shown that high-level features such as timbre [18] and emotion [19] are effectively captured. However, low-level acoustic attributes have received far less attention. [20] shows that applying audio effects at varying intensities traces consistent trajectories in the CLAP embedding space. The authors exploit this property to improve embedding robustness. Conversely, FXEncoder++ [21] learns representations that capture audio effects independently of content. Text2FX [7] uses CLAP embeddings for audio e

FAQs

What does the probing study investigate in CLAP embeddings?

It analyzes how CLAP audio embeddings encode low-level acoustic attributes, focusing on reverberation (RT60), loudness (LUFS), and spectral content.

How are the acoustic attributes predicted during probing?

Regression probes are trained to predict scalar ground-truth attribute values from frozen CLAP embeddings across multiple datasets.

What are the main findings about recoverability and encoding regimes?

The attributes are reliably recoverable from the CLAP embedding space: RT60, LUFS, and relative pitch are approximately linearly encoded, while spectral centroid needs nonlinear probes.

Sep 21, 2026

Uploaded by Asher