Weakly Guided and Autoregressive Beamformer Parameterization for Generalizable Moving Speaker Extraction in Higher-Order Ambisonics

Jul 16, 2026

Upload by Hazel

Linear spatial filters (beamformers) support robust, generalizable and interpretable speech enhancement when parameterization matches ideal assumptions. Modern systems often rely on deep neural parameterization, but performance degrades when multiple moving speakers with unknown directions are present. This work presents a weakly guided, data-driven beamforming pipeline using only an estimate of the target’s initial DoA. A higher-order ambisonics framework decouples neural temporal-spectral processing from linear spatial filtering, enabling array-agnostic generalization with autoregression for consistent results during fast motion and long recordings.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages5

Reading Time13 min

Abstract

Introduction

Speech enhancement and target speaker extraction

Spatial guidance, weak guidance, and tracking

Mask-based beamformer parameterization

Causal processing and autoregressive context

WEAKLY GUIDED AND AUTOREGRESSIVE BEAMFORMER PARAMETERIZATION FOR GENERALIZABLE MOVING SPEAKER EXTRACTION IN HIGHER-ORDER AMBISONICS Jakob Kienegger, Tal Peer, Sina Khanagha, Timo Gerkmann Signal Processing (SP) Group, University of Hamburg arXiv :2607 .04471v1 [ ee ss .AS] 5 Jul 2026 ABSTRACT Linear spatial filters (beamformers) enable robust, generalizable and interpretable speech enhancement with performance guarantees under ideal parameterization. Modern beamformers are often parameterized by deep neural networks, whose performance degrades in dynamic scenarios with multiple moving speakers of unknown directions. We propose a data-driven beamforming pipeline, which only requires an estimate of the target’s initial direction. Building on a higher-order ambisonics representation, we show that neural temporal-spectral processing can be decoupled from linear spatial processing, and thereby achieve generalizable and array-agnostic enhancement. By incorporating autoregression into a frame-wise causal framework, we maintain consistent performance throughout fast speaker motion and long recordings. Evaluation on synthetic data demonstrates robust enhancement under challenging conditions with closely spaced and crossing speakers. Real-world recordings in a dynamic office meeting scenario complement these findings and show generalizability across varying ambisonics orders. Index Terms— Ambisonics, autoregressive, moving speaker, multi-channel speech enhancement, mask-based beamforming. 1. INTRODUCTION Speech enhancement aims to improve the perceptual quality and intelligibility of a recorded speech signal by suppressing noise and reverberation. In a multi-speaker environment, target speaker extraction (TSE) solves the additional challenge of differentiating between the desired target speech and remaining speakers, which are to be treated as noise. When a recording from a microphone array is available, the relative direction of the target to the array, known as direction of arrival (DoA), can be employed to identify the desired speaker. In case of a spherical array, ambisonics [1] provides a directional and array-agnostic representation of the recorded sound field, making it a popular audio format for enhancement [2–5] . Assuming the target’s DoA is known, a spatial filter can be steered toward the desired direction and extract the corresponding speech signal. By jointly processing temporal-spectral and spatial information, recently proposed deep, non-linear spatial filters achieve outstanding enhancement performance [6, 7] . While linear spatial filters, known as beamformers, are proven to be inferior under realistic, non-Gaussian statistical assumptions [8], they provide robustness and interpretability by being a parametric, statistics-based approach. When combined with deep neural networks (DNNs) to This work was supported by the Deutsche Forschungsgemeinschaft (DFG) under Grant 508337379 . Computational resources were provided by the Regional Computer Center (RRZ) of the University of Hamburg and the Erlangen National High Performance Computing Center (NHR@FAU) under Project f104ac. NHR is funded by the Federal Government and Bavaria. Both facilities received DFG funding under Grants 440719683 and 498394658 . estimate these statistics, such hybrid approaches can achieve strong enhancement performance even with linear spatial filters [9, 10] . In stationary scenarios, the target’s DoA may be available a priori. However, when speaker motion becomes non-negligible, the continuous directional information necessary to steer a spatial filter—which we refer to as strong guidance—is typically not available. Weakly guided speaker extraction relaxes this constraint and only assumes knowledge of the target’s direction at recording start. To continue using a spatial filter for extraction, an additional tracking algorithm is typically required to infer the target’s movement from the starting direction and automate the s

FAQs

What problem does the proposed method address in multi-speaker environments?

It targets moving target speaker extraction when multiple speakers are active and their directions can change over time, making guidance and alignment difficult.

What information does the method require about the target’s direction?

It only needs an estimate of the target’s initial direction of arrival at the start of the recording, using weak guidance rather than full trajectory knowledge.

How does the approach maintain performance during fast speaker motion and long recordings?

It incorporates autoregression into a frame-wise causal framework so the method can exploit temporal context consistently across the recording.

Jul 16, 2026

Upload by Hazel