MelT A Portable Single-GEMM Mel Audio Frontend via Non-Uniform DFT with Measured Latency and Energy Gains on GPUs

Jul 16, 2026

Upload by Patrick

Modern neural audio inference increasingly runs on accelerators optimized for dense matrix multiplication, yet conventional acoustic frontends remain multi-stage pipelines built around STFT and sparse Mel aggregation, introducing overheads beyond the dominant dense kernels. MelT replaces this with a portable single-stage Mel frontend: precomputing Mel-spaced NDFT bases and applying them to time-domain frames using GEMM. Despite using more arithmetic than the FFT pipeline, MelT reduces latency 1.64×–3.29× and measured active energy up to 3.03× across devices, while preserving task utility and improving cepstral extensions like MFCCT.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages17

Reading Time43 min

Introduction

Background and motivation

Conventional frontend inefficiencies

MelT approach

Single-stage portable Mel frontend

Experimental design and evaluation

Efficiency metrics

Utility preservation

MelT: A Portable, Single-GEMM Mel Audio Frontend via Non-Uniform DFT with Measured Latency and Energy Gains on GPUs AUGUSTO CAMARGO 1 AND MARCELO FINGER1 1Instituto de Ciências Matemáticas e de Computação, University of São Paulo, São Paulo, Brazil ([e-mail: augustoc@usp.br](e-mail: augustoc@usp.br); [mfinger@ime.usp.br](mfinger@ime.usp.br)) Corresponding author: Augusto Camargo (e-mail: [augustoc@usp.br](augustoc@usp.br)). [ cs . SD] 13 Jul 2026 ABSTRACT Modern neural audio models run on accelerators whose peak throughput comes from dense matrix multiplication, increasingly at the edge and in datacenters. The conventional acoustic frontend, however—a Short-Time Fourier Transform (STFT) followed by sparse Mel aggregation—remains a multistage pipeline centered on the Fast Fourier Transform (FFT), with execution overheads unlike the dense linear algebra dominating the inference stack. This work introduces MelT, a portable single-stage Mel frontend that precomputes Mel-spaced Non-Uniform Discrete Fourier Transform (NDFT) bases and applies them to time-domain frames through General Matrix Multiplication (GEMM) . The contribution is a computational design principle: decoupling Mel feature extraction from vendor-specific FFT primitives and lowering it onto the matrix-multiplication substrate accelerators already optimize. It is not a new spectral operator. MelT’s direct projection performs more arithmetic than the FFT pipeline. Yet in the compact-resolution regime of neural audio frontends, it achieves a 1.64-times to 3.29-times latency reduction and up to a 3.03-times reduction in measured active energy, from the Apple A18 Pro to the NVIDIA H100 . All gains are withinplatform comparisons, accompanied by task-level validation. Word error rate stays statistically equivalent to the native frontend’s on frozen Whisper models of medium size and larger; speaker-attribute classification on VoxCeleb1 is non-inferior. The cepstral extension MFCCT preserves utility on a clinical respiratoryinsufficiency classification task (SPIRA) while improving on the MFCC baseline. These results indicate that, in practical regimes on the accelerators evaluated here, hardware alignment rather than arithmetic count can govern the realized cost of feature extraction. INDEX TERMS Audio frontends, discrete Fourier transforms, energy efficiency, feature extraction, general matrix multiplication (GEMM), graphics processing units (GPUs), Mel frequency cepstral coefficients, nonuniform discrete Fourier transform (NDFT) I. INTRODUCTION LARGE-SCALE audio systems—including speech rec ognizers [1], audio classifiers [2], [3], and selfsupervised models [4]—are increasingly deployed on hardware whose peak throughput is derived from dense matrix multiplication. Modern accelerators, from mobile GPUs to datacenter-scale systems, devote substantial silicon resources to matrix engines such as Tensor Cores [5] and specialized matrix coprocessors [6] . As a consequence, many of the dominant workloads in contemporary machine learning are to applications [9]; Google AI Edge provides an end-to-end stack for on-device generative and multimodal models [10]; Microsoft’s Foundry Local targets local AI applications and agents [11]; and NVIDIA supports edge inference through its Jetson platforms [12] . These efforts are driven by demands for privacy, responsiveness, offline operation, and infrastructure efficiency. As inference spans ever more heterogeneous hardware, latency, energy, and hardware efficiency matter across the entire stack, including the signal-processing frontends that precede neural inference. arXiv :2606 .01009v2 now executed primarily as dense matrix multiplication. This architectural shift coincides with a change in where inference runs. Foundation models and other AI workloads increasingly run on local and edge hardware—smartphones, personal computers, embedded systems, and acceleratorequipped devices—rather than exclusively in centralized

FAQs

What problem does MelT target in modern neural audio inference pipelines?

It targets the mismatch between accelerator-friendly dense matrix workloads and conventional frontends built on STFT plus sparse Mel aggregation, which add structural overheads such as extra memory traffic and kernel dispatch costs.

How does MelT compute Mel features in a single-stage manner?

MelT precomputes Mel-spaced Non-Uniform Discrete Fourier Transform (NDFT) bases, then projects time-domain frames using General Matrix Multiplication (GEMM) to obtain Mel features.

What efficiency and utility results are reported for MelT?

MelT achieves 1.64×–3.29× latency reduction and up to 3.03× measured active energy reduction on evaluated GPU platforms, while task outputs remain statistically equivalent or non-inferior under frozen downstream Whisper and classification models.

Jul 16, 2026

Upload by Patrick