Hardware-Software Co-Design of an Audio Feature Extraction Pipeline for Machine Learning Applications - Article

Aug 5, 2026

Upload by Maeve

Keyword spotting is a critical component of modern speech recognition pipelines, but conventional systems often rely on MFCC audio features that are computationally demanding. This work investigates simplifying MFCC features to better support always-on, resource-constrained embedded keyword spotting. A hardware generator is implemented to produce a matching hardware pipeline for the simplified extraction. Using the Chisel4ml framework, hardware generators are integrated into the Python Keras workflow, enabling model training with the proposed features.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages14

Reading Time35 min

AccessFree to download

Introduction

Simplified MFCC for energy-efficient keyword spotting

Proposed contributions and paper structure

MFCC standard calculation and studied simplifications

Hardware modules for simplified MFCC features

Chisel4ml framework and Python-compiler integration

Accuracy impact of MFCC simplifications in KWS

Hardware synthesis results

electronics Article Hardware–Software Co-Design of an Audio Feature Extraction Pipeline for Machine Learning Applications Jure Vreˇca 1,2, *, Ratko Pilipovi´c 3 and Anton Biasizzo 1 Citation: Vreˇca, J.; Pilipovi´c, R.; Biasizzo, A. Hardware–Software Co-Design of an Audio Feature Extraction Pipeline for Machine Learning Applications. Electronics 2024, 13, 875. [https://doi.org/](https://doi.org/)[ ](https://doi.org/)[10.3390/electronics13050875](10.3390/electronics13050875) Academic Editor: Chunping Li Received: 31 January 2024 Revised: 17 February 2024 Accepted: 22 February 2024 Published: 24 February 2024 Copyright: © 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license ([https://](https://)[ ](https://)[creativecommons.org/licenses/by/](creativecommons.org/licenses/by/)[ ](creativecommons.org/licenses/by/)[4.0/](4.0/)) . 1 Jožef Stefan Institute, 1000 Ljubljana, Slovenia; anton.biasizzo@ijs.si 2 Jožef Stefan International Postgraduate School (IPS), 1000 Ljubljana, Slovenia 3 Faculty of Computer and Information Science, University of Ljubljana, 1000 Ljubljana, Slovenia; [ratko.pilipovic@fri.uni-lj.si](ratko.pilipovic@fri.uni-lj.si) * [Correspondence: jure.vreca@ijs.si](Correspondence: jure.vreca@ijs.si) Abstract: Keyword spotting is an important part of modern speech recognition pipelines. Typical contemporary keyword-spotting systems are based on Mel-Frequency Cepstral Coefficient (MFCC) audio features, which are relatively complex to compute. Considering the always-on nature of many keyword-spotting systems, it is prudent to optimize this part of the detection pipeline. We explore the simplifications of the MFCC audio features and derive a simplified version that can be more easily used in embedded applications. Additionally, we implement a hardware generator that generates an appropriate hardware pipeline for the simplified audio feature extraction. Using Chisel4ml framework, we integrate hardware generators into Python-based Keras framework, which facilitates the training process of the machine learning models using our simplified audio features. Keywords: FPGA; MFCC; keyword spotting; chisel 1. Introduction The development of deep neural networks has opened up possibilities for applications in diverse areas. One such area is speech recognition, where researchers have been able to show promising results using large transformer-based neural networks [1] . These networks are, however, very computationally expensive and thus are hard to implement on battery-powered devices. This can be overcome using a simpler keyword detection system that merely listens for specific keywords (e.g., Hey Siri) and then wakes up a more powerful system generally implemented in the cloud. In this way, the simpler low-power system acts as an always-on listener, and the more powerful system is used only when needed. Henceforth, we will refer to the first kind of system as the keyword-spotting (KWS) system [2], and the second type as the Large Vocabulary Speech Recognition system. The keyword-spotting system is an always-on system. This requirement facilitates the need for it to be as energy-efficient as possible. The authors of [3] recognized this problem and explored an integer-only implementation of the MFCC algorithm. They achieved good results. However, they refrained from modifying the MFCC algorithm. Furthermore, they targeted a DSP processing unit. Dedicated hardware circuits could make this process even more energy-efficient. The authors of [4] developed a custom MFCC extraction processing unit. However, they also failed to explore simplifications of the MFCC algorithm and instead focused exclusively on the hardware implementation. The authors of [5] showcased a system that simplifies the MFCC features to a certain degree. While they did obtain satisfactory classifi

FAQs

Why are MFCC features targeted for optimization in keyword spotting systems?

Keyword-spotting systems are always-on and must be energy-efficient. MFCC-based systems compute relatively complex features, making optimization beneficial for embedded deployment.

What does the paper contribute regarding MFCC audio features?

The paper formulates a simplified version of MFCC features and analyzes how these simplifications affect keyword-spotting (KWS) performance accuracy.

How are hardware generators used in this work?

A hardware generator is implemented to generate a hardware pipeline that computes the simplified audio feature extraction efficiently, focusing on resource efficiency.

How does Chisel4ml connect the hardware pipeline with machine learning training?

Chisel4ml integrates hardware generators into a Python-based Keras workflow, allowing training of machine learning models using the simplified audio features.

Aug 5, 2026

Upload by Maeve