INCREASING DRUM TRANSCRIPTION VOCABULARY USING DATA SYNTHESIS - Proceedings of the 21st International Conference on Digital Audio Effects (DAFx-18)

Sep 29, 2026

Automatic drum transcription relies on annotated onset events, yet current datasets are small and often cover only a few percussion classes because extended vocabularies appear rarely. This scarcity makes it hard to train models for large vocabularies. The paper proposes large-vocabulary ADT by generating a synthetic set of 210,000 eight-second audio examples with ground-truth transcriptions. Training convolutional-recurrent networks in a multi-task framework with synthetic and real datasets improves large-vocabulary performance and also benefits beat/downbeat detection for small-vocabulary settings.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages8

Reading Time20 min

AccessFree to download

Abstract

1. Introduction

2. Related Work

Proceedings of the 21st International Conference on Digital Audio Effects (DAFx-18), Aveiro, Portugal, September 4–8, 2018 INCREASING DRUM TRANSCRIPTION VOCABULARY USING DATA SYNTHESIS Mark Cartwright Music and Audio Research Lab New York University New York, New York [mark.cartwright@nyu.edu](mark.cartwright@nyu.edu) Juan Pablo Bello Music and Audio Research Lab New York University New York, New York [jpbello@nyu.edu](jpbello@nyu.edu) ABSTRACT Current datasets for automatic drum transcription (ADT) are small and limited due to the tedious task of annotating onset events. While some of these datasets contain large vocabularies of percussive instrument classes (e.g. ~20 classes), many of these classes occur very infrequently in the data. This paucity of data makes it difﬁcult to train models that support such large vocabularies. Therefore, data-driven drum transcription models often focus on a small number of percussive instrument classes (e.g. 3 classes) . In this paper, we propose to support large-vocabulary drum transcription by generating a large synthetic dataset (210,000 eight second examples) of audio examples for which we have groundtruth transcriptions. Using this synthetic dataset along with existing drum transcription datasets, we train convolutional-recurrent neural networks (CRNNs) in a multi-task framework to support large-vocabulary ADT. We ﬁnd that training on both the synthetic and real music drum transcription datasets together improves performance on not only large-vocabulary ADT, but also beat / downbeat detection small-vocabulary ADT. 1. INTRODUCTION Automatic Drum Transcription (ADT) is the task of creating a symbolic score of the percussion instrument events within an audio recording of a musical piece. It is a subtask within Automatic Music Transcription (AMT), which aims to create a symbolic score of all the events within a musical piece. With accurate AMT, tens of millions of musical recordings could be indexed, compared, recommended, mined, and studied at scale using familiar musical concepts like pitch, harmony, rhythm, meter, and tempo. Accurate ADT could also aid in the development of generative rhythm models, descriptive models of rhythmic style, and intelligent digital audio effects that are informed by transcription and style. ADT researchers typically simplify this problem by focusing solely on detecting the onset time and instrument class of all the notes sounded by percussion instruments in the signal. And very often, they simplify it even further by limiting the problem to only the notes sounded by the bass drum (BD), snare drum (SD), and hihat (HH) [1, 2, 3, 4, 5, 6]—a limit that greatly decreasing the utility of these systems. This is due to the tedious and time consuming nature of annotating recordings, which results in ADT datasets that are small and limited, often consisting of just a couple of hours of audio [1, 7, 8, 9] . In addition, while some datasets have annotations of a large number of percussion instrument classes [8, 7], others are limited to the BD, SD, and HH classes [9, 1], and datasets that do have larger vocabularies have minimal examples of these extended classes. The recent successful ADT algorithms have used deep learning architectures incorporating forms of recurrent neural networks [5, 4, 1, 2] . While powerful, these models require training on many audio examples in order to generalize well, making it even more difﬁcult to expand the vocabulary of ADT. Our goal is to learn ADT models that support a large vocabulary of percussion sounds. To address this, we generated a synthetic dataset that is 126 times larger than four of the most popular drum transcription datasets combined. We constructed this dataset using a large collection of MIDI drum loops, a large collection of drum hit recordings, and non-rhythmic harmonic backgrounds. We then trained and evaluated a multi-task convolutional-recurrent neural network (CRNN) drum transcription model u

FAQs

Why is large-vocabulary automatic drum transcription difficult with current datasets?

Large vocabularies are hard to support because annotated datasets are small and onset annotation is time-consuming. Extended percussion classes often occur very infrequently, limiting model training coverage.

What synthetic data does the paper introduce for training?

The authors generate a synthetic dataset of 210,000 eight-second audio examples with ground-truth transcriptions, created from MIDI drum loops, drum hit recordings, and non-rhythmic harmonic backgrounds.

How is the model trained to handle multiple objectives?

The paper trains convolutional-recurrent neural networks in a multi-task framework using both the synthetic dataset and existing real drum transcription datasets.

Sep 29, 2026

Upload by Theodore