EMOTION MANIPULATION THROUGH MUSIC - A DEEP LEARNING INTERACTIVE VISUAL APPROACH

Oct 12, 2026

Upload by Sarah

Music evokes emotion in many people, and this work introduces an AI-driven method to manipulate the emotional content of a song. The research targets achieving a desired emotion while preserving the original melody as much as possible. It proposes an interactive pipeline that shifts an input track toward a diametrically opposed emotion and visualizes outcomes using Russel’s Circumplex model. A deep learning model evaluates modification accuracy across key, SoundFont instrumentation, and other musical features, aligning with state-of-the-art results on the 4Q Emotion dataset. Further refinement supports on-demand custom music generation, automated remixing, and emotion-progressed playlists.

LanguageEnglish

CategoryPresentations

FormatPDF

Pages9

Reading Time3 min

AccessFree to download

Abstract

Introduction

Previous Work

Emotion analysis using deep learning

EMOTION MANIPULATION THROUGH MUSIC-A DEEP LEARNING INTERACTIVE VISUAL APPROACH arXiv :2406 .08623v1 [ cs . SD] 12 Jun 2024 Adel N. Abdalla Data Science Institute Columbia University, New York, NY, USA [ana2169@columbia.edu](ana2169@columbia.edu) Jared Osborne Computer Science Department Central Washington University, Ellensburg, WA, USA [osbornejar@cwu.edu](osbornejar@cwu.edu) Rzvan Andonie Computer Science Department Central Washington University, Ellensburg, WA, USA and Transilvania University of Bra¸sov, Romania [razvan.andonie@cwu.edu](razvan.andonie@cwu.edu) June 14, 2024 ABSTRACT Music evokes emotion in many people. We introduce a novel way to manipulate the emotional content of a song using AI tools. Our goal is to achieve the desired emotion while leaving the original melody as intact as possible. For this, we create an interactive pipeline capable of shifting an input song into a diametrically opposed emotion and visualize this result through Russel’s Circumplex model. Our approach is a proof-of-concept for Semantic Manipulation of Music, a novel field aimed at modifying the emotional content of existing music. We design a deep learning model able to assess the accuracy of our modifications to key, SoundFont instrumentation, and other musical features. The accuracy of our model is in-line with the current state of the art techniques on the 4Q Emotion dataset. With further refinement, this research may contribute to on-demand custom music generation, the automated remixing of existing work, and music playlists tuned for emotional progression. Keywords Music Emotion Recognition · Music Information Retrieval · Deep Learning · Semantic Manipulation · Audio Analysis · Explainable AI 1 Introduction In recent years, the fields of Music Information Retrieval (MIR) and Music Emotion Recognition (MER) have received significant attention, leading to multiple advances in how music is analyzed [1, 2] . These developments have increased the accuracy in determining what emotions are present in a given music sample, but the current state of the art is only now passing 75% through the use of Random Forest and Support Vector Machine models [3] . This is in contrast to the field of speech recognition, where current models are approaching 100% accuracy across hundreds of languages for word identification [4] and 85% for standard speech emotion recognition [5] . The additional challenges in music recognition come from the nature of music itself as the lyrical and emotional content of a vocalist’s contribution are only one part of the whole. Tempo, rhythm, timbre, instrumentation choice, perceived genre, and other factors contribute together to shape the emotional and tonal landscape of any given work into a unique blend that is interpreted subjectively by individual listeners [6] . The goal of our paper is to show that by changing the underlying structure of a small subset of musical features of any given musical piece, we can adjust the perceived emotional content of the work towards a specific desired emotion. A PREPRINT-JUNE 14, 2024 This appears to be a novel approach, as current research into MIR and MER is focused on improving the accuracy of existing emotional content identification rather than changing it to a new emotion. Our approach fits into the visual and deep learning framework. In other words, transformations are applied to the audio, a deep learning model is applied, and visuals are produced. This makes it very different than the interactive control system for emotional expression in music presented in [7], which allows users to make changes to both structural and expressive cues (tempo, pitch, dynamics, articulation, brightness, and mode) of music in real-time. The contributions in this paper are threefold: 1. We create a pipeline capable of shifting an input song into a diametrically opposed emotion and visualize this result through Russel’s Circumplex model [8] . 2. We g

FAQs

What is the paper’s main goal for emotion manipulation in music?

It aims to shift a song into a desired emotion while keeping the original melody as intact as possible, using AI tools and an interactive pipeline.

How does the approach visualize the emotional change?

It visualizes results through Russel’s Circumplex model, using a 2D representation of arousal and valence.

What aspects of the music does the deep learning model evaluate for modification accuracy?

It assesses accuracy across key, SoundFont instrumentation, and other musical features.

Oct 12, 2026

Upload by Sarah