[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-121583-en":3,"doc-seo-121583-105":30,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":4,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":27,"seo_description":14,"update_tm":28,"read_time":29},121583,13056703020460,"Valentina","https://ap-avatar.wpscdn.com/avatar/be000253dac470eee5d?_k=1778207105932848923",8,"Research & Report","Sonicolour - Exploring Colour Control of Sound Synthesis with Interactive Machine Learning","This paper explores crossmodal mappings of colour to sound through a custom instrument that analyses physical objects’ colour using a colour light-to-frequency sensor and translates red, green, and blue values into parameters of an additive synthesiser. Interactive machine learning supports discovering new, unexpected relationships between visual object features and sound synthesis outcomes. Evaluation focuses on enabling playful, interactive exploration combining visual and tactile inspection of coloured objects with the generation of synthetic sounds. The work closes by considering implications for musical interaction design and music performance.","Sonicolour: Exploring Colour Control of Sound Synthesis with Interactive Machine Learning  \nTug F. O’Flaherty  \nCentre for Digital Music Queen Mary University of London London, UK [t.f.oflaherty@se24.qmul.ac.uk](t.f.oflaherty@se24.qmul.ac.uk)  \nCharalampos Saitis  \nCentre for Digital Music Queen Mary University of London London, UK [c.saitis@qmul.ac.uk](c.saitis@qmul.ac.uk)  \nLuigi Marino  \nCentre for Digital Music Queen Mary University of London London, UK [l.marino@qmul.ac.uk](l.marino@qmul.ac.uk)  \nAnna Xambó Sedó  \nCentre for Digital Music Queen Mary University of London London, UK[a.xambosedo@qmul.ac.uk](a.xambosedo@qmul.ac.uk)  \nAbstract  \nThis paper explores crossmodal mappings of colour to sound. The instrument presented analyses the colour of physical objects via a colour light-to-frequency sensor and maps the corresponding red, green, and blue data values to parameters of a synthesiser. Interactive machine learning is used to facilitate the discovery of new relationships between sound and colour. The role ofinteractive machine learning is to find unexpected relationships between the visual features of the objects and the sound synthesis. The performance is evaluated by its ability to provide the user with a playful interaction between the visual and tactile exploration of coloured objects, and the generation of synthetic sounds. We conclude by outlining the potential of this approach for musical interaction design and music performance.  \nKeywords  \nSupervised learning, crossmodal mapping, additive synthesis, synaesthesia  \n1 Introduction  \nGenerating audio through colour-to-sound mapping is a developing area in which many devices have been produced, including visual impairment aids and art installations [10] . Previous research commonly uses pitch height as a basis for colour and sound associations [33]. In contrast, whilst many sound-to-colour mapping methods have been proposed, most involve audio analysis of spectral features, such as the spectral centroid [32] . Studies have shown an association between soft timbres and blue, green, or light greyscales, and harsh timbres with red, yellow, or black [1] . Nonetheless, fewer research exists on creating performable instruments capable of facilitating the desired colour-to-sound associations of instrumentalists and composers.  \nSound-to-colour mappings may be presented through a relatively common cognitive phenomenon known as crossmodal correspondences, where associations between music, and notably timbres, are often made with specific colours [16] . A rare medical form of such colour and sound mappings, with little research and creative potential, is known as audiovisual synaesthesia, for  \nThis work is licensed under a Creative Commons Attribution 4.0 International License.  \nNIME’25, June 24–27, 2025, Canberra, Australia © 2025 Copyright held by the owner/author(s) .  \nwhich mappings can be consistent and memorable to the listener [19] . Audiovisual synaesthetes may precisely and idiosyncratically associate particular sounds with distinct shapes, colours, brightness, and spatial location [14, 17] .  \nThere exists a long history of music and colour associations, although commonly related to tonality, perceived by famous composers including Messiaen and Rimsky-Korsakov [16, 27] . Such literature suggests crossmodal correspondences explored through an instrument, associating performers’ perceptions of visual colour and sound colour (i.e., timbre), would offer performative benefits, for instance, facilitating expressivity in musical interaction through action-perception loops, subtlety and even creative ambiguity [8] . Inspired by NIME infra-instrument aesthetics [6], this paper seeks to explore the performability and creative freedom afforded by mapping colour and brightness to sound, through the creation of a custom instrument, utilising supervised machine learning and additive synthesis. A focus is placed on answering the following research question: what can colour","cbCaivTuTJU5h0Cs","https://ap.wps.com/l/cbCaivTuTJU5h0Cs","pdf",1597072,1,6,"English","en",105,"# 1 Introduction\n## Crossmodal correspondences and synaesthesia\n## Research question and instrument motivation\n# 2 Related Work\n## 2.1 Crossmodal associations between sound and colour","[{\"question\":\"How does the Sonicolour instrument map colour to sound?\",\"answer\":\"It uses a colour light-to-frequency sensor to analyze object colour and maps red, green, and blue values to parameters of a synthesiser for generating timbre and sound.\"},{\"question\":\"What role does interactive machine learning play in this work?\",\"answer\":\"It helps find unexpected relationships between visual features of objects and the resulting sound synthesis, enabling the discovery of new mappings.\"},{\"question\":\"Why is additive synthesis used here?\",\"answer\":\"Additive synthesis provides control over the harmonics that form timbre, supporting exploration of colour-driven sound synthesis from varied harmonic content.\"}]","Sonicolour - Exploring Colour Control of Sound Synthesis with Interactive Machine Learning | PDF",1785736355,15,{"code":4,"msg":31,"data":32},"ok",{"site_id":24,"language":23,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":28},"sonicolour-exploring-colour-control-of-sound-synthesis-with-interactive-machine-learning","",{"@graph":36,"@context":85},[37,54,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":53},"https://docshare.wps.com/document/sonicolour-exploring-colour-control-of-sound-synthesis-with-interactive-machine-learning/121583/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":23,"description":14,"dateModified":62,"datePublished":62,"encodingFormat":61,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":41,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-03",true,{"@type":65,"interactionType":66,"userInteractionCount":4},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"How does the Sonicolour instrument map colour to sound?","Question",{"text":75,"@type":76},"It uses a colour light-to-frequency sensor to analyze object colour and maps red, green, and blue values to parameters of a synthesiser for generating timbre and sound.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"What role does interactive machine learning play in this work?",{"text":80,"@type":76},"It helps find unexpected relationships between visual features of objects and the resulting sound synthesis, enabling the discovery of new mappings.",{"name":82,"@type":73,"acceptedAnswer":83},"Why is additive synthesis used here?",{"text":84,"@type":76},"Additive synthesis provides control over the harmonics that form timbre, supporting exploration of colour-driven sound synthesis from varied harmonic content.","https://schema.org",{"og:url":52,"og:type":87,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":89,"canonical":52},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,114,119,122,127,130,134],{"id":20,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":46,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":21,"doc_module":4,"doc_module_name":46,"category_name":111,"show_sort_weight":112,"slug":113},"Technology",50,"technology",{"id":115,"doc_module":4,"doc_module_name":46,"category_name":116,"show_sort_weight":117,"slug":118},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":120,"slug":121},30,"research-report",{"id":123,"doc_module":4,"doc_module_name":46,"category_name":124,"show_sort_weight":125,"slug":126},9,"Religion & Spirituality",20,"religion-spirituality",{"id":125,"doc_module":4,"doc_module_name":46,"category_name":128,"show_sort_weight":125,"slug":129},"World Cup","world-cup",{"id":131,"doc_module":4,"doc_module_name":46,"category_name":132,"show_sort_weight":131,"slug":133},10,"Lifestyle","lifestyle",{"id":135,"doc_module":4,"doc_module_name":46,"category_name":136,"show_sort_weight":106,"slug":137},19,"General","general"]