[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-84544-en":3,"doc-seo-84544-105":29,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":13,"seo_description":14,"update_tm":27,"read_time":28},84544,34359740700684,"Finn","https://ap-avatar.wpscdn.com/avatar/1f400023980c374ae676?_k=1777273430885731487",6,"Technology","A Text-Steerable Instrument for Sketching Procedural Soundscapes via Language Models","A real-time musical interface converts natural-language scene descriptions into evolving procedural soundscapes. Performers type prompts (e.g., “warm jazz café at midnight”) and steer the sound through direct, named parameter edits such as brightness and rhythm without re-prompting. Unlike GPU-bound text-to-audio methods that output monolithic waveforms, the system emits human-readable configurations over a categorical schema. Three backends share the same schema, while a live generator continuously streams audio and resolves updates in the background to hide latency. Semantic alignment is evaluated with LAION-CLAP and reinforced by retrieval-based configuration design, alongside performance observations and open release materials for the SDK, dataset artifacts, and interface.","A Text-Steerable Instrument for Sketching Procedural Soundscapes via Language Models  \nPrabal Gupta  \nRama Labs  \nKitchener, Ontario, Canada  \n[prabal@rjeinc.ca](prabal@rjeinc.ca)  \narXiv :2607 .00309v 1 [ cs . SD] 1 Jul 2026  \nAbstract  \nWe present a real-time musical interface that converts naturallanguage scene descriptions into evolving procedural soundscapes. A performer types a prompt such as “warm jazz café at midnight” and steers it through direct parameter adjustments—stepping brightness down, switching a rhythm style—each producing a predictable, audible shift without re-prompting. Where GPU-bound text-to-audio systems synthesize monolithic waveforms, our instrument generates human-readable configurations over a categorical schema, enabling fine-grained performer control; most valid combinations are designed to sound musically coherent. Three interchangeable backends—embedding retrieval for sub-second CPU-only use, hosted LLMs via API, and a finetuned 270M local model—all emit the same schema. A live generator architecture continuously emits audio while resolving new instructions in the background, crossfading seamlessly when ready; even when an LLM takes 5–12 seconds to respond, the audience hears uninterrupted sound—reframing text-to-music as an ongoing performable stream rather than a one-shot generation. We evaluate text–audio semantic alignment using LAION-CLAP on held-out prompts as a technical proxy, finding that retrievalbased configuration outperforms random valid configurationson this metric, while noting that LAION-CLAP also informed retrieval-map construction. We report performance observations, informal listener feedback, and release materials for the SDK, dataset artifacts, model, and audiovisual performance interface.  \nKeywords  \ntext-to-music, procedural synthesis, real-time performance, language models, parametric control, live coding  \n1 Introduction  \nIn a live coding set, the performer types \"neon city at 2am\"and the ambient drone keeps playing. Five seconds later, the new configuration resolves—bright, electronic, fast—and the sound crossfades into it. The performer nudges brightness down two steps. The city gets darker. They switch rhythm to \"heartbeat\". The pulse slows. None of this required stopping the music, waiting for a GPU, or understanding DSP.  \nNeural text-to-audio systems such as MusicGen [6], MusicLM [1], and Stable Audio [7] produce high-quality audio from text. However, such systems typically return generated audio rather than interpretable synthesizer parameters; CTAG explicitly frames this as a tweakability problem for neural audio systems [4] . Azimi and Zareei [2] fine-tuned Stable Audio Open  \nThis work is licensed under a Creative Commons Attribution 4.0 International License.  \nNIME’26, June 23–26, 2026, London, UK  \n© 2026 Copyright held by the owner/author(s) .  \nfor live improvisation, demonstrating that even with GPU acceleration, generation latency and unpredictable outputs remain challenges.  \nOur SDK addresses this tension by generating interpretable parameter configurations for a procedural synthesizer rather than raw audio. Its live generator architecture decouples instruction resolution from audio emission—the current configuration keeps sounding while the next one resolves in the background. This brings LLM-powered text-to-music into live performance without audible interruption.  \nFor the performer, the key affordance is persistent steerability: a prompt establishes a scene, but the musical work happens through named parameter edits—darkening the spectrum, switching rhythm, widening the space—while the stream continues. Text becomes a playable control surface, not a one-shot trigger.  \nThe system is not a neural audio generator: language models resolve prompts into symbolic synthesizer configurations, and a deterministic procedural engine renders the sound—less timbrally general than neural text-to-audio, but inspectable, steerable, and stable during performance","cbCaiogkiw6Hd3ci","https://ap.wps.com/l/cbCaiogkiw6Hd3ci","pdf",1072827,1,10,"English","en",105,"# Introduction\n# Related Work","[{\"question\":\"How does the system let performers steer music without stopping audio?\",\"answer\":\"A prompt is resolved into an interpretable synthesizer parameter configuration, while a live generator keeps emitting audio. The next configuration resolves in the background and crossfades seamlessly when ready, so the audience hears continuous sound even with response latency.\"},{\"question\":\"What is the main technical difference from neural text-to-audio generators?\",\"answer\":\"Instead of generating raw audio waveforms directly, language models map prompts into structured, categorical synthesizer parameters. A deterministic procedural engine renders the sound, making it inspectable, steerable, and stable during performance.\"},{\"question\":\"What backends does the system use to generate the same configuration schema?\",\"answer\":\"The paper describes three interchangeable options: embedding retrieval for sub-second CPU-only use, hosted LLMs via API, and a finetuned 270M local model. All emit the same configuration schema to support consistent performer control.\"}]",1784196567,25,{"code":4,"msg":30,"data":31},"ok",{"site_id":24,"language":23,"slug":32,"title":13,"keywords":33,"description":14,"schema_data":34,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":27},"a-text-steerable-instrument-for-sketching-procedural-soundscapes-via-language-models","",{"@graph":35,"@context":85},[36,53,68],{"@type":37,"itemListElement":38},"BreadcrumbList",[39,43,47,50],{"item":40,"name":41,"@type":42,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":44,"name":45,"@type":42,"position":46},"https://docshare.wps.com/document/","Document",2,{"item":48,"name":12,"@type":42,"position":49},"https://docshare.wps.com/document/technology/",3,{"item":51,"name":13,"@type":42,"position":52},"https://docshare.wps.com/document/a-text-steerable-instrument-for-sketching-procedural-soundscapes-via-language-models/84544/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":23,"description":14,"dateModified":61,"datePublished":62,"encodingFormat":60,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":40,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-17","2026-07-16",true,{"@type":65,"interactionType":66,"userInteractionCount":20},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"How does the system let performers steer music without stopping audio?","Question",{"text":75,"@type":76},"A prompt is resolved into an interpretable synthesizer parameter configuration, while a live generator keeps emitting audio. The next configuration resolves in the background and crossfades seamlessly when ready, so the audience hears continuous sound even with response latency.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"What is the main technical difference from neural text-to-audio generators?",{"text":80,"@type":76},"Instead of generating raw audio waveforms directly, language models map prompts into structured, categorical synthesizer parameters. A deterministic procedural engine renders the sound, making it inspectable, steerable, and stable during performance.",{"name":82,"@type":73,"acceptedAnswer":83},"What backends does the system use to generate the same configuration schema?",{"text":84,"@type":76},"The paper describes three interchangeable options: embedding retrieval for sub-second CPU-only use, hosted LLMs via API, and a finetuned 270M local model. All emit the same configuration schema to support consistent performer control.","https://schema.org",{"og:url":51,"og:type":87,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":89,"canonical":51},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,113,118,123,128,131,134],{"id":20,"doc_module":4,"doc_module_name":45,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":46,"doc_module":4,"doc_module_name":45,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":45,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":45,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":11,"doc_module":4,"doc_module_name":45,"category_name":12,"show_sort_weight":111,"slug":112},50,"technology",{"id":114,"doc_module":4,"doc_module_name":45,"category_name":115,"show_sort_weight":116,"slug":117},7,"Healthcare",40,"healthcare",{"id":119,"doc_module":4,"doc_module_name":45,"category_name":120,"show_sort_weight":121,"slug":122},8,"Research & Report",30,"research-report",{"id":124,"doc_module":4,"doc_module_name":45,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":45,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":21,"doc_module":4,"doc_module_name":45,"category_name":132,"show_sort_weight":21,"slug":133},"Lifestyle","lifestyle",{"id":135,"doc_module":4,"doc_module_name":45,"category_name":136,"show_sort_weight":106,"slug":137},19,"General","general"]