[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-85738-en":3,"doc-seo-85738-105":29,"detail-sidebar-cat-0-en-105":90},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":4,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":13,"seo_description":14,"update_tm":27,"read_time":28},85738,5909877438554,"Maeve","https://ap-avatar.wpscdn.com/avatar/5600025385ad2bf12a7?_k=1778553567797529272",8,"Research & Report","Generative Testing of Automated Speech Recognition Systems","Transformer-based automatic speech recognition (ASR) systems deliver high accuracy and are used in high-stakes settings, yet they remain susceptible to adversarial manipulation. This work presents GATAS, a black-box testing method that generates failure-inducing inputs by searching in a text-to-speech model’s phoneme-level latent space. Instead of perturbing waveforms directly, it interpolates latent representations to induce transcription errors while staying within the natural speech manifold. The attack is posed as multi-objective optimization balancing semantic divergence and perceptual quality, reaching 98% success with lower distortion and improved perceptual scores in human studies.","Generative Testing of Automated Speech Recognition Systems  \nYanis Xabier Wilbrand Peña  \nTechnical University of Munich Germany  \nOliver Weißl  \n[o.weissl@tum.de](o.weissl@tum.de)[ ](o.weissl@tum.de)Technical University of Munich Germany  \nAndrea Stocco  \n[andrea.stocco@tum.de](andrea.stocco@tum.de)[ ](andrea.stocco@tum.de)Technical University of Munich & fortiss GmbH Germany  \narXiv :2607 .09833v 1 [ cs .CR] 10 Jul 2026  \nAbstract  \nAutomatic speech recognition (ASR) systems have achieved high accuracy with transformer-based models, enabling deployment in critical applications. However, they remain vulnerable to adversarial manipulation, particularly in black-box settings where attacks must preserve perceptual naturalness. This work introduces GATAS, a black-box testing approach that generates failure inducing inputs by operating in the phoneme-level latent space of a textto-speech model. Instead of perturbing waveforms directly, the approach interpolates latent representations to induce transcription errors while remaining within the manifold of natural speech. The attack is formulated as a multi-objective optimization problem balancing semantic divergence and perceptual quality. Our empirical evaluation against both white-box and black-box baselines shows that GATAS achieves a 98% success rate while producing lower distortion and higher perceptual quality, as confirmed by human studies. Despite operating without gradient access, GATAS remains competitive against white-box methods, highlighting that representation and perceptual alignment are more critical than access to model internals. Overall, our results demonstrate that untargeted latent-space optimization enables the efficient generation of realistic and effective test cases for ASR systems.  \n1 Introduction  \nAutomatic speech recognition (ASR) has become a central interface between humans and machines, enabling applications such as voice assistants, transcription services, and accessibility tools [2] . Recent transformer-based models, including Whisper [38], have reached performance levels that support deployment in high-stakes settings such as medical documentation, legal proceedings, and security-critical environments [29, 47] . This increased reliance demands systematic testing, as learning-enabled systems such as ASRare vulnerable to subtle input variations that can lead to incorrect behavior [23, 42] . In particular, adversarial manipulation has shown that benign-sounding speech can be modified to induce incorrect transcriptions without introducing perceptible artifacts, raising the question of whether ASR systems behave reliably not only under nominal conditions, but also under realistic, hard-todetect variations of valid inputs.  \nPrior work has demonstrated the feasibility of adversarial attacks on ASR systems under both white-box and black-box assumptions, but with important limitations. White-box approaches [13] rely on gradient access to the target model, which restricts their applicability in real-world deployments. Black-box methods typically operate in waveform space, where perturbations tend to introduce audible distortions [4, 50] . Transfer-based approaches avoid iterative querying but require training surrogate models [16], and the resulting adversarial audio still contains perceptible artifacts. In  \nboth cases, audio quality is treated as a secondary concern rather than an explicit optimization objective. While black-box attacks on ASR are well studied, methods that generate audio preserving both similarity to the original utterance and naturalness while inducing semantic divergence in the transcription remain limited. This gap motivates testing approaches that operate within a structured representation space rather than applying unconstrained perturbations to the raw audio signal.  \nTo address this gap, this work proposes GATAS (Generative Testing of Automatic Speech Recognition Systems), a novel blackbox testing approach forASR systems that gene","cbCaimgCRKMOZ0kC","https://ap.wps.com/l/cbCaimgCRKMOZ0kC","pdf",1118819,1,12,"English","en",105,"# Introduction\n## Motivation and prior work\n## Proposed method: GATAS\n## Experimental setup and evaluation","[{\"question\":\"What problem does GATAS address for automated speech recognition systems?\",\"answer\":\"GATAS addresses the vulnerability of ASR to adversarial manipulation, especially in black-box settings where attacks must remain perceptually natural while causing incorrect transcriptions.\"},{\"question\":\"How does GATAS generate adversarial test cases without perturbing raw audio directly?\",\"answer\":\"GATAS uses a text-to-speech model to represent speech via phoneme-level embeddings, then interpolates in the latent space to induce transcription errors while the TTS decoder keeps the audio natural-sounding.\"},{\"question\":\"How is the attack objective defined and what does the evaluation show?\",\"answer\":\"The method formulates test generation as multi-objective optimization balancing semantic divergence and perceptual quality. Empirical results against white-box and black-box baselines report a 98% success rate with lower distortion and higher perceptual quality, supported by human studies.\"}]",1784205928,30,{"code":4,"msg":30,"data":31},"ok",{"site_id":24,"language":23,"slug":32,"title":13,"keywords":33,"description":14,"schema_data":34,"social_meta":85,"head_meta":87,"extra_data":89,"updated_unix":27},"generative-testing-of-automated-speech-recognition-systems","",{"@graph":35,"@context":84},[36,53,67],{"@type":37,"itemListElement":38},"BreadcrumbList",[39,43,47,50],{"item":40,"name":41,"@type":42,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":44,"name":45,"@type":42,"position":46},"https://docshare.wps.com/document/","Document",2,{"item":48,"name":12,"@type":42,"position":49},"https://docshare.wps.com/document/research-report/",3,{"item":51,"name":13,"@type":42,"position":52},"https://docshare.wps.com/document/generative-testing-of-automated-speech-recognition-systems/85738/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":23,"description":14,"dateModified":61,"datePublished":61,"encodingFormat":60,"isAccessibleForFree":62,"interactionStatistic":63},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":40,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-16",true,{"@type":64,"interactionType":65,"userInteractionCount":4},"InteractionCounter",{"@type":66},"ViewAction",{"@type":68,"mainEntity":69},"FAQPage",[70,76,80],{"name":71,"@type":72,"acceptedAnswer":73},"What problem does GATAS address for automated speech recognition systems?","Question",{"text":74,"@type":75},"GATAS addresses the vulnerability of ASR to adversarial manipulation, especially in black-box settings where attacks must remain perceptually natural while causing incorrect transcriptions.","Answer",{"name":77,"@type":72,"acceptedAnswer":78},"How does GATAS generate adversarial test cases without perturbing raw audio directly?",{"text":79,"@type":75},"GATAS uses a text-to-speech model to represent speech via phoneme-level embeddings, then interpolates in the latent space to induce transcription errors while the TTS decoder keeps the audio natural-sounding.",{"name":81,"@type":72,"acceptedAnswer":82},"How is the attack objective defined and what does the evaluation show?",{"text":83,"@type":75},"The method formulates test generation as multi-objective optimization balancing semantic divergence and perceptual quality. Empirical results against white-box and black-box baselines report a 98% success rate with lower distortion and higher perceptual quality, supported by human studies.","https://schema.org",{"og:url":51,"og:type":86,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":88,"canonical":51},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":91},[92,96,100,104,109,114,119,121,126,129,133],{"id":20,"doc_module":4,"doc_module_name":45,"category_name":93,"show_sort_weight":94,"slug":95},"Story & Novel",90,"story-novel",{"id":46,"doc_module":4,"doc_module_name":45,"category_name":97,"show_sort_weight":98,"slug":99},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":45,"category_name":101,"show_sort_weight":102,"slug":103},"Exam",70,"exam",{"id":105,"doc_module":4,"doc_module_name":45,"category_name":106,"show_sort_weight":107,"slug":108},5,"Comic",60,"comic",{"id":110,"doc_module":4,"doc_module_name":45,"category_name":111,"show_sort_weight":112,"slug":113},6,"Technology",50,"technology",{"id":115,"doc_module":4,"doc_module_name":45,"category_name":116,"show_sort_weight":117,"slug":118},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":45,"category_name":12,"show_sort_weight":28,"slug":120},"research-report",{"id":122,"doc_module":4,"doc_module_name":45,"category_name":123,"show_sort_weight":124,"slug":125},9,"Religion & Spirituality",20,"religion-spirituality",{"id":124,"doc_module":4,"doc_module_name":45,"category_name":127,"show_sort_weight":124,"slug":128},"World Cup","world-cup",{"id":130,"doc_module":4,"doc_module_name":45,"category_name":131,"show_sort_weight":130,"slug":132},10,"Lifestyle","lifestyle",{"id":134,"doc_module":4,"doc_module_name":45,"category_name":135,"show_sort_weight":105,"slug":136},19,"General","general"]