[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-125920-en":3,"doc-seo-125920-105":31,"detail-sidebar-cat-0-en-105":93},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":21,"is_downloadable":21,"audit_status":21,"page_count":22,"language":23,"language_code":24,"site_id":25,"html_lang":24,"table_of_contents":26,"faqs":27,"seo_title":28,"seo_description":14,"update_tm":29,"read_time":30},125920,2336474459895,"Aria","https://ap-avatar.wpscdn.com/avatar/22000baeef7a5ed0655?x-image-process=image/resize,m_fixed,w_180,h_180&k=1786071322749376916",8,"Research & Report","Enhancing Autonomous Vehicle Safety through N-version Machine Learning Systems","Unreliable machine learning outputs threaten safety-critical autonomous driving when models face out-of-distribution samples, distribution shifts, transient hardware faults, or malicious/adversarial attacks. N-version ML addresses reliability via diversification across models and inputs, yet prior work rarely evaluates impacts in practical deployments. This paper studies N-version perception in CARLA, using two- and three-version systems with both healthy and fault-compromised models generated via fault injection. Results show compromised models significantly raise collision rates, while three-version perception can tolerate one compromised model and delay collisions when at least one model remains healthy.","Enhancing Autonomous Vehicle Safety through N-version Machine Learning Systems  \nQiang Wen1, * , Júlio Mendonça2, Fumio Machida1 and Marcus Völp2  \n1 Department of Computer Science, University of Tsukuba, 305-8573, Japan  \n2 Interdisciplinary Centre for Security, Reliability and Trust (SnT), University of Luxembourg, L-1855, Luxembourg  \nAbstract  \nUnreliable outputs of machine learning (ML) models are a significant concern, particularly for safety-critical applications such as autonomous driving. ML models are susceptible to out-of-distribution samples, distribution shifts, hardware transient faults, and even malicious attacks. To address the concerns, the N-version ML system gives a general solution to enhance the reliability of ML system outputs by employing diversification on ML models and their inputs. However, the existing studies of N-version ML systems mainly focused on classification errors and did not consider their impacts in a practical application scenario. In this paper, we investigate the applicability of N-version ML approach in an autonomous vehicle (AV) scenario within the AV simulator CARLA. We deploy two-version and three-version perception systems in an AV implemented in CARLA, using healthy ML models and compromised ML models, which are generated using fault-injection techniques and analyze the behavior of the AV in the simulator. Our findings reveal the critical impacts of compromised models on AV collision rates and show the potential of three-version perception systems in mitigating the risk. Our three-version perception system improves driving safety by tolerating one compromised model and delaying collisions when having at least one healthy model.  \nKeywords  \nautonomous driving, fault injection, machine learning system, N-version programming, perception  \n1. Introduction  \nRapid machine learning (ML) advancements have led to widespread applications across various domains. MLbased intelligent software systems, including face recognition, medical diagnosis, and autonomous robots, have become integral parts of our daily lives [1, 2] . However, ML models cannot guarantee a correct output in the application context due to ML models’ uncertainties in dealing with real samples [3] . Additionally, transient faults (e.g., leading to bit-flip errors [4]) and malicious attacks such as adversarial attacks [5] may affect the system’s capability to provide correct outputs, especially when a single ML model is in the software stack [6, 7] . When ML-based applications are incorporated into safety-critical systems, incorrect outputs can cause undesirable consequences. For example, the misrecognition of traffic signs by MLbased classifiers could result in accidents in autonomous driving scenarios. By using this example, we should agree that ensuring the correctness of ML-based system outputs has become a critical concern, especially for systems in safety-critical domains.  \nVarious approaches have been proposed to enhance  \nThe IJCAI-24 Workshop on Artificial Intelligence Safety (AISafety 2024), August 04, 2024, Jeju, South Korea  \n* Corresponding author.  \n$ [wen.qiang@sd.cs.tsukuba.ac.jp](wen.qiang@sd.cs.tsukuba.ac.jp) (Q. Wen);  \n[julio.mendonca@uni.lu](julio.mendonca@uni.lu) (J. Mendonça); [machida@cs.tsukuba.ac.jp](machida@cs.tsukuba.ac.jp)  \n(F. Machida); [marcus.voelp@uni.lu](marcus.voelp@uni.lu) (M. Völp)  \n© 2024 Copyright © 2024 for this paper by its authors. Use permitted under Creative Commons  \n\n|  | CEUR Workshop Proceedings |\n| --- | --- |\n\nLicense Attribution 4 .0 International (CC BY 4 .0) .  \nCEUR Workshop Proceedings ([CEUR-WS.org](CEUR-WS.org))  \n[http://ceur-ws.org](http://ceur-ws.org)  \n[ISSN 1613-0073](ISSN 1613-0073)  \nthe robustness of ML systems. ML testing is one of these approaches that focuses on detecting differences between existing and required behaviors of machine learning systems [8] . However, the existing works mainly focus on offline testing rather than runtime monitoring. To impr","cbCaijCC3aj7nF43","https://ap.wps.com/l/cbCaijCC3aj7nF43","pdf",12264207,5,1,9,"English","en",105,"# Introduction\n## Problem: unreliable ML outputs in safety-critical domains\n## Related approaches and their limitations\n# N-version ML for autonomous vehicle perception\n## Method: two- and three-version systems in CARLA\n## Fault model: compromised perception via fault injection\n# Results and findings\n## Collision-rate impact under compromised models\n## Safety mitigation using three-version perception\n# Conclusion","[{\"question\":\"What makes machine learning outputs unreliable in autonomous driving systems?\",\"answer\":\"ML models can produce incorrect outputs under out-of-distribution samples and distribution shifts, as well as due to transient hardware faults and malicious/adversarial attacks in the software stack.\"},{\"question\":\"How does the N-version ML system improve reliability?\",\"answer\":\"It improves output reliability by employing diversification across multiple independently functioning ML models and their inputs, reducing single points of failure.\"},{\"question\":\"What do the CARLA simulation results show about two- vs. three-version perception?\",\"answer\":\"Compromised models critically increase AV collision rates, while three-version perception can mitigate risk by tolerating one compromised model and delaying collisions when at least one model is healthy.\"}]","Enhancing Autonomous Vehicle Safety through N-version Machine Learning Systems | PDF",1785902033,23,{"code":4,"msg":32,"data":33},"ok",{"site_id":25,"language":24,"slug":34,"title":13,"keywords":35,"description":14,"schema_data":36,"social_meta":88,"head_meta":90,"extra_data":92,"updated_unix":29},"enhancing-autonomous-vehicle-safety-through-n-version-machine-learning-systems","",{"@graph":37,"@context":87},[38,55,70],{"@type":39,"itemListElement":40},"BreadcrumbList",[41,45,49,52],{"item":42,"name":43,"@type":44,"position":21},"https://docshare.wps.com","Home","ListItem",{"item":46,"name":47,"@type":44,"position":48},"https://docshare.wps.com/document/","Document",2,{"item":50,"name":12,"@type":44,"position":51},"https://docshare.wps.com/document/research-report/",3,{"item":53,"name":13,"@type":44,"position":54},"https://docshare.wps.com/document/enhancing-autonomous-vehicle-safety-through-n-version-machine-learning-systems/125920/",4,{"url":53,"name":13,"@type":56,"author":57,"headline":13,"publisher":59,"fileFormat":62,"inLanguage":24,"description":14,"dateModified":63,"datePublished":64,"encodingFormat":62,"isAccessibleForFree":65,"interactionStatistic":66},"DigitalDocument",{"name":9,"@type":58},"Person",{"url":42,"name":60,"@type":61},"DocShare","Organization","application/pdf","2026-08-24","2026-08-05",true,{"@type":67,"interactionType":68,"userInteractionCount":20},"InteractionCounter",{"@type":69},"ViewAction",{"@type":71,"mainEntity":72},"FAQPage",[73,79,83],{"name":74,"@type":75,"acceptedAnswer":76},"What makes machine learning outputs unreliable in autonomous driving systems?","Question",{"text":77,"@type":78},"ML models can produce incorrect outputs under out-of-distribution samples and distribution shifts, as well as due to transient hardware faults and malicious/adversarial attacks in the software stack.","Answer",{"name":80,"@type":75,"acceptedAnswer":81},"How does the N-version ML system improve reliability?",{"text":82,"@type":78},"It improves output reliability by employing diversification across multiple independently functioning ML models and their inputs, reducing single points of failure.",{"name":84,"@type":75,"acceptedAnswer":85},"What do the CARLA simulation results show about two- vs. three-version perception?",{"text":86,"@type":78},"Compromised models critically increase AV collision rates, while three-version perception can mitigate risk by tolerating one compromised model and delaying collisions when at least one model is healthy.","https://schema.org",{"og:url":53,"og:type":89,"og:title":13,"og:site_name":60,"og:description":14},"article",{"robots":91,"canonical":53},"index,follow",{"doc_id":7,"site_id":25},{"code":4,"msg":5,"data":94},[95,99,103,107,111,116,121,124,128,131,135],{"id":21,"doc_module":4,"doc_module_name":47,"category_name":96,"show_sort_weight":97,"slug":98},"Story & Novel",90,"story-novel",{"id":48,"doc_module":4,"doc_module_name":47,"category_name":100,"show_sort_weight":101,"slug":102},"Literature",80,"literature",{"id":54,"doc_module":4,"doc_module_name":47,"category_name":104,"show_sort_weight":105,"slug":106},"Exam",70,"exam",{"id":20,"doc_module":4,"doc_module_name":47,"category_name":108,"show_sort_weight":109,"slug":110},"Comic",60,"comic",{"id":112,"doc_module":4,"doc_module_name":47,"category_name":113,"show_sort_weight":114,"slug":115},6,"Technology",50,"technology",{"id":117,"doc_module":4,"doc_module_name":47,"category_name":118,"show_sort_weight":119,"slug":120},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":47,"category_name":12,"show_sort_weight":122,"slug":123},30,"research-report",{"id":22,"doc_module":4,"doc_module_name":47,"category_name":125,"show_sort_weight":126,"slug":127},"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":47,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":47,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":47,"category_name":137,"show_sort_weight":20,"slug":138},19,"General","general"]