[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-128122-en":3,"doc-seo-128122-105":30,"detail-sidebar-cat-0-en-105":92},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":27,"seo_description":14,"update_tm":28,"read_time":29},128122,3985741905716,"Rowan","https://ap-avatar.wpscdn.com/davatar_994ba38a5ba835b3df7d355c54d3ed8d",8,"Research & Report","Pilots and Pixels - A Comparative Analysis of Machine Learning Error Effects on Aviation Decision Making","Despite rapid advances in machine learning (ML)-based decision support systems (DSS), persistent errors remain a safety concern in aviation. This study tests explainable and non-explainable ML-based DSSs by presenting pilots with different ML error types during 222 flight-simulation scenes. Results show false positives and false negatives both harm pilot trust and performance, with false negatives having the stronger impact. Explainable designs reduce some negative effects while increasing mental workload, informing ML-DSS development guided by error management theory.","Association for Information Systems  \nAIS Electronic Library (AISeL)  \n\n| ECIS 2024 Proceedings | European Conference on Information Systems\u003Cbr>(ECIS) |\n| --- | --- |\n| June 2024\u003Cbr>Pilots and Pixels: A Comparative Analysis of Machine Learning Error Effects on Aviation Decision Making\u003Cbr>Sara Ellenrieder\u003Cbr>Technical University of Darmstadt, ellenrieder@is.tu-darmstadt.de\u003Cbr>Nils Ellenrieder\u003Cbr>Technical University of Darmstadt, nils.ellenrieder@stud.tu-darmstadt.de\u003Cbr>Patrick Hendriks\u003Cbr>Technical University of Darmstadt, [patrick.hendriks@tu-darmstadt.de](patrick.hendriks@tu-darmstadt.de)\u003Cbr>Maren Mehler\u003Cbr>Technical University of Darmstadt, [maren.mehler@tu-darmstadt.de](maren.mehler@tu-darmstadt.de)\u003Cbr>Follow this and additional works at: [https://aisel.aisnet.org/ecis2024](https://aisel.aisnet.org/ecis2024) |  |\n\nRecommended Citation  \nEllenrieder, Sara; Ellenrieder, Nils; Hendriks, Patrick; and Mehler, Maren, \"Pilots and Pixels: A Comparative Analysis of Machine Learning Error Effects on Aviation Decision Making\" (2024) . ECIS 2024 Proceedings. 6.  \n[https://aisel.aisnet.org/ecis2024/track06_humanaicollab/track06_humanaicollab/6](https://aisel.aisnet.org/ecis2024/track06_humanaicollab/track06_humanaicollab/6)  \nThis material is brought to you by the European Conference on Information Systems (ECIS) at AIS Electronic Library (AISeL) . It has been accepted for inclusion in ECIS 2024 Proceedings by an authorized administrator of AIS Electronic Library (AISeL) . For more information, please [contact](contact elibrary@aisnet.org)[ elibrary@aisnet.org](contact elibrary@aisnet.org).  \nPILOTS AND PIXELS: A COMPARATIVE ANALYSIS OF MACHINE LEARNING ERROR EFFECTS ON AVIATION  \nDECISION MAKING  \nCompleted Research Paper  \nSara Ellenrieder, Technical University of Darmstadt, Darmstadt, Germany, ellenrieder@is.tu[darmstadt.de](darmstadt.de)  \nNils Ellenrieder, Technical University of Darmstadt, Darmstadt, Germany, nils.ellenrieder@stud.[tu-darmstadt.de](tu-darmstadt.de)  \nPatrick Hendriks, Technical University of Darmstadt, Darmstadt, Germany, hendriks@is.tu[darmstadt.de](darmstadt.de)  \nMaren F. Mehler, Technical University of Darmstadt, Darmstadt, Germany, mehler@is.tu[darmstadt.de](darmstadt.de)  \nAbstract  \nDespite immense improvements in machine learning (ML)-based decision support systems (DSSs), these systems are still prone to errors. For use in high-risk environments such as aviation it is critical, to find out what costs the different types of ML error cause for decision makers. Thus, we provide pilots holding a valid flight license with explainable and non-explainable ML-based DSSs that output different types of ML errors while supporting the visual detection of other aircraft in the vicinity in 222 recorded scenes of flight simulations. The study reveals that both false positives (FPs) and false negatives (FNs) detrimentally affect pilot trust and performance, with a more pronounced effect observedfor FNs. While explainable ML output design mitigates some negative effects, it significantly increases the mental workload for pilots when dealing with FPs. These findings inform the development of ML-based DSSs aligned with Error Management Theory to enhance applications in high-stakes environments.  \nKeywords: Machine Learning Error, Explainable Artificial Intelligence, Human-AI Interaction, Aviation Decision Making.  \n1 Introduction  \nAdvances in machine learning (ML) have driven the development of increasingly sophisticated MLbased decision support systems (DSS) that enable human decision makers to gain valuable insights in complex situations, ultimately improving their ability to make more informed and data-driven decisions (e.g., Berente et al., 2021; Jussupow et al., 2021; Sturm et al., 2023) . As ML-based DSSs have demonstrated remarkable capabilities, at times surpassing human experts in specific tasks (e.g., Shen et al., 2019), these systems are now increasingly being adopted in high-stakes environments (Maedche et al.","cbCaimLVqx2a6o7g","https://ap.wps.com/l/cbCaimLVqx2a6o7g","pdf",516980,1,17,"English","en",105,"# Introduction\n## Problem Background: ML-based DSSs in high-stakes aviation\n## Research Motivation: error management and explainability","[{\"question\":\"What does the study investigate about machine learning (ML) errors in aviation?\",\"answer\":\"It examines what costs different types of ML errors impose on decision makers, specifically pilots using ML-based decision support in aviation-relevant settings.\"},{\"question\":\"How do false positives and false negatives affect pilots?\",\"answer\":\"Both false positives and false negatives reduce pilot trust and performance, with false negatives producing a more pronounced negative effect.\"},{\"question\":\"What role does explainability play in handling ML errors?\",\"answer\":\"Explainable ML output designs mitigate some negative effects, but they also increase pilots' mental workload when dealing with false positives.\"}]","Pilots and Pixels - A Comparative Analysis of Machine Learning Error Effects on Aviation Decision Making | PDF",1785944941,43,{"code":4,"msg":31,"data":32},"ok",{"site_id":24,"language":23,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":87,"head_meta":89,"extra_data":91,"updated_unix":28},"pilots-and-pixels-a-comparative-analysis-of-machine-learning-error-effects-on-aviation-decision-making","",{"@graph":36,"@context":86},[37,54,69],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":53},"https://docshare.wps.com/document/pilots-and-pixels-a-comparative-analysis-of-machine-learning-error-effects-on-aviation-decision-making/128122/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":23,"description":14,"dateModified":62,"datePublished":63,"encodingFormat":61,"isAccessibleForFree":64,"interactionStatistic":65},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":41,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-22","2026-08-05",true,{"@type":66,"interactionType":67,"userInteractionCount":20},"InteractionCounter",{"@type":68},"ViewAction",{"@type":70,"mainEntity":71},"FAQPage",[72,78,82],{"name":73,"@type":74,"acceptedAnswer":75},"What does the study investigate about machine learning (ML) errors in aviation?","Question",{"text":76,"@type":77},"It examines what costs different types of ML errors impose on decision makers, specifically pilots using ML-based decision support in aviation-relevant settings.","Answer",{"name":79,"@type":74,"acceptedAnswer":80},"How do false positives and false negatives affect pilots?",{"text":81,"@type":77},"Both false positives and false negatives reduce pilot trust and performance, with false negatives producing a more pronounced negative effect.",{"name":83,"@type":74,"acceptedAnswer":84},"What role does explainability play in handling ML errors?",{"text":85,"@type":77},"Explainable ML output designs mitigate some negative effects, but they also increase pilots' mental workload when dealing with false positives.","https://schema.org",{"og:url":52,"og:type":88,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":90,"canonical":52},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":93},[94,98,102,106,111,116,121,124,129,132,136],{"id":20,"doc_module":4,"doc_module_name":46,"category_name":95,"show_sort_weight":96,"slug":97},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":99,"show_sort_weight":100,"slug":101},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":46,"category_name":103,"show_sort_weight":104,"slug":105},"Exam",70,"exam",{"id":107,"doc_module":4,"doc_module_name":46,"category_name":108,"show_sort_weight":109,"slug":110},5,"Comic",60,"comic",{"id":112,"doc_module":4,"doc_module_name":46,"category_name":113,"show_sort_weight":114,"slug":115},6,"Technology",50,"technology",{"id":117,"doc_module":4,"doc_module_name":46,"category_name":118,"show_sort_weight":119,"slug":120},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":122,"slug":123},30,"research-report",{"id":125,"doc_module":4,"doc_module_name":46,"category_name":126,"show_sort_weight":127,"slug":128},9,"Religion & Spirituality",20,"religion-spirituality",{"id":127,"doc_module":4,"doc_module_name":46,"category_name":130,"show_sort_weight":127,"slug":131},"World Cup","world-cup",{"id":133,"doc_module":4,"doc_module_name":46,"category_name":134,"show_sort_weight":133,"slug":135},10,"Lifestyle","lifestyle",{"id":137,"doc_module":4,"doc_module_name":46,"category_name":138,"show_sort_weight":107,"slug":139},19,"General","general"]