[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-125856-en":3,"doc-seo-125856-105":31,"detail-sidebar-cat-0-en-105":93},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":21,"is_downloadable":21,"audit_status":21,"page_count":22,"language":23,"language_code":24,"site_id":25,"html_lang":24,"table_of_contents":26,"faqs":27,"seo_title":28,"seo_description":14,"update_tm":29,"read_time":30},125856,1099523882367,"Hazel","https://ap-avatar.wpscdn.com/davatar_9964176cb1d06d4a9deccf72a44ae3dc",8,"Research & Report","Evaluating Explainability in Machine Learning Predictions through Explainer-Agnostic Metrics","Rapid integration of AI into decision-making introduces governance and regulatory challenges, especially the need to understand complex model behavior. Decision-makers require explanations of machine learning predictions to support trust and ethical practice. This work proposes six explainer-agnostic metrics that quantify how well model predictions can be explained across local importance, global importance, and surrogate predictions, enabling comparative ranking by concentration, consistency, fluctuation, and surrogate fidelity/stability. Results are demonstrated on classification and regression tasks, and the metrics are released in a public Python package.","arXiv :2302 . 12094v3 [ cs .LG] 6 Nov 2024  \nEVALUATING EXPLAINABILITY IN MACHINE LEARNING PREDICTIONS THROUGH EXPLAINER-AGNOSTIC METRICS  \nCristian Munoz * 1 Kleyton da Costa * 1 2 Bernardo Modenesi * 3 Adriano Koshiyama 1  \nABSTRACT  \nThe rapid integration of artificial intelligence (AI) into various industries has introduced new challenges in governance and regulation, particularly regarding the understanding of complex AI systems. A critical demand from decision-makers is the ability to explain the results of machine learning models, which is essential for fostering trust and ensuring ethical AI practices. In this paper, we develop six distinct model-agnostic metrics designed to quantify the extent to which model predictions can be explained. These metrics measure different aspects of model explainability, ranging from local importance, global importance, and surrogate predictions, allowing for a comprehensive evaluation of how models generate their outputs. Furthermore, by computing our metrics, we can rank models in terms of explainability criteria such as importance concentration and consistency, prediction fluctuation, and surrogate fidelity and stability, offering a valuable tool for selecting models based not only on accuracy but also on transparency. We demonstrate the practical utility of these metrics on classification and regression tasks, and integrate these metrics into an existing Python package for public use.  \n1 INTRODUCTION  \nDespite the remarkable recent evolution in prediction performance by artificial intelligence (AI) models, they are often deemed as “black boxes”, i.e. models whose prediction mechanisms cannot be understood simply from their parameters. An explainable or interpretable algorithm isone for which the rules guiding its prediction decisions can be questioned and explained in a way that is intelligible to humans. Specifically, interpretability regards the ability to extract causal knowledge about the world from a model, and explainability pertains to the capability to articulate precisely how a complex model arrived at specific predictions, detailing its mechanics. Understanding AI models’behavior is essential for explaining predictions to support decision-making, debugging unexpected behaviors (contributing to improving model accuracy), refining modeling and data mining processes, verifying that model behavior is reasonable and fair, and effectively presenting predictions to stakeholders.  \nThe main goal of explainable artificial intelligence (XAI) encompasses several critical objectives (Ali et al., 2023) . XAI aims to empower individuals by enabling them to make  \n*Equal contribution 1Holistic AI, London, UK 2Department of Computer Science, Pontifical Catholic University of Rio de Janeiro, Rio de Janeiro, Brazil 3 College of Health, University of Utah, Utah, USA. Correspondence to: Cristian Munoz \u003Ccris[tian.munoz@holisticai.com](tian.munoz@holisticai.com) > .  \ninformed decisions, mitigating the potential harms of fully autonomous decision-making systems. XAI also identifies and addresses vulnerabilities that could compromise machine learning-based systems, bolstering their resilience. Lastly, it boosts user confidence in AI systems by promoting transparency and fostering a more clear understanding of the decisions made by these models.  \nThe AI literature offers various approaches to assessing explainability methods. Quantitative metrics can be used to assess whether these methods meet specific quality and reliability criteria (Bodria et al., 2023) . Common metrics include fidelity—how well the explanation aligns with the underlying model (Guidotti et al., 2018), stability—whether similar inputs yield consistent explanations (Alvarez Melis & Jaakkola, 2018 ; Guidotti & Ruggieri, 2019), faithfulness—how accurately the explanation reflects the true behavior of the model (Alvarez Melis & Jaakkola, 2018), monotonicity—whether more of a certain feature leads toa stronger explanation (Luss et","cbCaisv4UXBiROyC","https://ap.wps.com/l/cbCaisv4UXBiROyC","pdf",1526269,7,1,11,"English","en",105,"# Abstract\n# Introduction\n## Black-box models and the need for explainability\n## Goals and benefits of XAI\n## Existing quantitative and qualitative evaluation metrics\n## Limitations of prior evaluation approaches\n## Contribution: explainer-agnostic metrics for classification and regression","[{\"question\":\"What problem does the paper address in machine learning explainability?\",\"answer\":\"It addresses the need to evaluate how well predictions from complex AI models can be explained, supporting trust, transparency, and ethical AI governance.\"},{\"question\":\"What are explainer-agnostic metrics in this work?\",\"answer\":\"They are model-prediction explanation measures that do not depend on a specific explanation method, quantifying different aspects such as local/global importance and surrogate-based behavior.\"},{\"question\":\"How do the proposed metrics help compare models?\",\"answer\":\"By computing the metrics, models can be ranked using criteria like importance concentration and consistency, prediction fluctuation, and surrogate fidelity and stability in addition to accuracy.\"}]","Evaluating Explainability in Machine Learning Predictions through Explainer-Agnostic Metrics | PDF",1785901612,28,{"code":4,"msg":32,"data":33},"ok",{"site_id":25,"language":24,"slug":34,"title":13,"keywords":35,"description":14,"schema_data":36,"social_meta":88,"head_meta":90,"extra_data":92,"updated_unix":29},"evaluating-explainability-in-machine-learning-predictions-through-explainer-agnostic-metrics","",{"@graph":37,"@context":87},[38,55,70],{"@type":39,"itemListElement":40},"BreadcrumbList",[41,45,49,52],{"item":42,"name":43,"@type":44,"position":21},"https://docshare.wps.com","Home","ListItem",{"item":46,"name":47,"@type":44,"position":48},"https://docshare.wps.com/document/","Document",2,{"item":50,"name":12,"@type":44,"position":51},"https://docshare.wps.com/document/research-report/",3,{"item":53,"name":13,"@type":44,"position":54},"https://docshare.wps.com/document/evaluating-explainability-in-machine-learning-predictions-through-explainer-agnostic-metrics/125856/",4,{"url":53,"name":13,"@type":56,"author":57,"headline":13,"publisher":59,"fileFormat":62,"inLanguage":24,"description":14,"dateModified":63,"datePublished":64,"encodingFormat":62,"isAccessibleForFree":65,"interactionStatistic":66},"DigitalDocument",{"name":9,"@type":58},"Person",{"url":42,"name":60,"@type":61},"DocShare","Organization","application/pdf","2026-08-25","2026-08-05",true,{"@type":67,"interactionType":68,"userInteractionCount":20},"InteractionCounter",{"@type":69},"ViewAction",{"@type":71,"mainEntity":72},"FAQPage",[73,79,83],{"name":74,"@type":75,"acceptedAnswer":76},"What problem does the paper address in machine learning explainability?","Question",{"text":77,"@type":78},"It addresses the need to evaluate how well predictions from complex AI models can be explained, supporting trust, transparency, and ethical AI governance.","Answer",{"name":80,"@type":75,"acceptedAnswer":81},"What are explainer-agnostic metrics in this work?",{"text":82,"@type":78},"They are model-prediction explanation measures that do not depend on a specific explanation method, quantifying different aspects such as local/global importance and surrogate-based behavior.",{"name":84,"@type":75,"acceptedAnswer":85},"How do the proposed metrics help compare models?",{"text":86,"@type":78},"By computing the metrics, models can be ranked using criteria like importance concentration and consistency, prediction fluctuation, and surrogate fidelity and stability in addition to accuracy.","https://schema.org",{"og:url":53,"og:type":89,"og:title":13,"og:site_name":60,"og:description":14},"article",{"robots":91,"canonical":53},"index,follow",{"doc_id":7,"site_id":25},{"code":4,"msg":5,"data":94},[95,99,103,107,112,117,121,124,129,132,136],{"id":21,"doc_module":4,"doc_module_name":47,"category_name":96,"show_sort_weight":97,"slug":98},"Story & Novel",90,"story-novel",{"id":48,"doc_module":4,"doc_module_name":47,"category_name":100,"show_sort_weight":101,"slug":102},"Literature",80,"literature",{"id":54,"doc_module":4,"doc_module_name":47,"category_name":104,"show_sort_weight":105,"slug":106},"Exam",70,"exam",{"id":108,"doc_module":4,"doc_module_name":47,"category_name":109,"show_sort_weight":110,"slug":111},5,"Comic",60,"comic",{"id":113,"doc_module":4,"doc_module_name":47,"category_name":114,"show_sort_weight":115,"slug":116},6,"Technology",50,"technology",{"id":20,"doc_module":4,"doc_module_name":47,"category_name":118,"show_sort_weight":119,"slug":120},"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":47,"category_name":12,"show_sort_weight":122,"slug":123},30,"research-report",{"id":125,"doc_module":4,"doc_module_name":47,"category_name":126,"show_sort_weight":127,"slug":128},9,"Religion & Spirituality",20,"religion-spirituality",{"id":127,"doc_module":4,"doc_module_name":47,"category_name":130,"show_sort_weight":127,"slug":131},"World Cup","world-cup",{"id":133,"doc_module":4,"doc_module_name":47,"category_name":134,"show_sort_weight":133,"slug":135},10,"Lifestyle","lifestyle",{"id":137,"doc_module":4,"doc_module_name":47,"category_name":138,"show_sort_weight":108,"slug":139},19,"General","general"]