[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-122212-en":3,"doc-seo-122212-105":30,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":4,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":27,"seo_description":14,"update_tm":28,"read_time":29},122212,13056703019404,"Miles","https://ap-avatar.wpscdn.com/davatar_29158cc5080c5b710cf443261637dec0",8,"Research & Report","A Theory of Interpretable Approximations - Marco Bressan and Colleagues","Deep neural networks can be difficult to explain, driving demand for machine learning models whose predictions are intelligible to humans. This work studies interpretable approximations, where a target concept c is approximated by a small aggregation of concepts from a base class H using decision trees. Measuring complexity via tree depth, the paper establishes a trichotomy: either c is not arbitrarily approximable, or it is approximable but admits no universal rate, or a constant κ guarantees bounded complexity across all distributions and accuracy levels. It further derives results for unbounded VC dimension through algebraic characterizations.","A Theory of Interpretable Approximations  \nMarco Bressan  \nUniversita` degli Studi di Milano, Italy  \nNicol Cesa-Bianchi  \nUniversita` degli Studi di Milano, Italy Politecnico di Milano, Italy  \nEmmanuel Esposito  \nUniversita` degli Studi di Milano, Italy Istituto Italiano di Tecnologia, Italy  \nYishay Mansour  \nTel Aviv University, Israel Google Research  \nShay Moran  \nTechnion, Israel Google Research  \nMaximilian Thiessen  \nTU Wien, Austria  \nMARCO . BRESSAN @UNIMI. IT NICOLO . CESA-BIANCHI@UNIMI. IT  \nEMMANUEL @EMMANUELESPOSITO . IT  \nMANSOUR . YISHAY @ GMAIL . COM  \nSMORAN @TECHNION . AC . IL  \nMAXIMILIAN . THIESSEN @TUWIEN . AC . AT  \nEditors: Shipra Agrawal and Aaron Roth  \nAbstract  \nCan a deep neural network be approximated by a small decision tree based on simple features? This question and its variants are behind the growing demand for machine learning models that are interpretable by humans. In this work we study such questions by introducing interpretable approximations, a notion that captures the idea of approximating a target concept c by a small aggregation of concepts from some base class H. In particular, we consider the approximation of a binary concept c by decision trees based on a simple class H (e.g., of bounded VC dimension), and use the tree depth as a measure of complexity. Our primary contribution is the following remarkable trichotomy. For any given pair of H and c, exactly one of these cases holds: (i) c cannot be approximated by H with arbitrary accuracy; (ii) c can be approximated by H with arbitrary accuracy, but there exists no universal rate that bounds the complexity of the approximations as a function of the accuracy; or (iii) there exists a constant κ that depends only on H and c such that, for any data distribution and any desired accuracy level, c can be approximated by H with a complexity not exceeding κ . This taxonomy stands in stark contrast to the landscape of supervised classification, which offers a complex array of distribution-free and universally learnable scenarios. We show that, in the case of interpretable approximations, even a slightly nontrivial a-priori guarantee on the complexity of approximations implies approximations with constant (distribution-free and accuracy-free) complexity. We extend our trichotomy to classes H of unbounded VC dimension and give characterizations of interpretability based on the algebra generated by H.  \nKeywords: interpretability, learning theory, boosting  \n© 2024 M. Bressan, N. Cesa-Bianchi, E. Esposito, Y. Mansour, S. Moran & M. Thiessen.  \nBRESSAN CESA-BIANCHI ESPOSITO MANSOUR MORAN THIESSEN  \n1. Introduction  \nMany machine learning techniques, such as deep neural networks, produce large and complex models whose inner workings are difficult to grasp. In sectors such as healthcare and law enforcement, where the stakes of automated decisions are high, this is a serious problem: complex models make it hard to explain the rationale behind an outcome, or why two similar inputs produce different outcomes. In those cases, interpretable models may become the preferred choice. Although there is an ongoing debate around the notion of interpretability (Erasmus, Brunet, and Fisher, 2021), decision trees are typically considered as the quintessential example of interpretable models (Molnar, 2022): ones that favor a transparent decision-making process, and that allow users to understand how individual features influence predictions. A line of research in this area studies the extent to which small decision trees can approximate some specific learning models, such as neural networks (Craven and Shavlik, 1995) and k-means classifiers (Dasgupta, Frost, Moshkovitz, and Rashtchian, 2020) . Inspired by these results, we develop a general theory of interpretability viewed as approximability via simple decision trees. Our guiding principle can be summarized as follows.  \nInterpretable approximations = Small aggregations of simple hypotheses.  \nIn analogy with PAC","cbCailc8CCie6oZx","https://ap.wps.com/l/cbCailc8CCie6oZx","pdf",352092,1,21,"English","en",105,"# Introduction\n## Interpretable models and decision trees\n## Interpretable approximations via small aggregations\n## Approximability vs interpretability\n## Scope and lack of distributional assumptions","[{\"question\":\"What are interpretable approximations in this paper?\",\"answer\":\"They approximate a target concept c using a small aggregation of simple hypotheses from a base class H, realized via decision trees with hypotheses drawn from H.\"},{\"question\":\"What does the paper’s main trichotomy state?\",\"answer\":\"For any pair of H and c, exactly one holds: c is not arbitrarily approximable by H; it is arbitrarily approximable but has no universal complexity rate; or there is a constant κ such that any distribution and accuracy level admit approximations with complexity at most κ.\"},{\"question\":\"How is complexity measured for the decision-tree approximations?\",\"answer\":\"The tree depth is used as the measure of complexity for the approximations.\"}]","A Theory of Interpretable Approximations - Marco Bressan and Colleagues | PDF",1785809388,53,{"code":4,"msg":31,"data":32},"ok",{"site_id":24,"language":23,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":28},"a-theory-of-interpretable-approximations-marco-bressan-and-colleagues","",{"@graph":36,"@context":85},[37,54,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":53},"https://docshare.wps.com/document/a-theory-of-interpretable-approximations-marco-bressan-and-colleagues/122212/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":23,"description":14,"dateModified":62,"datePublished":62,"encodingFormat":61,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":41,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-04",true,{"@type":65,"interactionType":66,"userInteractionCount":4},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"What are interpretable approximations in this paper?","Question",{"text":75,"@type":76},"They approximate a target concept c using a small aggregation of simple hypotheses from a base class H, realized via decision trees with hypotheses drawn from H.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"What does the paper’s main trichotomy state?",{"text":80,"@type":76},"For any pair of H and c, exactly one holds: c is not arbitrarily approximable by H; it is arbitrarily approximable but has no universal complexity rate; or there is a constant κ such that any distribution and accuracy level admit approximations with complexity at most κ.",{"name":82,"@type":73,"acceptedAnswer":83},"How is complexity measured for the decision-tree approximations?",{"text":84,"@type":76},"The tree depth is used as the measure of complexity for the approximations.","https://schema.org",{"og:url":52,"og:type":87,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":89,"canonical":52},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":20,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":46,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":46,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":46,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":46,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":46,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":46,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]