[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-127015-en":3,"doc-seo-127015-105":30,"detail-sidebar-cat-0-en-105":92},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":27,"seo_description":14,"update_tm":28,"read_time":29},127015,2336474466412,"Ezra","https://ap-avatar.wpscdn.com/davatar_155a257f0dc6eb9ab79c44ca47cae57d",8,"Research & Report","Local Feature Selection without Label or Feature Leakage for Interpretable Machine Learning Predictions - ArXiv 2407.11778v1 - first proposed SUWR","Local feature selection in interpretable machine learning generates instance-specific explanations by keeping only the most relevant input features for each prediction. This work addresses a key limitation: selections can become misleading when additional information is encoded in the selected features. The paper formalizes label leakage and feature leakage, derives necessary and sufficient conditions to guarantee absence of leakage, and shows existing methods fail to satisfy them. It introduces SUWR, the first leakage-free local selector with experiments showing reduced overfitting, strong predictive performance, and high sparsity.","Local Feature Selection without Label or Feature Leakage for Interpretable Machine Learning Predictions  \nHarrie Oosterhuis * 1 Lijun Lyu * 2 Avishek Anand 2  \narXiv :2407 . 1 1778v 1 [ cs .LG] 16 Jul 2024  \nAbstract  \nLocal feature selection in machine learning provides instance-specific explanations by focusing on the most relevant features for each prediction, enhancing the interpretability of complex models.  \nHowever, such methods tend to produce misleading explanations by encoding additional information in their selections. In this work, we attribute the problem of misleading selections by formalizing the concepts of label and feature leakage. We rigorously derive the necessary and sufficient conditions under which we can guarantee no leakage, and show existing methods do not meet these conditions. Furthermore, we propose the first local feature selection method that is proven to have no leakage called SUWR. Our experimental results indicate that SUWR is less prone to overfitting and combines state-of-the-art predictive performance with high feature-selection sparsity. Our generic and easily extendable formal approach provides a strong theoretical basis for future work on interpretability with reliable explanations.  \n1. Introduction  \nFeature attributions and feature selections in interpretable machine learning (ML) help users understand how much each input feature influences the output of the model (Duet al., 2019 ; Molnar, 2020) . One prominent family of methods are designed for local feature selection, a.k.a. instancewise feature selection, for interpretable ML (Gurrapu et al., 2023) . These approaches aim to only select the mostimportant features per instance and to exclude the rest during inference (Li et al., 2017), thereby making the predictions by the model easier to interpret.  \nLet i refer to an instance in a dataset with xi ∈ Rd as its  \n*Equal contribution 1Radboud University, Nijmegen, The Netherlands 2TU Delft, Delft, The Netherlands. Correspondence to: Harrie Oosterhuis \u003C[harrie.oosterhuis@ru.nl](harrie.oosterhuis@ru.nl) > .  \nProceedings of the 41 st International Conference on Machine Learning, Vienna, Austria. PMLR 235, 2024 . Copyright 2024 by the author(s) .  \nd-dimensional feature vector representation and yi as its accompanying label to be predicted. A feature selector ζ takes xi as input and outputs a feature mask hi ∈ {0, 1}d , either through a stochastic or deterministic process: hi ∼ζ (xi) . Let xi ⊙ hi indicate the masked features that results from applying hi to xi , where all non-selected features are masked. We denote a masked feature with ∅, to clearly differentiate it from a zero value, and the jth element in a vector with [j]:  \n(xi ⊙ hi )[j] := (i [j]  \nif hi [j] = 1,  \nif hi [j] = 0 .  \n(1)  \nhere ζ is a local selector and produces a different mask for each instance. We note that local feature selection differs from global feature selection which reveals feature importance on the dataset level, as it has a fixed mask for all instances (Balın et al., 2019 ; Lemhadri et al., 2021 ; Yamada et al., 2020 ; Lee et al., 2021) . By being able to vary masks, local methods are more flexible and can give more in-depth insight into the importance of features in individual instances (Yoon et al., 2018 ; Arik & Pfister, 2021) .  \nA widely used setup for local feature selection is to follow aselector-predictor architecture that is typically jointly optimized (Yoon et al., 2018 ; Jethani et al., 2021 ; Arik & Pfister, 2021) . More precisely, let f be the predictor model that can take masked features as input: f (x ⊙ h), importantly, h can be inferred exactly from x ⊙ h. The optimization of theselector ζ and predictor f is usually based on a linear combination of a prediction loss L and the sparsity of a mask ∥h∥ to enforce high sparsity (and hence interpretability) . For adataset of N instances, we use:  \nN  \nL (ζ,f):= ~~ ~~1N X Ehi ∼ζ (xi)􀀂L (f(xi ⊙hi), yi )+λ∥hi ∥ 􀀃 , (2) i=1  \nwhere the parameter λ","cbCaiuyDSUJ8wq4D","https://ap.wps.com/l/cbCaiuyDSUJ8wq4D","pdf",853556,1,22,"English","en",105,"# Introduction\n## Feature-attribution and local feature selection background\n## Selector-predictor setup and optimization objective\n## Leakage problem and motivation","[{\"question\":\"What problem does the paper identify in local feature selection methods?\",\"answer\":\"Local feature selection can produce misleading explanations when the selector encodes extra information in the feature mask, rather than reflecting true feature importance.\"},{\"question\":\"How does the paper formalize “leakage”?\",\"answer\":\"It introduces two concepts: label leakage, where label information is encoded in the local selection, and feature leakage, where additional information is encoded in the selected features.\"},{\"question\":\"What is SUWR and what results does it show?\",\"answer\":\"SUWR is the first local feature selection method proven to have no leakage. Experiments indicate it is less prone to overfitting while maintaining strong predictive performance and high feature-selection sparsity.\"}]","Local Feature Selection without Label or Feature Leakage for Interpretable Machine Learning Predictions - ArXiv 2407.11778v1 - first proposed SUWR | PDF",1785936331,55,{"code":4,"msg":31,"data":32},"ok",{"site_id":24,"language":23,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":87,"head_meta":89,"extra_data":91,"updated_unix":28},"local-feature-selection-without-label-or-feature-leakage-for-interpretable-machine-learning-predictions-arxiv-240711778v1-first-proposed-suwr","",{"@graph":36,"@context":86},[37,54,69],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":53},"https://docshare.wps.com/document/local-feature-selection-without-label-or-feature-leakage-for-interpretable-machine-learning-predictions-arxiv-240711778v1-first-proposed-suwr/127015/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":23,"description":14,"dateModified":62,"datePublished":63,"encodingFormat":61,"isAccessibleForFree":64,"interactionStatistic":65},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":41,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-22","2026-08-05",true,{"@type":66,"interactionType":67,"userInteractionCount":20},"InteractionCounter",{"@type":68},"ViewAction",{"@type":70,"mainEntity":71},"FAQPage",[72,78,82],{"name":73,"@type":74,"acceptedAnswer":75},"What problem does the paper identify in local feature selection methods?","Question",{"text":76,"@type":77},"Local feature selection can produce misleading explanations when the selector encodes extra information in the feature mask, rather than reflecting true feature importance.","Answer",{"name":79,"@type":74,"acceptedAnswer":80},"How does the paper formalize “leakage”?",{"text":81,"@type":77},"It introduces two concepts: label leakage, where label information is encoded in the local selection, and feature leakage, where additional information is encoded in the selected features.",{"name":83,"@type":74,"acceptedAnswer":84},"What is SUWR and what results does it show?",{"text":85,"@type":77},"SUWR is the first local feature selection method proven to have no leakage. Experiments indicate it is less prone to overfitting while maintaining strong predictive performance and high feature-selection sparsity.","https://schema.org",{"og:url":52,"og:type":88,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":90,"canonical":52},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":93},[94,98,102,106,111,116,121,124,129,132,136],{"id":20,"doc_module":4,"doc_module_name":46,"category_name":95,"show_sort_weight":96,"slug":97},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":99,"show_sort_weight":100,"slug":101},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":46,"category_name":103,"show_sort_weight":104,"slug":105},"Exam",70,"exam",{"id":107,"doc_module":4,"doc_module_name":46,"category_name":108,"show_sort_weight":109,"slug":110},5,"Comic",60,"comic",{"id":112,"doc_module":4,"doc_module_name":46,"category_name":113,"show_sort_weight":114,"slug":115},6,"Technology",50,"technology",{"id":117,"doc_module":4,"doc_module_name":46,"category_name":118,"show_sort_weight":119,"slug":120},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":122,"slug":123},30,"research-report",{"id":125,"doc_module":4,"doc_module_name":46,"category_name":126,"show_sort_weight":127,"slug":128},9,"Religion & Spirituality",20,"religion-spirituality",{"id":127,"doc_module":4,"doc_module_name":46,"category_name":130,"show_sort_weight":127,"slug":131},"World Cup","world-cup",{"id":133,"doc_module":4,"doc_module_name":46,"category_name":134,"show_sort_weight":133,"slug":135},10,"Lifestyle","lifestyle",{"id":137,"doc_module":4,"doc_module_name":46,"category_name":138,"show_sort_weight":107,"slug":139},19,"General","general"]