[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-117877-en":3,"doc-seo-117877-105":30,"detail-sidebar-cat-0-en-105":83},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":4,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":27,"seo_description":14,"update_tm":28,"read_time":29},117877,34359740700684,"Finn","https://ap-avatar.wpscdn.com/avatar/1f400023980c374ae676?_k=1777273430885731487",8,"Research & Report","Distributionally Robust Machine Learning with Multi-source Data - slideshare_146406946","Classical machine learning can fail under distribution shift between training and target populations. This work proposes a group distributionally robust prediction model using multi-source data, optimizing an adversarial reward about explained variance over a class of target distributions. The method yields improved target prediction accuracy versus empirical risk minimization and shows the robust predictor forms a weighted average of sources’ conditional outcome models. It further provides a bias-corrected estimator to learn optimal aggregation weights for general algorithms, demonstrating better convergence rates. Applied to random forests and neural networks on simulated and real data, the approach is efficient, interpretable, and compatible with federated-learning-style implementation with some privacy constraints.","arXiv :2309 .02211v1 [ stat .ML] 5 Sep 2023  \nDistributionally Robust Machine Learning with Multi-source Data ∗  \nZhenyu Wang 1 , Peter B¨uhlmann2 , and Zijian Guo 1  \n1 Department of Statistics, Rutgers University, USA  \n2 Seminar for Statistics, ETH Z¨urich, Switzerland  \nAbstract  \nClassical machine learning methods may lead to poor prediction performance when the target distribution differs from the source populations. This paper utilizes data from multiple sources and introducesa group distributionally robust prediction model defined to optimize an adversarial reward about explained variance with respect to a class of target distributions. Compared to classical empirical risk minimization, the proposed robust prediction model improves the prediction accuracy for target populations with distribution shifts. We show that our group distributionally robust prediction model is a weighted average of the source populations’ conditional outcome models. We leverage this key identification result to robustify arbitrary machine learning algorithms, including, for example, random forests and neural networks. We devise a novel bias-corrected estimator to estimate the optimal aggregation weight for general machine-learning algorithms and demonstrate its improvement in the convergence rate. Our proposal can be seen as a distributionally robust federated learning approach that is computationally efficient and easy to implement using arbitrary machine learning base algorithms, satisfies some privacy constraints, and has a nice interpretation of different sources’ importance for predicting a given target covariate distribution. We demonstrate the performance of our proposed group distributionally robust method on simulated and real data with random forests and neural networks as base-learning algorithms.  \nKey words: federated learning; interpretable machine learning; distributionally robust random forests; distributionally robust deep neural network; minimax optimization  \n1 Introduction  \nA fundamental assumption for the success of machine learning algorithms is that the training and test datasets share the same generating distribution. However, in many applications, the underlying distribution of the test data may exhibit a shift from that of the training data, which might be due to changing environments or data being collected at different times and locations [Quinonero-Candela et al. , 2008 , Koh et al. , 2021 , Malinin et al. , 2021 , Nado et al. , 2021] . Such distributional shifts may lead to most machine learning algorithms having a poor or unstable prediction performance for the test data even when the algorithms are fine-tuned on the training data. It is critical yet challenging to construct a generalizable machine learning algorithm that guarantees excellent prediction performance even in the presence of distributional shifts.  \n∗ The research of Z. Wang and Z. Guo was partly supported by the NSF grant DMS 2015373 and NIH grants R01GM140463 and R01LM013614 . P. B¨uhlmann received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation program (grant agreement No. 786461)  \nDistributionally robust optimization (DRO) has proven effective in improving the prediction model’s generalizability for unseen target populations; see Rahimian and Mehrotra [2019], Namkoong and Duchi [2017], Sinha et al. [2017], Ben-Tal et al. [2013], Bertsimas et al. [2018], Blanchet et al. [2019b,a], Gao and Kleywegt [2023], Gao et al. [2022], Kuhn et al. [2019] for examples. The main idea of DRO is to incorporate the distributional uncertainty of the test data into the optimization process, with the crucial operational step of minimizing the adversarial loss defined over a class of target distributions centered around the training data’s empirical distribution.  \nSince DRO can be overly pessimistic in practice, the group DRO was proposed as a more accurate prediction model when analyzing the data co","cbCaieclAztBiLv0","https://ap.wps.com/l/cbCaieclAztBiLv0","pdf",1028960,1,37,"English","en",105,"# Abstract\n# 1 Introduction\n## 1.1 Our results and contribution","[{\"question\":\"What key characterization does the paper provide for the robust predictor?\",\"answer\":\"The paper shows the group distributionally robust predictor can be expressed as a weighted average of the source populations’ conditional outcome models, which enables further robustification of general algorithms.\"}]","Distributionally Robust Machine Learning with Multi-source Data - slideshare_146406946 | PDF",1785680109,93,{"code":4,"msg":31,"data":32},"ok",{"site_id":24,"language":23,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":78,"head_meta":80,"extra_data":82,"updated_unix":28},"distributionally-robust-machine-learning-with-multi-source-data-slideshare_146406946","",{"@graph":36,"@context":77},[37,54,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":53},"https://docshare.wps.com/document/distributionally-robust-machine-learning-with-multi-source-data-slideshare_146406946/117877/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":23,"description":14,"dateModified":62,"datePublished":62,"encodingFormat":61,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":41,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-02",true,{"@type":65,"interactionType":66,"userInteractionCount":4},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71],{"name":72,"@type":73,"acceptedAnswer":74},"What key characterization does the paper provide for the robust predictor?","Question",{"text":75,"@type":76},"The paper shows the group distributionally robust predictor can be expressed as a weighted average of the source populations’ conditional outcome models, which enables further robustification of general algorithms.","Answer","https://schema.org",{"og:url":52,"og:type":79,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":81,"canonical":52},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":84},[85,89,93,97,102,107,112,115,120,123,127],{"id":20,"doc_module":4,"doc_module_name":46,"category_name":86,"show_sort_weight":87,"slug":88},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":90,"show_sort_weight":91,"slug":92},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Exam",70,"exam",{"id":98,"doc_module":4,"doc_module_name":46,"category_name":99,"show_sort_weight":100,"slug":101},5,"Comic",60,"comic",{"id":103,"doc_module":4,"doc_module_name":46,"category_name":104,"show_sort_weight":105,"slug":106},6,"Technology",50,"technology",{"id":108,"doc_module":4,"doc_module_name":46,"category_name":109,"show_sort_weight":110,"slug":111},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":113,"slug":114},30,"research-report",{"id":116,"doc_module":4,"doc_module_name":46,"category_name":117,"show_sort_weight":118,"slug":119},9,"Religion & Spirituality",20,"religion-spirituality",{"id":118,"doc_module":4,"doc_module_name":46,"category_name":121,"show_sort_weight":118,"slug":122},"World Cup","world-cup",{"id":124,"doc_module":4,"doc_module_name":46,"category_name":125,"show_sort_weight":124,"slug":126},10,"Lifestyle","lifestyle",{"id":128,"doc_module":4,"doc_module_name":46,"category_name":129,"show_sort_weight":98,"slug":130},19,"General","general"]