[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"detail-sidebar-cat-0-en-105":3,"doc-seo-290501-105":59,"doc-detail-290501-en":131},{"code":4,"msg":5,"data":6},0,"success",[7,13,18,23,28,33,38,43,48,51,55],{"id":8,"doc_module":4,"doc_module_name":9,"category_name":10,"show_sort_weight":11,"slug":12},1,"Document","Story & Novel",90,"story-novel",{"id":14,"doc_module":4,"doc_module_name":9,"category_name":15,"show_sort_weight":16,"slug":17},2,"Literature",80,"literature",{"id":19,"doc_module":4,"doc_module_name":9,"category_name":20,"show_sort_weight":21,"slug":22},4,"Exam",70,"exam",{"id":24,"doc_module":4,"doc_module_name":9,"category_name":25,"show_sort_weight":26,"slug":27},5,"Comic",60,"comic",{"id":29,"doc_module":4,"doc_module_name":9,"category_name":30,"show_sort_weight":31,"slug":32},6,"Technology",50,"technology",{"id":34,"doc_module":4,"doc_module_name":9,"category_name":35,"show_sort_weight":36,"slug":37},7,"Healthcare",40,"healthcare",{"id":39,"doc_module":4,"doc_module_name":9,"category_name":40,"show_sort_weight":41,"slug":42},8,"Research & Report",30,"research-report",{"id":44,"doc_module":4,"doc_module_name":9,"category_name":45,"show_sort_weight":46,"slug":47},9,"Religion & Spirituality",20,"religion-spirituality",{"id":46,"doc_module":4,"doc_module_name":9,"category_name":49,"show_sort_weight":46,"slug":50},"World Cup","world-cup",{"id":52,"doc_module":4,"doc_module_name":9,"category_name":53,"show_sort_weight":52,"slug":54},10,"Lifestyle","lifestyle",{"id":56,"doc_module":4,"doc_module_name":9,"category_name":57,"show_sort_weight":24,"slug":58},19,"General","general",{"code":4,"msg":60,"data":61},"ok",{"site_id":62,"language":63,"slug":64,"title":65,"keywords":66,"description":67,"schema_data":68,"social_meta":124,"head_meta":126,"extra_data":128,"updated_unix":130},105,"en","outlier-robust-training-of-machine-learning-models","Outlier-Robust Training of Machine Learning Models","","Robust training of machine learning models under outliers has become important across many application areas. This work contrasts two design traditions for robust losses: M-estimation, widely used in robotics and computer vision, and risk-minimization, common in deep learning. A modified Black-Rangarajan duality yields a unified robust-loss kernel definition shared across both literatures. An Adaptive Alternation Algorithm (AAA) iteratively trains with a weighted non-robust loss while updating weights as inlier probabilities, reducing complex parameter tuning. Convergence results show improved regions toward outlier-free optima for arbitrary outliers, supported by experiments in regression, classification, and neural scene reconstruction.",{"@graph":69,"@context":123},[70,84,106],{"@type":71,"itemListElement":72},"BreadcrumbList",[73,77,79,82],{"item":74,"name":75,"@type":76,"position":8},"https://docshare.wps.com","Home","ListItem",{"item":78,"name":9,"@type":76,"position":14},"https://docshare.wps.com/document/",{"item":80,"name":40,"@type":76,"position":81},"https://docshare.wps.com/document/research-report/",3,{"item":83,"name":65,"@type":76,"position":19},"https://docshare.wps.com/document/outlier-robust-training-of-machine-learning-models/290501/",{"url":83,"name":65,"@type":85,"image":86,"author":91,"headline":65,"publisher":94,"fileFormat":97,"inLanguage":63,"description":67,"dateModified":98,"datePublished":99,"encodingFormat":97,"isAccessibleForFree":100,"interactionStatistic":101},"DigitalDocument",{"url":87,"@type":88,"width":89,"height":90},"https://docshare.wps.com/thumbnails/outlier-robust-training-of-machine-learning-models/290501.png","ImageObject",300,407,{"name":92,"@type":93},"Noah","Person",{"url":74,"name":95,"@type":96},"DocShare","Organization","application/pdf","2026-11-08","2026-09-17",true,{"@type":102,"interactionType":103,"userInteractionCount":105},"InteractionCounter",{"@type":104},"ViewAction",14,{"@type":107,"mainEntity":108},"FAQPage",[109,115,119],{"name":110,"@type":111,"acceptedAnswer":112},"What problem does the paper address about training machine learning models?","Question",{"text":113,"@type":114},"It addresses robust training when data contains outliers that can degrade the quality of model parameters and predictions.","Answer",{"name":116,"@type":111,"acceptedAnswer":117},"How does the paper unify two existing robust-loss design approaches?",{"text":118,"@type":114},"It uses a simple modification of the Black-Rangarajan duality to derive a shared definition of a robust loss kernel satisfied by robust losses from both literatures.",{"name":120,"@type":111,"acceptedAnswer":121},"What is the Adaptive Alternation Algorithm (AAA) and what advantage does it provide?",{"text":122,"@type":114},"AAA alternates between training with a weighted version of a non-robust loss and updating weights at each iteration. Interpreting weights as inlier probabilities introduces a parameter update rule that avoids complex parameter tuning.","https://schema.org",{"og:url":83,"og:type":125,"og:title":65,"og:site_name":95,"og:description":67},"article",{"robots":127,"canonical":83},"index,follow",{"doc_id":129,"site_id":62},290501,1789641850,{"code":4,"msg":5,"data":132},{"doc_id":129,"user_id":133,"nickname":92,"user_avatar":134,"doc_module":4,"category_id":39,"category_name":40,"doc_title":65,"doc_description":67,"doc_content":135,"file_id":136,"file_url":137,"file_type":138,"file_size":139,"view_count":105,"is_deleted":4,"is_public":8,"is_downloadable":8,"audit_status":8,"page_count":140,"language":141,"language_code":63,"site_id":62,"html_lang":63,"table_of_contents":142,"faqs":143,"seo_title":144,"seo_description":67,"update_tm":130,"read_time":145,"preview_page_count":4},8796095462418,"https://ap-avatar.wpscdn.com/avatar/80000253c1241d02b47?x-image-process=image/resize,m_fixed,w_180,h_180&k=1778826106357471780","arXiv :2501 .00265v1 [ cs .LG] 31 Dec 2024  \nOutlier-Robust Training of Machine Learning Models  \nRajat Talak [talak@mit. edu](talak@mit. edu)  \n[Laboratory of Information & Decision Systems](Laboratory of Information & Decision Systems)[ ](Laboratory of Information & Decision Systems)Massachusetts Institute of Technology Cambridge, MA 02139, USA  \nCharis Georgiou [cgeo@mit. edu](cgeo@mit. edu)  \nDepartment of Electrical Engineering and Computer Science Massachusetts Institute of Technology  \nCambridge, MA 02139, USA  \nJingnan Shi [jnshi@mit. edu](jnshi@mit. edu)  \n[Laboratory of Information & Decision Systems](Laboratory of Information & Decision Systems)[ ](Laboratory of Information & Decision Systems)Massachusetts Institute of Technology Cambridge, MA 02139, USA  \nLuca Carlone [lcarlone@mit. edu](lcarlone@mit. edu)  \nLaboratory of Information & Decision Systems Massachusetts Institute of Technology Cambridge, MA 02139, USA  \nAbstract  \nRobust training of machine learning models in the presence of outliers has garnered attention across various domains. The use of robust losses is a popular approach and is known to mitigate the impact of outliers. We bring to light two literatures that have diverged in their ways of designing robust losses: one using M-estimation, which is popular in robotics and computer vision, and another using a risk-minimization framework, which is popular in deep learning. We first show that a simple modification of the Black-Rangarajan duality provides a unifying view. The modified duality brings out a definition of a robust loss kernel σ that is satisfied by robust losses in both the literatures. Secondly, using the modified duality, we propose an Adaptive Alternation Algorithm (AAA) for training machine learning models with outliers. The algorithm iteratively trains the model by using a weighted version of thenon-robust loss, while updating the weights at each iteration. The algorithm is augmented with a novel parameter update rule by interpreting the weights as inlier probabilities, and obviates the need for complex parameter tuning. Thirdly, we investigate convergence of the adaptive alternation algorithm to outlier-free optima. Considering arbitrary outliers (i. e . , with no distributional assumption on the outliers), we show that the use of robust loss kernels σ increases the region of convergence. We experimentally show the efficacy of our algorithm on regression, classification, and neural scene reconstruction problems. 1  \n1 Introduction  \nHumans are good at detecting and isolating outliers (Chai et al., 2020) . This is not the case when it comes to training machine learning models (Sukhbaatar et al., 2015; Wang et al., 2024a; Sabour et al., 2023) . Robustly training deep learning models in the presence of outliers is an important challenge. In particular, it can offset the high cost of obtaining accurate annotations. Many works now implement automatic or semi-automatic annotation pipelines which can be leveraged to train models (Armeni et al., 2016; Changet al., 2017; Tkachenko et al., 2020; Yang et al., 2021; Gadre et al., 2023) . Recent efforts in robotics envision  \n1 We release our implementation code: [https://github.com/MIT-SPARK/ORT](https://github.com/MIT-SPARK/ORT).  \nrobots that can self-train their models by collecting and self-annotating data (Schmidt & Fox, 2020; Denget al., 2020; Lu et al., 2022; Talak et al., 2023; Shi et al., 2023; Jawaid et al., 2024; Wang et al., 2024b) .  \nIn typical learning problems one computes the unknowns (e.g. , network weights) by optimizing a loss function fi for each training sample i:  \nMinimize w∈W  \nn  \nX fi (w), (1)  \ni=1  \nwhere W is the set of allowed parameters. For instance, fi (·) may be the cross-entropy loss orthe ℓ2 norm squared measuring the mismatch between the i-th training label and the corresponding network prediction.  \nM-estimation (Huber, 1981) suggests that in the presence of outliers, one needs to wrap typical losses into a r","cbCaistRF5k5mNwa","https://ap.wps.com/l/cbCaistRF5k5mNwa","pdf",3618444,31,"English","# Abstract\n# Introduction\n## Robust training and outliers\n## Robust losses via M-estimation\n## Risk-minimization in deep learning","[{\"question\":\"What problem does the paper address about training machine learning models?\",\"answer\":\"It addresses robust training when data contains outliers that can degrade the quality of model parameters and predictions.\"},{\"question\":\"How does the paper unify two existing robust-loss design approaches?\",\"answer\":\"It uses a simple modification of the Black-Rangarajan duality to derive a shared definition of a robust loss kernel satisfied by robust losses from both literatures.\"},{\"question\":\"What is the Adaptive Alternation Algorithm (AAA) and what advantage does it provide?\",\"answer\":\"AAA alternates between training with a weighted version of a non-robust loss and updating weights at each iteration. Interpreting weights as inlier probabilities introduces a parameter update rule that avoids complex parameter tuning.\"}]","Outlier-Robust Training of Machine Learning Models | PDF",78]