[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-122649-en":3,"doc-seo-122649-105":30,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":4,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":27,"seo_description":14,"update_tm":28,"read_time":29},122649,687197100911,"Himbo","https://ap-avatar.wpscdn.com/avatar/a000239b6f1da00475?x-image-process=image/resize,m_fixed,w_180,h_180&k=1785132997149421697",8,"Research & Report","Interpretable Differencing of Machine Learning Models - paper summary","Understanding how two machine learning models differ matters when selecting between competing models or updating a deployed model with new training data. The work formulates model differencing as learning a dissimilarity function over pairs of model outputs, but requires that the representation remain human-interpretable. It introduces a Joint Surrogate Tree (JST) built from conjoined decision-tree surrogates to show differences in the context of decision logic. A refinement procedure further improves precision, and empirical results indicate contextual differencing stays concise without losing fidelity versus naive baselines.","Interpretable Differencing of Machine Learning Models  \nSwagatam Haldar 1 Diptikalyan Saha 1 Dennis Wei2 Rahul Nair3 Elizabeth M. Daly3  \n1IBM Research, Bangalore, India  \n2IBM Research, Yorktown Heights, New York, USA,  \n3IBM Research, Dublin, Ireland  \narXiv :2306 .06473v2 [ cs .LG] 13 Jun 2023  \nAbstract  \nUnderstanding the differences between machine learning (ML) models is of interest in scenarios ranging from choosing amongst a set of competing models, to updating a deployed model with new training data. In these cases, we wish to go beyond differences in overall metrics such as accuracy to identify where in the feature space do the differences occur. We formalize this problem of model differencing as one of predicting a dissimilarity function of two ML models’ outputs, subject to the representation of the differences being human-interpretable. Our solution is to learn a Joint Surrogate Tree (JST), which is composed of two conjoined decision tree surrogates for the two models. A JST provides an intuitive representation of differences and places the changes in the context of the models’ decision logic. Context is important as it helps users to map differences to an underlying mental model of an AI system. We also propose a refinement procedure to increase the precision of a JST. We demonstrate, through an empirical evaluation, that such contextual differencing is concise and can be achieved with no loss in fidelity over naive approaches.  \n1 INTRODUCTION  \nAt various stages of the AI model lifecycle, data scientists make decisions regarding which model to use. For instance, they may choose from a range of pre-built models, select from a list of candidate models generated from automated tools like AutoML, or simply update a model based on new training data to incorporate distributional changes. In these settings, the choice of a model is preceded by an evaluation that typically focuses on accuracy and other metrics, instead of how it differs from other models.  \nWe address the problem of model differencing. Given two models for the same task and a dataset, we seek to learn where in the feature space the models’ predicted outcomes differ. Our objective is to provide accurate and interpretable mechanisms to uncover these differences.  \nThe comparison is helpful in several scenarios. In a model marketplace, multiple pre-built models for the same task need to be compared. The models usually are black-box and possibly trained on different sets of data drawn from the same distribution. During model selection, a data scientist trains multiple models and needs to select one model for deployment. In this setting, the models are white-box and typically trained on the same training data. For model change, where a model is retrained with updated training data with a goal towards model improvement, the data scientist needs to understand changes in the model beyond accuracy metrics. Finally, decision pipelines consisting of logic and ML models occur in business contexts where a combination of business logic and the output of ML models work together for a final output. Changes might occur either due to model retraining or adjustments in business logic which can impact the behavior of the overall pipeline.  \nIn this work we address the problem of interpretable model differencing as follows. First, we formulate the problem as one of predicting the values of a dissimilarity function of the two models’ outputs. We focus herein on 0-1 dissimilarity for two classifiers, where 0 means “same output” and 1 means “different”, so that prediction quality can be quantified by any binary classification metric such as precision and recall. Second, we propose a method that learns a Joint Surrogate Tree (JST), composed of two conjoined decision tree surrogates to jointly approximate the two models. The root and lower branches of the conjoined decision trees are common to both models, while higher branches (farther from root) may be specific to one model.","cbCaidYLgkZRNZXO","https://ap.wps.com/l/cbCaidYLgkZRNZXO","pdf",1434031,1,40,"English","en",105,"# Introduction\n## Problem of model differencing\n## Interpretable formulation with dissimilarity prediction\n# Related Works\n## Surrogate models and model refinement","[{\"question\":\"What problem does interpretable model differencing address?\",\"answer\":\"Given two models for the same task and a dataset, it aims to identify where in the feature space their predicted outcomes differ, beyond comparing overall accuracy metrics.\"},{\"question\":\"How does the Joint Surrogate Tree (JST) help make differences interpretable?\",\"answer\":\"A JST uses two conjoined decision-tree surrogates whose shared lower branches align the models, while higher branches can be model-specific. This visualization places differences within the models’ decision logic.\"},{\"question\":\"What is the role of the refinement procedure in JST learning?\",\"answer\":\"The method grows surrogates selectively in regions that improve the precision of dissimilarity prediction, increasing how accurately and precisely differences are represented.\"}]","Interpretable Differencing of Machine Learning Models - paper summary | PDF",1785811938,101,{"code":4,"msg":31,"data":32},"ok",{"site_id":24,"language":23,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":28},"interpretable-differencing-of-machine-learning-models-paper-summary","",{"@graph":36,"@context":85},[37,54,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":53},"https://docshare.wps.com/document/interpretable-differencing-of-machine-learning-models-paper-summary/122649/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":23,"description":14,"dateModified":62,"datePublished":62,"encodingFormat":61,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":41,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-04",true,{"@type":65,"interactionType":66,"userInteractionCount":4},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"What problem does interpretable model differencing address?","Question",{"text":75,"@type":76},"Given two models for the same task and a dataset, it aims to identify where in the feature space their predicted outcomes differ, beyond comparing overall accuracy metrics.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"How does the Joint Surrogate Tree (JST) help make differences interpretable?",{"text":80,"@type":76},"A JST uses two conjoined decision-tree surrogates whose shared lower branches align the models, while higher branches can be model-specific. This visualization places differences within the models’ decision logic.",{"name":82,"@type":73,"acceptedAnswer":83},"What is the role of the refinement procedure in JST learning?",{"text":84,"@type":76},"The method grows surrogates selectively in regions that improve the precision of dissimilarity prediction, increasing how accurately and precisely differences are represented.","https://schema.org",{"og:url":52,"og:type":87,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":89,"canonical":52},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,119,122,127,130,134],{"id":20,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":46,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":46,"category_name":117,"show_sort_weight":21,"slug":118},7,"Healthcare","healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":120,"slug":121},30,"research-report",{"id":123,"doc_module":4,"doc_module_name":46,"category_name":124,"show_sort_weight":125,"slug":126},9,"Religion & Spirituality",20,"religion-spirituality",{"id":125,"doc_module":4,"doc_module_name":46,"category_name":128,"show_sort_weight":125,"slug":129},"World Cup","world-cup",{"id":131,"doc_module":4,"doc_module_name":46,"category_name":132,"show_sort_weight":131,"slug":133},10,"Lifestyle","lifestyle",{"id":135,"doc_module":4,"doc_module_name":46,"category_name":136,"show_sort_weight":106,"slug":137},19,"General","general"]