[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-124878-en":3,"doc-seo-124878-105":30,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":4,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":27,"seo_description":14,"update_tm":28,"read_time":29},124878,137441390410,"Hazel","https://ap-avatar.wpscdn.com/avatar/2000252f4ab5702993?_k=1776741390130283984",8,"Research & Report","Reconstructing a Machine Learning Model Based on Thresholded Model Differences - technical disclosure","A machine learning model typically contains many floating-point weights and an associated computation graph, making it expensive to duplicate storage when one model is derived from another. The approach constructs a sparse representation, thresholded model diff, from reversible reductions of weight differences between a fine-tuned derived model and its base model. The diff can be optionally thresholded, selectively applied to the base model, and iteratively adjusted via evaluation to preserve task performance.","Technical Disclosure Commons  \nDefensive Publications Series  \nDecember 2023  \nReconstructing a Machine Learning Model Based on Thresholded Model Differences  \nNam Nguyen Jeffrey Hui Aroma Mahendru  \nFollow this and additional works at: [https://www.tdcommons.org/dpubs_series](https://www.tdcommons.org/dpubs_series)  \nRecommended Citation  \nNguyen, Nam; Hui, Jeffrey; and Mahendru, Aroma, \"Reconstructing a Machine Learning Model Based on Thresholded Model Differences\", Technical Disclosure Commons,(December 05, 2023)  \n[https://www.tdcommons.org/dpubs_series/6476](https://www.tdcommons.org/dpubs_series/6476)  \nThis work is licensed under a Creative Commons Attribution 4.0 License.  \nThis Article is brought to you for free and open access by Technical Disclosure Commons. It has been accepted for inclusion in Defensive Publications Series by an authorized administrator of Technical Disclosure Commons.  \nReconstructing a Machine Learning Model Based on Thresholded Model Differences  \nABSTRACT  \nA machine learning (ML) model often comprises many floating point numbers (model weights) and the operations applied on them (the computation graph) . Ifa model is derived from another, the derived model often has similar weights and is of the same size. It is a waste of storage to store copies of the similar weights and computation graphs for two related models. This document describes techniques to obtain a sparse representation, referred to herein as thresholded model diff, that can be applied to a base model toreconstruct a version of a derived model. Differences between weights ofthe base model anda derived model obtained by fine-tuning the base model are identified and reduced with reversible operations. The reduced differences are (optionally) subjected to thresholding to obtain the thresholded model diff. A reconstructed model is obtained by selectively applying the thresholded model diff to the base model. The reconstructed model is evaluated to ensure that it can adequately perform the task that the derived model is fine-tuned for. The thresholding and application of the diff to the base model is adjusted based on the evaluation.  \nKEYWORDS  \n● Base model  \n● Derived model  \n● Model diff  \n● Model compression  \n● Model weights  \n● Parameter thresholding  \n● Model fine-tuning  \n● Customized model  \n● Machine learning  \nPublished by Technical Disclosure Commons, 2023 2  \nBACKGROUND  \nA machine learning (ML) model often comprises many floating point numbers (model weights) and the operations applied on them (the computation graph). Many ML  \nmodels can be very large in size, running into gigabytes of compressed data. Ifa model is  \nderived from another, the derived model often has similar weights and is of the same size. It  \nis a waste of storage to store copies of the similar weights (a portion of the weights may be  \nthe same across the two models) and computation graphs for two related models.  \nSome techniques (as described in [1]) can reduce the number of trainable parameters by several orders of magnitude and can also reduce memory requirements for the model. These techniques create a small adaptation model of selective layers in the base model. At inference time, the adaptation overrides the layers in the base model. However, such techniques require a totally new training method to derive the adaptation and work only on selective layers, not the whole model. Another technique is to use a deterministic first-order weight pruning method (as described in [2]) that is more adaptive to pretrained model fine-tuning. When pruning large  \npretrained language models, such methods show improvement in high sparsity regimes.  \nHowever, these techniques do not take into account the similarity between two related models.  \nDESCRIPTION  \nThis document describes techniques to obtain a sparse representation, referred to herein  \nas thresholded model diff, that can be applied to a base model to reconstruct a version of a  \nderived model. Diffe","cbCaidavhgYbt5He","https://ap.wps.com/l/cbCaidavhgYbt5He","pdf",211333,1,10,"English","en",105,"# Abstract\n# Background\n# Description\n## Reconstructing via thresholded model diff\n## Fine-tuning and evaluation","[{\"question\":\"What is thresholded model diff and what problem does it solve?\",\"answer\":\"Thresholded model diff is a sparse representation of differences between a base model and a derived model. It reduces storage waste by avoiding full copies of similar weights and computation graphs across related models.\"},{\"question\":\"How is a reconstructed model obtained from a base model?\",\"answer\":\"Weight differences are identified between the base model and a fine-tuned derived model, then reduced using reversible operations. Optionally, the reduced differences are thresholded to form the thresholded model diff, which is selectively applied to the base model to reconstruct the derived version.\"},{\"question\":\"How does the method ensure the reconstructed model performs the intended task?\",\"answer\":\"The reconstructed model is evaluated to confirm it can adequately perform the task for which the derived model was fine-tuned. Thresholding and diff application are then adjusted based on the evaluation results.\"}]","Reconstructing a Machine Learning Model Based on Thresholded Model Differences - technical disclosure | PDF",1785895176,25,{"code":4,"msg":31,"data":32},"ok",{"site_id":24,"language":23,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":28},"reconstructing-a-machine-learning-model-based-on-thresholded-model-differences-technical-disclosure","",{"@graph":36,"@context":85},[37,54,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":53},"https://docshare.wps.com/document/reconstructing-a-machine-learning-model-based-on-thresholded-model-differences-technical-disclosure/124878/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":23,"description":14,"dateModified":62,"datePublished":62,"encodingFormat":61,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":41,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-05",true,{"@type":65,"interactionType":66,"userInteractionCount":4},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"What is thresholded model diff and what problem does it solve?","Question",{"text":75,"@type":76},"Thresholded model diff is a sparse representation of differences between a base model and a derived model. It reduces storage waste by avoiding full copies of similar weights and computation graphs across related models.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"How is a reconstructed model obtained from a base model?",{"text":80,"@type":76},"Weight differences are identified between the base model and a fine-tuned derived model, then reduced using reversible operations. Optionally, the reduced differences are thresholded to form the thresholded model diff, which is selectively applied to the base model to reconstruct the derived version.",{"name":82,"@type":73,"acceptedAnswer":83},"How does the method ensure the reconstructed model performs the intended task?",{"text":84,"@type":76},"The reconstructed model is evaluated to confirm it can adequately perform the task for which the derived model was fine-tuned. Thresholding and diff application are then adjusted based on the evaluation results.","https://schema.org",{"og:url":52,"og:type":87,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":89,"canonical":52},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,134],{"id":20,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":46,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":46,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":46,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":46,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":21,"doc_module":4,"doc_module_name":46,"category_name":132,"show_sort_weight":21,"slug":133},"Lifestyle","lifestyle",{"id":135,"doc_module":4,"doc_module_name":46,"category_name":136,"show_sort_weight":106,"slug":137},19,"General","general"]