[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-124664-en":3,"doc-seo-124664-105":29,"detail-sidebar-cat-0-en-105":90},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":11,"language":21,"language_code":22,"site_id":23,"html_lang":22,"table_of_contents":24,"faqs":25,"seo_title":26,"seo_description":14,"update_tm":27,"read_time":28},124664,962075006959,"Anda","https://ap-avatar.wpscdn.com/avatar/e0002397efbe92a78e?_k=1776741047341049297",8,"Research & Report","Model Reprogramming - Resource-Efficient Cross-Domain Machine Learning","Deep learning dominates in data-rich domains like vision, language, and speech, yet resource-limited settings still struggle with scarce data, high model development cost, and insufficient pre-trained models for effective fine-tuning. Model reprogramming bridges this gap by reusing a well-developed pre-trained model from a source domain to solve target-domain tasks without fine-tuning, freezing the original model parameters. It can outperform transfer learning and training from scratch, with a clear methodology, summarized applications, theory-backed explanations, and open research directions.","Model Reprogramming: Resource-Efficient Cross-Domain Machine Learning  \nPin-Yu Chen 1  \n1IBM Research  \n[pin-yu.chen@ibm.com](pin-yu.chen@ibm.com)  \narXiv :2202 . 10629v3 [ cs .LG] 13 Sep 2023  \nAbstract  \nIn data-rich domains such as vision, language, and speech, deep learning prevails to deliver high-performance taskspecific models and can even learn general task-agnostic representations for efficient finetuning to downstream tasks. However, deep learning in resource-limited domains still faces multiple challenges including (i) limited data,(ii) constrained model development cost, and (iii) lack of adequate pre-trained models for effective finetuning. This paper provides an overview of model reprogramming to bridge this gap. Model reprogramming enables resource-efficient crossdomain machine learning by repurposing and reusing a welldeveloped pre-trained model from a source domain to solve tasks in a target domain without model finetuning, where the source and target domains can be vastly different. In many applications, model reprogramming outperforms transfer learning and training from scratch. This paper elucidates the methodology of model reprogramming, summarizes existing use cases, provides a theoretical explanation of the success of model reprogramming, and concludes with a discussion on open-ended research questions and opportunities. A list of model reprogramming studies is actively maintained and updated at [https://github.com/IBM/model-reprogramming](https://github.com/IBM/model-reprogramming).  \n1 Introduction  \nDesigning and developing a top-notch deep learning model is a time-consuming and costly process. It is no secret that employing a high-capacity neural network model consisting of a tremendous number of trainable parameters, together with a proper selection of the network architecture and hyperparameter optimization, can lead to state-of-theart machine learning performance when trained on a massive amount of data. Take the Generative Pre-trained Transformer 3 (GPT-3) (Brown et al. 2020) as an example, which is one of the largest language models ever trained to date. GPT-3 has 175 billion parameters and is trained on a dataset consisting of 499 Billion tokens. The estimated training cost is about 4.6 Million US dollars even with the lowest priced GPU cloud on the market in 2020 1. Such a large-scale language model is shown to be effective when applied to several downstream language-related tasks in the same domain (source). However, having invested so much to obtain a topnotch model, one interesting question to ask is: Can we reuse  \n1See [https://lambdalabs.com/blog/demystifying-gpt-3](https://lambdalabs.com/blog/demystifying-gpt-3)  \nFigure 1: Visual illustration of data scale (bottom to top: small to large) and number (\\#) of trainable parameters (left to right: small to large) in different machine learning paradigms. We note that the visualization does not reflect the actual relative differences due to excessively varying orders. A foundation model like GPT-3 has 175 billion trainable parameters and 499 billion tokens as training data. The trainable parameters in model reprogramming can be as few as the size of the data input (e.g., the number of image pixels can be in the order of thousands or fewer), and model reprogramming is particularly suited to small-scale data regime. In model reprogramming, the visualization does not take into account the pre-trained source model because it is kept intact and unchanged. The dashed box in transfer learning means variations in the number of model parameters used for fine-tuning, ranging from only training the last dense layer (linear head) to fine-tuning all parameters. The number of training epochs may also vary for each paradigm.  \nthis valuable asset for machine learning in another domain (target), especially in the resource-limited setting when at least one of the following scenarios is concerned: (i) lack of high-quality pre-trained models in the target domai","cbCaila4MRA9Wkuw","https://ap.wps.com/l/cbCaila4MRA9Wkuw","pdf",347269,1,"English","en",105,"# Abstract\n# Introduction\n## Motivation and challenges in resource-limited domains\n## Core rationale of model reprogramming\n## Figure-based comparison of paradigms\n## Training requirements and parameter efficiency","[{\"question\":\"What problem does model reprogramming aim to solve?\",\"answer\":\"It targets resource-limited scenarios where there is limited target-domain data, constrained model development cost, or inadequate pre-trained models for fine-tuning. The goal is to enable effective cross-domain learning with minimal additional training.\"},{\"question\":\"How does model reprogramming work without fine-tuning the source model?\",\"answer\":\"It keeps the pre-trained source model intact and introduces an input transformation layer and an output mapping layer. Only these added components are trained, while the source model parameters are frozen.\"},{\"question\":\"When can model reprogramming outperform transfer learning or training from scratch?\",\"answer\":\"It is especially effective in small-scale data regimes and cases where target-domain pre-trained models are missing or training resources are constrained. The approach leverages reusable representations from data-rich domains.\"}]","Model Reprogramming - Resource-Efficient Cross-Domain Machine Learning | PDF",1785893673,20,{"code":4,"msg":30,"data":31},"ok",{"site_id":23,"language":22,"slug":32,"title":13,"keywords":33,"description":14,"schema_data":34,"social_meta":85,"head_meta":87,"extra_data":89,"updated_unix":27},"model-reprogramming-resource-efficient-cross-domain-machine-learning","",{"@graph":35,"@context":84},[36,53,67],{"@type":37,"itemListElement":38},"BreadcrumbList",[39,43,47,50],{"item":40,"name":41,"@type":42,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":44,"name":45,"@type":42,"position":46},"https://docshare.wps.com/document/","Document",2,{"item":48,"name":12,"@type":42,"position":49},"https://docshare.wps.com/document/research-report/",3,{"item":51,"name":13,"@type":42,"position":52},"https://docshare.wps.com/document/model-reprogramming-resource-efficient-cross-domain-machine-learning/124664/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":22,"description":14,"dateModified":61,"datePublished":61,"encodingFormat":60,"isAccessibleForFree":62,"interactionStatistic":63},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":40,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-08-05",true,{"@type":64,"interactionType":65,"userInteractionCount":20},"InteractionCounter",{"@type":66},"ViewAction",{"@type":68,"mainEntity":69},"FAQPage",[70,76,80],{"name":71,"@type":72,"acceptedAnswer":73},"What problem does model reprogramming aim to solve?","Question",{"text":74,"@type":75},"It targets resource-limited scenarios where there is limited target-domain data, constrained model development cost, or inadequate pre-trained models for fine-tuning. The goal is to enable effective cross-domain learning with minimal additional training.","Answer",{"name":77,"@type":72,"acceptedAnswer":78},"How does model reprogramming work without fine-tuning the source model?",{"text":79,"@type":75},"It keeps the pre-trained source model intact and introduces an input transformation layer and an output mapping layer. Only these added components are trained, while the source model parameters are frozen.",{"name":81,"@type":72,"acceptedAnswer":82},"When can model reprogramming outperform transfer learning or training from scratch?",{"text":83,"@type":75},"It is especially effective in small-scale data regimes and cases where target-domain pre-trained models are missing or training resources are constrained. The approach leverages reusable representations from data-rich domains.","https://schema.org",{"og:url":51,"og:type":86,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":88,"canonical":51},"index,follow",{"doc_id":7,"site_id":23},{"code":4,"msg":5,"data":91},[92,96,100,104,109,114,119,122,126,129,133],{"id":20,"doc_module":4,"doc_module_name":45,"category_name":93,"show_sort_weight":94,"slug":95},"Story & Novel",90,"story-novel",{"id":46,"doc_module":4,"doc_module_name":45,"category_name":97,"show_sort_weight":98,"slug":99},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":45,"category_name":101,"show_sort_weight":102,"slug":103},"Exam",70,"exam",{"id":105,"doc_module":4,"doc_module_name":45,"category_name":106,"show_sort_weight":107,"slug":108},5,"Comic",60,"comic",{"id":110,"doc_module":4,"doc_module_name":45,"category_name":111,"show_sort_weight":112,"slug":113},6,"Technology",50,"technology",{"id":115,"doc_module":4,"doc_module_name":45,"category_name":116,"show_sort_weight":117,"slug":118},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":45,"category_name":12,"show_sort_weight":120,"slug":121},30,"research-report",{"id":123,"doc_module":4,"doc_module_name":45,"category_name":124,"show_sort_weight":28,"slug":125},9,"Religion & Spirituality","religion-spirituality",{"id":28,"doc_module":4,"doc_module_name":45,"category_name":127,"show_sort_weight":28,"slug":128},"World Cup","world-cup",{"id":130,"doc_module":4,"doc_module_name":45,"category_name":131,"show_sort_weight":130,"slug":132},10,"Lifestyle","lifestyle",{"id":134,"doc_module":4,"doc_module_name":45,"category_name":135,"show_sort_weight":105,"slug":136},19,"General","general"]