[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-119767-en":3,"doc-seo-119767-105":30,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":4,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":27,"seo_description":14,"update_tm":28,"read_time":29},119767,549758146520,"Patrick","https://ap-avatar.wpscdn.com/avatar/80002397d8c0411e94?_k=1775819394049821470",8,"Research & Report","Delegating Data Collection in Decentralized Machine Learning","Decentralized machine learning ecosystems require principled mechanisms for delegating data collection, especially when the principal lacks reliable information about model quality and about the best achievable performance. Using contract theory, the study designs optimal and near-optimal incentive contracts for assessment under noisy evaluation and for rewarding agents without prior knowledge of the optimal model error. It shows linear contracts can handle uncertainty and attain a 1 − 1/e fraction of first-best utility, and provides conditions for vanishing additive approximation via sufficient test-set size, plus a convex program to compute the optimal contract adaptively.","arXiv :2309 .0 1837v 1 [ cs .LG] 4 Sep 2023  \nDelegating Data Collection in Decentralized Machine Learning Nivasini Ananthakrishnan, Stephen Bates, Michael I. Jordan, and Nika Haghtalab  \nUniversity of California, Berkeley  \nAbstract  \nMotivated by the emergence of decentralized machine learning ecosystems, we study the delegation of data collection. Taking the field of contract theory as our starting point, we design optimal and near-optimal contracts that deal with two fundamental machine learning challenges:  \nlack of certainty in the assessment of model quality and lack of knowledge regarding the optimal performance of any model. We show that lack of certainty can be dealt with via simple linear contracts that achieve 1 − 1/e fraction of the first-best utility, even if the principal has a small test set. Furthermore, we give sufficient conditions on the size of the principal’s test set that achieves a vanishing additive approximation to the optimal utility. To address the lack of a priori knowledge regarding the optimal performance, we give a convex program that can adaptively and efficiently compute the optimal contract.  \n1 Introduction  \nThe design of machine learning pipelines is increasingly a cooperative, distributed endeavor, in which the expertise needed for the design of various components of an overall pipeline is spread across many stakeholders. Such expertise pertains in part to classical design choices such as how much and what kind of data to use for training, how much test data to use for verification, how to train a model, and how to tune hyper-parameters, but, more broadly, expertise may reflect experience, access to certain resources, or knowledge of local conditions. To the extent that there is a central designer, their role may in large part be that of setting requirements, developing coordination mechanisms, and providing incentives.  \nOverall, we are seeing a flourishing new industry at the intersection of ML and operations which makes use of specialization and decentralization to achieve high performance and operational efficiency. Such an ML ecosystem creates a need for new design tools and insights that are not focused merely on how the designer could perform a task in this pipeline, but rather how she should delegate it to agents who are willing and capable of performing the task on her behalf. How should the designer interact with this ecosystem? How should she evaluate and compensate other agents for their work? How does the outcome of the delegated pipeline compare with the outcome if the designer were to perform the task by herself? In this work, we initiate the study of delegating machine learning pipelines through the lens of contract theory and take a step towards answering these questions.  \nContract theory provides a principal-agent perspective, where the principal—who is the designer interested in the outcome of the learning pipeline—can create a contractual arrangement—a menu of services and compensations—with an agent. At the heart of the issue is creating contracts that incentivize the agents, who may be more knowledgeable and skilled than the principal, to take the appropriate actions. The uncertain and data-centric nature of machine learning tasks brings to  \nlight interesting sources of knowledge asymmetry between the principal and the agent and requires extensions of classical contract theory.  \nConsider a scenario where a firm delegates a predictive task to an ML service provider. In this context, the service provider may offer the firm either a dataset for learning or a pre-trained predictive model based on that dataset. To ensure aligned incentives, the firm needs to assess the dataset or predictive model and design the payment structure for the service provider accordingly. Since the accuracy of the model is crucial to the firm as it directly influences revenue, a natural evaluation approach involves directly measuring the accuracy of the model that the service provider produc","cbCaisfs70dSYB08","https://ap.wps.com/l/cbCaisfs70dSYB08","pdf",738261,1,21,"English","en",105,"# Introduction\n## Contract theory principal-agent setting\n## Delegation scenario and evaluation challenges\n## Hidden actions (moral hazard)\n## Hidden state (adverse selection)\n## Our results","[{\"question\":\"What problem does the study address in decentralized machine learning?\",\"answer\":\"It addresses how a principal should delegate data collection to agents using incentive contracts when model-quality assessment is uncertain and the best achievable model performance is unknown.\"},{\"question\":\"How do the authors handle uncertainty in evaluating model quality?\",\"answer\":\"They show that simple linear contracts can mitigate assessment uncertainty, achieving a 1 − 1/e fraction of the first-best utility even with a small test set.\"},{\"question\":\"How are the authors’ contracts adapted when the principal lacks prior knowledge of optimal performance?\",\"answer\":\"They provide a convex program that can adaptively and efficiently compute the optimal contract under limited knowledge of the best achievable error.\"}]","Delegating Data Collection in Decentralized Machine Learning | PDF",1785726195,53,{"code":4,"msg":31,"data":32},"ok",{"site_id":24,"language":23,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":28},"delegating-data-collection-in-decentralized-machine-learning","",{"@graph":36,"@context":85},[37,54,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":53},"https://docshare.wps.com/document/delegating-data-collection-in-decentralized-machine-learning/119767/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":23,"description":14,"dateModified":62,"datePublished":62,"encodingFormat":61,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":41,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-03",true,{"@type":65,"interactionType":66,"userInteractionCount":4},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"What problem does the study address in decentralized machine learning?","Question",{"text":75,"@type":76},"It addresses how a principal should delegate data collection to agents using incentive contracts when model-quality assessment is uncertain and the best achievable model performance is unknown.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"How do the authors handle uncertainty in evaluating model quality?",{"text":80,"@type":76},"They show that simple linear contracts can mitigate assessment uncertainty, achieving a 1 − 1/e fraction of the first-best utility even with a small test set.",{"name":82,"@type":73,"acceptedAnswer":83},"How are the authors’ contracts adapted when the principal lacks prior knowledge of optimal performance?",{"text":84,"@type":76},"They provide a convex program that can adaptively and efficiently compute the optimal contract under limited knowledge of the best achievable error.","https://schema.org",{"og:url":52,"og:type":87,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":89,"canonical":52},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":20,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":46,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":46,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":46,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":46,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":46,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":46,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]