[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-118322-en":3,"doc-seo-118322-105":30,"detail-sidebar-cat-0-en-105":83},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":4,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":27,"seo_description":14,"update_tm":28,"read_time":29},118322,7971461740909,"Levi","https://ap-avatar.wpscdn.com/davatar_155a257f0dc6eb9ab79c44ca47cae57d",8,"Research & Report","Between Randomness and Arbitrariness - Some Lessons for Reliable Machine Learning at Scale","Reliable machine learning depends on trustworthy measurements of both models and the systems in which they operate, yet reliability is hard to achieve due to reproducibility limits, scalability constraints, uncertainty quantification issues, and deeper questions in epistemology. This dissertation develops criteria for meaningful metrics and methods to measure them dependably and efficiently at scale and in practice. It tackles arbitrariness and legal-aligned notions of due process, manages randomness in uncertainty estimation and optimization, and evaluates generative-AI systems via memorization and open-licensed data training with urgent copyright-law implications.","Between Randomness and Arbitrariness: Some Lessons for Reliable Machine Learning at Scale  \nA Dissertation  \nPresented to the Faculty of the Graduate School of Cornell University  \nin Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy  \nby  \nA. Feder Cooper  \n© 2024 A. Feder Cooper  \nALL RIGHTS RESERVED  \nBETWEEN RANDOMNESS AND ARBITRARINESS:  \nSOME LESSONS FOR RELIABLE MACHINE LEARNING AT SCALE  \nA. Feder Cooper, Ph.D.  \nCornell University 2024  \nTo develop rigorous knowledge about ML models—and the systems in which they are embedded—we need reliable measurements. But reliable measurement is fundamentally challenging, and touches on issues of reproducibility, scalability, uncertainty quantification, epistemology, and more. This dissertation addresses criteria needed to take reliability seriously: both criteria for designing meaningful metrics, and for methodologies that ensure that we can dependably and efficiently measure these metrics at scale and in practice. In doing so, this dissertation articulates a research vision for a new field of scholarship at the intersection of machine learning, law, and policy. Within this frame, we cover topics that fit under three different themes.  \nFirst, we quantify and mitigate sources of arbitrariness in machine learning, with respect to hyperparameter optimization and social prediction contexts. We clarify important connections between machine-learning arbitrariness, rooted in non-determinism, with legal notions of arbitrariness that implicate legal rules and due process.  \nSecond, we tame randomness in uncertainty estimation and optimization algorithms, in order to achieve scalability without sacrificing reliability. We discuss how across computing, and particularly in machine learning, scalability and reliability are typically in trade-off. Analogous trade-offs in law and policy make this type of trade-off a useful abstraction for communicating about  \nmachine-learning capabilities and risks to policymakers and other non-expert stakeholders.  \nThird, we provide methods for evaluating generative-AI systems, with specific focuses on quantifying memorization in language models and training latent diffusion models on open-licensed data. These contributions have urgent and significant connections to U.S. copyright law. We provide an abridged discussion of landmark legal scholarship that details the complicated relationships between generative-AI supply chain and copyright.  \nBy making contributions in these three themes, this dissertation serves asan empirical proof by example that research on reliable measurement for machine learning is intimately and inescapably bound up with research in law and policy. These different disciplines pose similar research questions about reliable measurement in machine learning. They are, in fact, two complementary sides of the same research vision, which, broadly construed, aims to construct machine-learning systems that cohere with broader societal values.  \nBIOGRAPHICAL SKETCH  \nA. Feder Cooper was born and raised in New York, NY, and obtained his B.A. in Computer Science and Archaeology from Columbia University in 2014 . Prior to a research career, Cooper worked for several years as a software engineer. In 2018, he began his Ph.D. in Computer Science at Cornell University. His doctoral work was chaired by Professor Christopher De Sa, with additional advising by James Grimmelmann, Jon Kleinberg, and Adrian Sampson. His Ph.D. work, broadly construed, studies reliable measurement and evaluation of machine learning, covering both computer science aspects of this work as well as their associated ethical, legal, and policy dimensions.1  \nHis contributions span uncertainty estimation, privacy and security of generative-AI systems, distributed training, hyperparameter optimization, and model selection. His work has been recognized by spotlight awards (NeurIPS 2020), oral presentation slots (e.g., AIES 2021), Best Student Paper (Honor","cbCaio5H1BAUMBHI","https://ap.wps.com/l/cbCaio5H1BAUMBHI","pdf",39276634,1,655,"English","en",105,"# Introduction\n## Motivation: reliable measurement for ML systems\n## Three research themes\n# Arbitrariness in machine learning\n## Hyperparameter optimization\n## Social prediction contexts and legal connections\n# Randomness control for scalability\n## Uncertainty estimation\n## Optimization algorithms\n## Reliability–scalability trade-offs\n# Evaluating generative-AI systems\n## Quantifying memorization in language models\n## Training latent diffusion models on open-licensed data\n## Connections to U.S. copyright law","[{\"question\":\"What methods does the dissertation propose for evaluating generative-AI systems?\",\"answer\":\"It provides approaches focused on quantifying memorization in language models and on training latent diffusion models using open-licensed data. It also discusses how these contributions relate to U.S. copyright law and generative-AI supply-chain scholarship.\"}]","Between Randomness and Arbitrariness - Some Lessons for Reliable Machine Learning at Scale | PDF",1785683046,1651,{"code":4,"msg":31,"data":32},"ok",{"site_id":24,"language":23,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":78,"head_meta":80,"extra_data":82,"updated_unix":28},"between-randomness-and-arbitrariness-some-lessons-for-reliable-machine-learning-at-scale","",{"@graph":36,"@context":77},[37,54,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":53},"https://docshare.wps.com/document/between-randomness-and-arbitrariness-some-lessons-for-reliable-machine-learning-at-scale/118322/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":23,"description":14,"dateModified":62,"datePublished":62,"encodingFormat":61,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":41,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-02",true,{"@type":65,"interactionType":66,"userInteractionCount":4},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71],{"name":72,"@type":73,"acceptedAnswer":74},"What methods does the dissertation propose for evaluating generative-AI systems?","Question",{"text":75,"@type":76},"It provides approaches focused on quantifying memorization in language models and on training latent diffusion models using open-licensed data. It also discusses how these contributions relate to U.S. copyright law and generative-AI supply-chain scholarship.","Answer","https://schema.org",{"og:url":52,"og:type":79,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":81,"canonical":52},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":84},[85,89,93,97,102,107,112,115,120,123,127],{"id":20,"doc_module":4,"doc_module_name":46,"category_name":86,"show_sort_weight":87,"slug":88},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":90,"show_sort_weight":91,"slug":92},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Exam",70,"exam",{"id":98,"doc_module":4,"doc_module_name":46,"category_name":99,"show_sort_weight":100,"slug":101},5,"Comic",60,"comic",{"id":103,"doc_module":4,"doc_module_name":46,"category_name":104,"show_sort_weight":105,"slug":106},6,"Technology",50,"technology",{"id":108,"doc_module":4,"doc_module_name":46,"category_name":109,"show_sort_weight":110,"slug":111},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":113,"slug":114},30,"research-report",{"id":116,"doc_module":4,"doc_module_name":46,"category_name":117,"show_sort_weight":118,"slug":119},9,"Religion & Spirituality",20,"religion-spirituality",{"id":118,"doc_module":4,"doc_module_name":46,"category_name":121,"show_sort_weight":118,"slug":122},"World Cup","world-cup",{"id":124,"doc_module":4,"doc_module_name":46,"category_name":125,"show_sort_weight":124,"slug":126},10,"Lifestyle","lifestyle",{"id":128,"doc_module":4,"doc_module_name":46,"category_name":129,"show_sort_weight":98,"slug":130},19,"General","general"]