[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-117628-en":3,"doc-seo-117628-105":30,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":4,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":27,"seo_description":14,"update_tm":28,"read_time":29},117628,962075114101,"Seraphina","https://ap-avatar.wpscdn.com/avatar/e000253a75eb197efd?x-image-process=image/resize,m_fixed,w_180,h_180&k=1780044092746381165",8,"Research & Report","Prediction-powered machine learning for model selection and uncertainty - thesis","This thesis explores model selection and uncertainty through the lens of prediction. Building on Bayesian predictive inference, it treats uncertainty as a missing data problem and extends bootstrap logic by using future observations as the basis for inference. The framework supports probabilistic uncertainty quantification without subjective prior specification, applied to model uncertainty and hypothesis testing via recursive imputation and one-step-ahead selection criteria. It further links sequential predictive behavior to coherence, enabling Bayesian-style uncertainty in plug-in and frequentist settings, and uses predictive ensembling for causal treatment effect estimation with a random forest targeting relative risk heterogeneity on cardiovascular trial data.","Prediction-powered machine learning for model selection and uncertainty  \nVik Shirvaikar St. Peter’s College University of Oxford  \nA thesis submitted for the degree of Doctor of Philosophy  \nTrinity 2025  \nIn the midst of this perplexity, I received from Oxford the manuscript you  \nhave just examined. I lingered, naturally, on the sentence: “I leave to various future times, but not to all, my garden of forking paths.”  \n— Jorge Luis Borges, 1941  \nAcknowledgements  \nMy research has greatly benefited from the creative supervision of Chris Holmes. I’m grateful to Choudur Lakshminarayan for inspiring me to be a statistician, and Stephen Walker for his guidance and insight.  \nI’ve had the honor of working with several excellent collaborators who have challenged and motivated me. In particular, thank you to Andrea Storås, Xi Lin, and Nic Steyn for helping me grow as a researcher.  \nMy doctorate was supported by the Engineering and Physical Sciences Research Council, through the StatML CDT, and Novo Nordisk. A special thanks to the team at the Department of Statistics, especially Joanna, Emma, Beverley, and Frédérique, for making it a warm and welcoming place to work. I’m grateful to the many friends who have supported me along the way, and to my partner Julia for being a constant source of joy and strength.  \nFinally, thank you to my brother and parents, Vinny, Anjali, and Mukul Shirvaikar, for always believing in and encouraging me. I could never have reached this point without you.  \nAbstract  \nThis thesis explores model selection and uncertainty through the lens of prediction.  \nBuilding on recent developments in Bayesian predictive inference, we approach uncertainty as a missing data problem, extending the logic of the bootstrap by treating future observations as the basis for inference. This framework offers a complementary perspective to conventional frequentist and Bayesian methods, as it supports probabilistic uncertainty quantification without requiring the subjective specification of a prior distribution. We first apply this lens to model uncertainty and hypothesis testing, proposing a novel procedure where uncertainty is propagated via the recursive imputation of new data, using a one-step-ahead model selection criterion. We then broaden this view, arguing that a model’s sequential predictive behavior—specifically, its production of conditionally identically distributed updates — can be used to characterize its coherence, allowing for Bayesian-style uncertainty even in plug-in or frequentist settings. Finally, we apply predictive model ensembling to causal treatment effect estimation, introducing a random forest method that targets relative risk heterogeneity, and demonstrating its application to data from a major cardiovascular clinical trial. Taken together, these results argue that predictive resampling methods, grounded in bootstrap principles, can provide flexible and principled tools for model evaluation and validation.  \nContents  \n1 Introduction 1  \n1.1 The eternal debate: inference or prediction? ................ 1  \n1.2 The puzzle of inverse probability ....................... 2  \n1.2.1 Bayes, Laplace, and uniform priors ................. 2  \n1.2.2 Fisher and fiducial inference ..................... 4  \n1.2.3 Modern attempts at unification ................... 5  \n1.3 Uncertainty through a predictive lens .................... 6  \n1.3.1 Bayesian inference as a missing data problem ........... 6  \n1.3.2 Martingale posterior distributions .................. 9  \n1.3.3 Conditionally identically distributed sequences ........... 12  \n1.3.4 Related work ............................. 14  \n1.4 From parameters to models ......................... 15  \n1.4.1 Why model uncertainty matters ................... 15  \n1.4.2 Bayesian model averaging and exploration ............. 16  \n1.4.3 Bagging, ensembling, and mixtures ................. 17  \n1.5 Thesis outline ................................. 18  \n2 A general framework for p","cbCaigy6fR1TJN17","https://ap.wps.com/l/cbCaigy6fR1TJN17","pdf",22417671,1,161,"English","en",105,"# 1 Introduction\n## 1.1 The eternal debate: inference or prediction?\n## 1.2 The puzzle of inverse probability\n## 1.3 Uncertainty through a predictive lens\n## 1.4 From parameters to models\n## 1.5 Thesis outline\n# 2 A general framework for probabilistic model uncertainty\n## 2.1 Introduction\n## 2.2 Model uncertainty via predictive resampling\n## 2.3 Convergence of one-step updates\n## 2.4 Illustrations\n## 2.5 Conclusion\n# 3 Hypothesis testing via predictive resampling\n## 3.1 Introduction\n## 3.2 Review of existing approaches\n## 3.3 Illustrations\n# 4 Bayesian prediction without parameter uncertainty\n## 4.1 Introduction\n## 4.2 Related work\n## 4.3 Rethinking Bayesian prediction\n## 4.4 Evaluating predictive coherence\n## 4.5 Illustrations","[{\"question\":\"How does the thesis connect uncertainty with prediction?\",\"answer\":\"It frames uncertainty as a missing data problem in Bayesian predictive inference, extending bootstrap logic by treating future observations as the basis for inference.\"},{\"question\":\"What method is proposed for model uncertainty and hypothesis testing?\",\"answer\":\"It proposes a procedure that propagates uncertainty via recursive imputation of new data, using a one-step-ahead model selection criterion.\"},{\"question\":\"How is predictive coherence characterized in the thesis?\",\"answer\":\"The thesis argues that a model’s sequential predictive behavior—specifically conditionally identically distributed updates—can characterize coherence, enabling Bayesian-style uncertainty beyond fully Bayesian settings.\"}]","Prediction-powered machine learning for model selection and uncertainty - thesis | PDF",1785677432,406,{"code":4,"msg":31,"data":32},"ok",{"site_id":24,"language":23,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":28},"prediction-powered-machine-learning-for-model-selection-and-uncertainty-thesis","",{"@graph":36,"@context":85},[37,54,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":53},"https://docshare.wps.com/document/prediction-powered-machine-learning-for-model-selection-and-uncertainty-thesis/117628/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":23,"description":14,"dateModified":62,"datePublished":62,"encodingFormat":61,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":41,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-02",true,{"@type":65,"interactionType":66,"userInteractionCount":4},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"How does the thesis connect uncertainty with prediction?","Question",{"text":75,"@type":76},"It frames uncertainty as a missing data problem in Bayesian predictive inference, extending bootstrap logic by treating future observations as the basis for inference.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"What method is proposed for model uncertainty and hypothesis testing?",{"text":80,"@type":76},"It proposes a procedure that propagates uncertainty via recursive imputation of new data, using a one-step-ahead model selection criterion.",{"name":82,"@type":73,"acceptedAnswer":83},"How is predictive coherence characterized in the thesis?",{"text":84,"@type":76},"The thesis argues that a model’s sequential predictive behavior—specifically conditionally identically distributed updates—can characterize coherence, enabling Bayesian-style uncertainty beyond fully Bayesian settings.","https://schema.org",{"og:url":52,"og:type":87,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":89,"canonical":52},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":20,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":46,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":46,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":46,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":46,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":46,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":46,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]