[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-118980-en":3,"doc-seo-118980-105":30,"detail-sidebar-cat-0-en-105":92},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":27,"seo_description":14,"update_tm":28,"read_time":29},118980,4398048949847,"Eliana","https://ap-avatar.wpscdn.com/avatar/400002536579ef2da7f?_k=1778318612642679267",8,"Research & Report","Managing uncertainty in machine learning techniques - An investigation of adaptive sampling strategies through land cover mappings","Machine learning classifiers are increasingly used for classification, but dependable decision-making requires reliable uncertainty quantification. A key approach is reference sampling, which draws limited ground-truth observations and compares them with predicted outputs to estimate precision and accuracy using statistical inference. Challenges arise because models are often trained without formal sampling inference and act as black boxes versus traditional methods, while suitable reference sampling can be costly. This dissertation studies adaptive reference sampling frameworks through land cover mapping case studies, addressing uncertainty in area estimation, efficient designs under uncertainty, and designs when sampling cost varies across the mapped region.","Managing uncertainty in machine learning techniques: An investigation of adaptive sampling strategies through land cover mappings  \nJordan Phillipson  \nSchool of Computing and Communications Lancaster University  \nThis dissertation is submitted for the degree of Doctor of Philosophy  \nMarch 2024  \nDeclaration  \nI hereby declare that except where specific reference is made to the work of others, the contents ofthis thesis are original and have not been submitted in whole or in part for consideration for any other degree or qualification in this, or any other university. This thesis is my own work and contains nothing which is the outcome of work done in collaboration with others, except as specified in the contribution statements at the beginning of this thesis. This thesis does not exceed the maximum permitted word length: it contains fewer than 80,000 words including appendices and footnotes but excluding the bibliography.  \nJordan Phillipson March 2024  \nList of Publications  \nThe following has been published as part of the research presented in this thesis. Where appropriate, portions ofthis thesis are based on my contributions to these publications without citation. Where research and text should be credited to a co-author, rather than myself, the work has been cited accordingly.  \nPhillipson, J., Blair, G., & Henrys, P. (2022) . Quantifying uncertainty in land cover mappings: An adaptive approach to sampling reference data using Bayesian inference.  \nEnvironmental Data Science, 1, e15. [https://doi.org/10.1017/EDS.2022.14](https://doi.org/10.1017/EDS.2022.14)  \n[Phillipson](Phillipson), Jordan, Gordon Blair, and Peter Henrys. 2020.“Quantifying Uncertainty for Estimates Derived from Error Matrices in Land Cover Mapping Applications: The Case for a Bayesian Approach.” In IFIP Advances in Information and Communication Technology.  \nPhillipson, J., G.S. Blair, and P. Henrys. 2019.“Uncertainty Quantification in Classification Problems: A Bayesian Approach for Predicting the Effects of Further Test Sampling.” In MODSIM2019, 23rd International Congress on Modelling and Simulation, ed. S Elsawah. Canberra: Modelling and Simulation Society of Australia and New Zealand.  \nAcknowledgements  \nFirstly, I would like to thank my supervisors, Gordon Blair and Peter Henrys. You two have been unbelievably supportive (and patient) throughout. I cannot thankyou both enough. In addition, I would like to thank my family, friends, and the wider Ensemble team for their support during this thesis.  \nAbstract  \nIn recent decades, the use of machine learning techniques in classification problems has become increasingly popular across a wide variety of domains. For users to have trust in such classifiers though, one must be able to reliably quantify uncertainty. A common way of quantifying uncertainty in classifiers is through reference sampling where a smaller set of ground-truths is sampled and compared to their predicted counterparts to make inferences about the precision and accuracy of classifiers using statistical methods.  \nHowever, classification via machine learning can bring some additional challenges to uncertainty quantification, as machine learning techniques are often (i) trained using data that has not been sampled with formal statistical inference in mind; (ii) are often black-box when compared to traditional modelling.  \nThese issues are further compounded when sampling reference data under conditions suitable for uncertainty quantification is expensive. Here, users are often forced to make a compromise between the degree of uncertainty and the costs of reference sampling, even when the original classifier built using machine learning may be performing well. In short, when it comes to quantifying and reducing uncertainty, it is not just about how well the classifier performs. One must also be able to collect enough data sampled under the right conditions.  \nThis thesis explores how users may better manage the cost-benefit trade-offs","cbCaiiLlhl8c1bnX","https://ap.wps.com/l/cbCaiiLlhl8c1bnX","pdf",5067181,1,222,"English","en",105,"# Chapter 1: Introduction\n## 1.1 Motivation\n## 1.2 Background\n### 1.2.1 Definitions and terminology\n### 1.2.2 Domain of application: land cover maps\n## 1.3 Programme of research","[{\"question\":\"What is the main problem this dissertation addresses?\",\"answer\":\"It addresses how to quantify and reduce uncertainty in machine learning classifiers while managing the cost-benefit trade-offs of reference sampling.\"},{\"question\":\"Why is reference sampling important for uncertainty quantification?\",\"answer\":\"Reference sampling collects a smaller set of ground truths and compares them with predicted outputs to infer classifier precision and accuracy statistically.\"},{\"question\":\"What specific scenarios are evaluated in the land cover mapping case studies?\",\"answer\":\"The work considers uncertainty in area estimation and mappings, efficient sample designs under uncertainty, and sample designs when reference sampling cost varies across a mapped region.\"}]","Managing uncertainty in machine learning techniques - An investigation of adaptive sampling strategies through land cover mappings | PDF",1785721348,559,{"code":4,"msg":31,"data":32},"ok",{"site_id":24,"language":23,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":87,"head_meta":89,"extra_data":91,"updated_unix":28},"managing-uncertainty-in-machine-learning-techniques-an-investigation-of-adaptive-sampling-strategies-through-land-cover-mappings","",{"@graph":36,"@context":86},[37,54,69],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":53},"https://docshare.wps.com/document/managing-uncertainty-in-machine-learning-techniques-an-investigation-of-adaptive-sampling-strategies-through-land-cover-mappings/118980/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":23,"description":14,"dateModified":62,"datePublished":63,"encodingFormat":61,"isAccessibleForFree":64,"interactionStatistic":65},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":41,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-04","2026-08-03",true,{"@type":66,"interactionType":67,"userInteractionCount":20},"InteractionCounter",{"@type":68},"ViewAction",{"@type":70,"mainEntity":71},"FAQPage",[72,78,82],{"name":73,"@type":74,"acceptedAnswer":75},"What is the main problem this dissertation addresses?","Question",{"text":76,"@type":77},"It addresses how to quantify and reduce uncertainty in machine learning classifiers while managing the cost-benefit trade-offs of reference sampling.","Answer",{"name":79,"@type":74,"acceptedAnswer":80},"Why is reference sampling important for uncertainty quantification?",{"text":81,"@type":77},"Reference sampling collects a smaller set of ground truths and compares them with predicted outputs to infer classifier precision and accuracy statistically.",{"name":83,"@type":74,"acceptedAnswer":84},"What specific scenarios are evaluated in the land cover mapping case studies?",{"text":85,"@type":77},"The work considers uncertainty in area estimation and mappings, efficient sample designs under uncertainty, and sample designs when reference sampling cost varies across a mapped region.","https://schema.org",{"og:url":52,"og:type":88,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":90,"canonical":52},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":93},[94,98,102,106,111,116,121,124,129,132,136],{"id":20,"doc_module":4,"doc_module_name":46,"category_name":95,"show_sort_weight":96,"slug":97},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":99,"show_sort_weight":100,"slug":101},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":46,"category_name":103,"show_sort_weight":104,"slug":105},"Exam",70,"exam",{"id":107,"doc_module":4,"doc_module_name":46,"category_name":108,"show_sort_weight":109,"slug":110},5,"Comic",60,"comic",{"id":112,"doc_module":4,"doc_module_name":46,"category_name":113,"show_sort_weight":114,"slug":115},6,"Technology",50,"technology",{"id":117,"doc_module":4,"doc_module_name":46,"category_name":118,"show_sort_weight":119,"slug":120},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":122,"slug":123},30,"research-report",{"id":125,"doc_module":4,"doc_module_name":46,"category_name":126,"show_sort_weight":127,"slug":128},9,"Religion & Spirituality",20,"religion-spirituality",{"id":127,"doc_module":4,"doc_module_name":46,"category_name":130,"show_sort_weight":127,"slug":131},"World Cup","world-cup",{"id":133,"doc_module":4,"doc_module_name":46,"category_name":134,"show_sort_weight":133,"slug":135},10,"Lifestyle","lifestyle",{"id":137,"doc_module":4,"doc_module_name":46,"category_name":138,"show_sort_weight":107,"slug":139},19,"General","general"]