[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-120017-en":3,"doc-seo-120017-105":30,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":4,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":27,"seo_description":14,"update_tm":28,"read_time":29},120017,137441390410,"Hazel","https://ap-avatar.wpscdn.com/avatar/2000252f4ab5702993?_k=1776741390130283984",8,"Research & Report","Assessing the performance of machine learning methods trained on public health observational data: a case study from COVID-19","Interest in using machine learning to predict COVID-19 infection status from vocal audio signals emerged early in the coronavirus disease 2019 pandemic, for example cough recordings. Early work faced constraints in data collection and in how model performance was evaluated. This study, conducted by the Turing-RSS Health Data Laboratory and the UK Health Security Agency, uses an acoustic dataset with SARS-CoV-2 infection status and rich participant metadata to enable rigorous assessment of state-of-the-art techniques. Findings inform future public-health evaluation practices for machine learning.","King’s Research Portal  \nDocument Version  \nPeer reviewed version  \nLink to publication record in King's Research Portal  \nCitation for published version (APA):  \nPigoli, D. , Baker, K. , Budd, J. , Butler, L. , Coppock, H. , Egglestone, S. , Gilmour, S. G. , Holmes, C. , Hurley, D. , Jersakova, R. , Kiskin, I. , Koutra, V. , Mellor, J. , Nicholson, G. , Packham, J. , Patel, S. , Payne, R. , Roberts, S. , Schuller, B. , ... Titcomb, A. (in press) . Assessing the performance of machine learning methods trained on public health observational data: a case study from COVID-19 . Statistics in Medicine.  \nCiting this paper  \nPlease note that where the full-text provided on King's Research Portal is the Author Accepted Manuscript or Post-Print version this may differ from the final Published version. If citing, it is advised that you check and use the publisher's definitive version for pagination, volume/issue, and date of publication details. And where the final published version is provided on the Research Portal, if citing you are again advised to check the publisher's website for any subsequent corrections.  \nGeneral rights  \nCopyright and moral rights for the publications made accessible in the Research Portal are retained by the authors and/or other copyright owners and it is a condition of accessing publications that users recognize and abide by the legal requirements associated with these rights.  \n•Users may download and print one copy of any publication from the Research Portal for the purpose of private study or research.  \n•You may not further distribute the material or use it for any profit-making activity or commercial gain  \n•You may freely distribute the URL identifying the publication in the Research Portal  \nTake down policy  \nIf you believe that this document breaches copyright please contact [librarypure@kcl.ac.uk](librarypure@kcl.ac.uk) providing details, and we will remove access to the work immediately and investigate your claim.  \nDownload date: 21. Sep. 2024  \nDOI: xxx/xxxx  \nRE SEA R CH A R TI CLE  \nAssessing the performance of machine learning methods trained on public health observational data: a case study from COVID-19  \nDavide Pigoli∗1,6  \nKieran Baker∗1,6  \nJobie Budd2  \nLorraine Butler3  \nHarry Coppock4,6  \n Sabrina Egglestone3  Steven G. Gilmour1,6  Chris Holmes5,6  David Hurley3  Radka Jersakova6  Ivan Kiskin7  Vasiliki Koutra1,6  Jonathon Mellor3  George Nicholson8,6  Joe Packham3  Selina Patel3,9  Richard Payne3  Stephen J. Roberts5,6  Björn W. Schuller6,4  Ana Tendero-Cañadas3,10  Tracey Thornley11  Alexander Titcomb3  \n1King’s College London, UK 2University College London, UK.  \n3UK Health Security Agency, London, UK. 4Imperial College London, UK.  \n5University of Oxford, UK  \n6The Alan Turing Institute, London, UK.  \n7University of Surrey, UK.  \n8University of Oxford, UK.  \n9University College London, UK.  \n10University of Brighton, UK.  \n11University of Nottingham, UK.  \nCorrespondence  \nDavide Pigoli, Department of Mathematics, King’s College London, Strand, London WC2R 2LS, United Kingdom.  \n∗ These authors contributed equally to this work. Email: [davide.pigoli@kcl.ac.uk](davide.pigoli@kcl.ac.uk).  \nAbstract  \nFrom early in the coronavirus disease 2019 (COVID-19) pandemic, there was interest in using machine learning methods to predict COVID-19 infection status based on vocal audio signals, for example, cough recordings. However, early studies had limitations in terms of data collection and of how the performances of the proposed predictive models were assessed. This paper describes how these limitations have been overcome in a study carried out by the Turing-RSS Health Data Laboratory and the UK Health Security Agency. As part of the study, the UK Health Security Agency collected a dataset of acoustic recordings, SARS-CoV-2 infection status and extensive study participant meta-data. This allowed us to rigorously assess state-of-the-art machine learning techniques to predict SARS-CoV-2 infe","cbCaifnLGQMNpRTB","https://ap.wps.com/l/cbCaifnLGQMNpRTB","pdf",3887909,1,16,"English","en",105,"# Abstract\n# Introduction\n# Methods and dataset\n## Acoustic recordings and infection status\n## Participant metadata\n# Evaluation and results\n# Discussion and implications","[{\"question\":\"What problem does the study address about early COVID-19 machine learning work?\",\"answer\":\"Early studies had limitations in data collection and in how predictive model performance was assessed.\"},{\"question\":\"How was the dataset constructed for rigorous evaluation in this study?\",\"answer\":\"The UK Health Security Agency collected acoustic recordings, SARS-CoV-2 infection status, and extensive participant metadata.\"},{\"question\":\"What is the intended impact of the project’s lessons learned?\",\"answer\":\"Lessons learned are meant to guide future studies on statistical evaluation methods for public health machine learning tasks.\"}]","Assessing the performance of machine learning methods trained on public health observational data: a case study from COVID-19 | PDF",1785727748,40,{"code":4,"msg":31,"data":32},"ok",{"site_id":24,"language":23,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":28},"assessing-the-performance-of-machine-learning-methods-trained-on-public-health-observational-data-a-case-study-from-covid-19","",{"@graph":36,"@context":85},[37,54,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":53},"https://docshare.wps.com/document/assessing-the-performance-of-machine-learning-methods-trained-on-public-health-observational-data-a-case-study-from-covid-19/120017/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":23,"description":14,"dateModified":62,"datePublished":62,"encodingFormat":61,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":41,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-03",true,{"@type":65,"interactionType":66,"userInteractionCount":4},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"What problem does the study address about early COVID-19 machine learning work?","Question",{"text":75,"@type":76},"Early studies had limitations in data collection and in how predictive model performance was assessed.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"How was the dataset constructed for rigorous evaluation in this study?",{"text":80,"@type":76},"The UK Health Security Agency collected acoustic recordings, SARS-CoV-2 infection status, and extensive participant metadata.",{"name":82,"@type":73,"acceptedAnswer":83},"What is the intended impact of the project’s lessons learned?",{"text":84,"@type":76},"Lessons learned are meant to guide future studies on statistical evaluation methods for public health machine learning tasks.","https://schema.org",{"og:url":52,"og:type":87,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":89,"canonical":52},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,119,122,127,130,134],{"id":20,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":46,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":46,"category_name":117,"show_sort_weight":29,"slug":118},7,"Healthcare","healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":120,"slug":121},30,"research-report",{"id":123,"doc_module":4,"doc_module_name":46,"category_name":124,"show_sort_weight":125,"slug":126},9,"Religion & Spirituality",20,"religion-spirituality",{"id":125,"doc_module":4,"doc_module_name":46,"category_name":128,"show_sort_weight":125,"slug":129},"World Cup","world-cup",{"id":131,"doc_module":4,"doc_module_name":46,"category_name":132,"show_sort_weight":131,"slug":133},10,"Lifestyle","lifestyle",{"id":135,"doc_module":4,"doc_module_name":46,"category_name":136,"show_sort_weight":106,"slug":137},19,"General","general"]