Assessing the performance of machine learning methods trained on public health observational data: a case study from COVID-19

Aug 3, 2026

Upload by Hazel

Interest in using machine learning to predict COVID-19 infection status from vocal audio signals emerged early in the coronavirus disease 2019 pandemic, for example cough recordings. Early work faced constraints in data collection and in how model performance was evaluated. This study, conducted by the Turing-RSS Health Data Laboratory and the UK Health Security Agency, uses an acoustic dataset with SARS-CoV-2 infection status and rich participant metadata to enable rigorous assessment of state-of-the-art techniques. Findings inform future public-health evaluation practices for machine learning.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages16

Reading Time40 min

AccessFree to download

Abstract

Introduction

Methods and dataset

Acoustic recordings and infection status

Participant metadata

Evaluation and results

Discussion and implications

King’s Research Portal Document Version Peer reviewed version Link to publication record in King's Research Portal Citation for published version (APA): Pigoli, D. , Baker, K. , Budd, J. , Butler, L. , Coppock, H. , Egglestone, S. , Gilmour, S. G. , Holmes, C. , Hurley, D. , Jersakova, R. , Kiskin, I. , Koutra, V. , Mellor, J. , Nicholson, G. , Packham, J. , Patel, S. , Payne, R. , Roberts, S. , Schuller, B. , ... Titcomb, A. (in press) . Assessing the performance of machine learning methods trained on public health observational data: a case study from COVID-19 . Statistics in Medicine. Citing this paper Please note that where the full-text provided on King's Research Portal is the Author Accepted Manuscript or Post-Print version this may differ from the final Published version. If citing, it is advised that you check and use the publisher's definitive version for pagination, volume/issue, and date of publication details. And where the final published version is provided on the Research Portal, if citing you are again advised to check the publisher's website for any subsequent corrections. General rights Copyright and moral rights for the publications made accessible in the Research Portal are retained by the authors and/or other copyright owners and it is a condition of accessing publications that users recognize and abide by the legal requirements associated with these rights. •Users may download and print one copy of any publication from the Research Portal for the purpose of private study or research. •You may not further distribute the material or use it for any profit-making activity or commercial gain •You may freely distribute the URL identifying the publication in the Research Portal Take down policy If you believe that this document breaches copyright please contact [librarypure@kcl.ac.uk](librarypure@kcl.ac.uk) providing details, and we will remove access to the work immediately and investigate your claim. Download date: 21. Sep. 2024 DOI: xxx/xxxx RE SEA R CH A R TI CLE Assessing the performance of machine learning methods trained on public health observational data: a case study from COVID-19 Davide Pigoli∗1,6 Kieran Baker∗1,6 Jobie Budd2 Lorraine Butler3 Harry Coppock4,6 Sabrina Egglestone3 Steven G. Gilmour1,6 Chris Holmes5,6 David Hurley3 Radka Jersakova6 Ivan Kiskin7 Vasiliki Koutra1,6 Jonathon Mellor3 George Nicholson8,6 Joe Packham3 Selina Patel3,9 Richard Payne3 Stephen J. Roberts5,6 Björn W. Schuller6,4 Ana Tendero-Cañadas3,10 Tracey Thornley11 Alexander Titcomb3 1King’s College London, UK 2University College London, UK. 3UK Health Security Agency, London, UK. 4Imperial College London, UK. 5University of Oxford, UK 6The Alan Turing Institute, London, UK. 7University of Surrey, UK. 8University of Oxford, UK. 9University College London, UK. 10University of Brighton, UK. 11University of Nottingham, UK. Correspondence Davide Pigoli, Department of Mathematics, King’s College London, Strand, London WC2R 2LS, United Kingdom. ∗ These authors contributed equally to this work. Email: [davide.pigoli@kcl.ac.uk](davide.pigoli@kcl.ac.uk). Abstract From early in the coronavirus disease 2019 (COVID-19) pandemic, there was interest in using machine learning methods to predict COVID-19 infection status based on vocal audio signals, for example, cough recordings. However, early studies had limitations in terms of data collection and of how the performances of the proposed predictive models were assessed. This paper describes how these limitations have been overcome in a study carried out by the Turing-RSS Health Data Laboratory and the UK Health Security Agency. As part of the study, the UK Health Security Agency collected a dataset of acoustic recordings, SARS-CoV-2 infection status and extensive study participant meta-data. This allowed us to rigorously assess state-of-the-art machine learning techniques to predict SARS-CoV-2 infe

FAQs

What problem does the study address about early COVID-19 machine learning work?

Early studies had limitations in data collection and in how predictive model performance was assessed.

How was the dataset constructed for rigorous evaluation in this study?

The UK Health Security Agency collected acoustic recordings, SARS-CoV-2 infection status, and extensive participant metadata.

What is the intended impact of the project’s lessons learned?

Lessons learned are meant to guide future studies on statistical evaluation methods for public health machine learning tasks.

Aug 3, 2026

Upload by Hazel