Machine learning route map - Route map for machine learning in psychiatry - absence of bias, reproducibility, and utility

Aug 2, 2026

Upload by Riley

Machine learning route map for psychiatry addresses three critical quality gaps that can undermine real-world value: absence of bias, reproducibility, and utility. The work contrasts earlier optimistic case examples with psychiatry-specific methodological risks, emphasizing how ignoring site effects or confounders can inflate apparent accuracy. It further highlights how flexible model searching and selective reporting can enable data torturing, making results difficult to reproduce. The paper proposes a practical route map to guide future studies toward rigorous, dependable psychiatric applications.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages6

Reading Time15 min

AccessFree to download

Absence of bias

Methodological rigor and confounding control

Reproducibility

Data torturing and publication bias

Machine learning route map Route map for machine learning in psychiatry: absence of bias, reproducibility, and utility Joaquim Radua 1,2,3,* and Andre F. Carvalho4 1 Imaging of Mood- and Anxiety-Related Disorders (IMARD) group, Institut d’Investigacions Biomèdiques August Pi i Sunyer (IDIBAPS), CIBERSAM, Barcelona, Spain 2 Early Psychosis: Interventions and Clinical-detection (EPIC) lab, Institute of Psychiatry, Psychology, and Neuroscience, King’s College London, London, UK 3 Department of Clinical Neuroscience, Stockholm Health Care Services, Stockholm County Council, Karolinska Institutet, Stockholm, Sweden. 4 IMPACT (Innovation in Mental and Physical Health and Clinical Treatment) Strategic Research Centre, School of Medicine, Barwon Health, Deakin University, Geelong, VIC, Australia. *Corresponding Author Joaquim Radua, MD PhD Institut d'Investigacions Biomèdiques August Pi i Sunyer (IDIBAPS), Barcelona, Spain. Email: [radua@clinic.cat](radua@clinic.cat) [Total word count:](Total word count: 1013)[ 1013](Total word count: 1013) [Number of references:](Number of references: 11)[ 11](Number of references: 11) Short title: Machine learning route map Acknowledgments JR is supported by a grant from the Instituto de Salud Carlos III and co-funded by European Union (ERDF/ESF,“Investing in your future”): Miguel Servet Research Contract CPII19/00009 . Machine learning route map TEXT In the past decade, several groups reported incredible achievements using machine learning. For instance, Google reported a neural network that taught itself how to identify cats (Markoff, 2012) . Or Facebook presented another network that recognized individuals in photographs with >97% accuracy (Taigman et al., 2014) . Such successes led to considerable interest in the application of machine learning techniques to many disciplines. Psychiatry was not an exception, and we embraced this perspective enthusiastically. There were reasons to be optimistic. For example, years ago, we used obscure stepwise regressions to find a model to predict treatment response from several baseline variables. We knew that stepwise regression led to inflated statistical significance (Mundry and Nunn, 2009), but we had fewer alternatives. Today, we have safe machine learning classifiers such as regularized regressions (e.g., lasso), random forests, or support vector machines (Salvador et al., 2017) . Ultimately, these tools have the potential to predict the therapeutic response at an individual level. However, we think that we should not give machine learning a blank cheque. The enthusiasm may have made us lower the guard in methodological rigor and preclinical/clinical utility. As we expose in the following, we believe that several hurdles may lead the community to think that machine learning is only about unbelievable predictions or useless studies. We also propose a route map to avoid these hurdles, guiding future studies so that machine learning becomes a reliable and valuable tool in psychiatry. Absence of bias The first hurdle refers to a permissive methodology that may lead to systematic biases. For instance, everyone involved in magnetic resonance imaging research knows that when you have data from different sites, you must very carefully control the effects of the site (Radua et al., 2020) . However, in novel machine learning applications, analysts usually estimate the accuracy of the prediction model without considering these effects. Unfortunately, ignoring them may yield severely inflated accuracy. In other words, machine learning models may seem to predict very well when they do not even predict (Solanes et al., 2021) . We propose ensuring that machine learning studies meet the same methodological rigor as any other study. We know that in machine learning, any algorithm is possible. We are openminded: we may accept that your algorithm includes astrology and tarot readings to conduct the predictions. But when it comes to

FAQs

Why can machine learning accuracy be misleading in psychiatric studies?

Ignoring key confounding effects such as data collected from different sites can inflate accuracy even when models are not truly predictive.

What does the paper mean by “data torturing” in machine learning?

Repeatedly trying many algorithms and stopping at favorable results can lead to overfitting to noise and overly optimistic conclusions that are hard to reproduce.

What guidance does the route map provide to improve the reliability of machine learning in psychiatry?

It calls for the same methodological rigor used in standard statistics, including controlling confounders, and ensuring reproducible modeling and evaluation rather than relying on permissive workflows.

Aug 2, 2026

Upload by Riley