An In-Depth Comparative Analysis of Machine Learning Techniques for Addressing Class Imbalance in Mental Health Prediction

Aug 6, 2026

Upload by Patrick

Machine learning applied to mental healthcare prediction is constrained by imbalanced datasets, where the majority of samples come from diagnosed conditions and minority classes are underrepresented. This imbalance can bias model training and reduce generalisation, limiting reliable prediction. The study compares multiple class-imbalance strategies, including resampling, ensemble learning, and algorithm-specific methods, evaluated with accuracy, precision, recall, and F1. Using data from Open Sourcing Mental Illness (2016–2021), ensemble methods—especially Random Forest—outperform alternatives, with additional metrics such as Kappa, balanced accuracy, and geometric mean to assess effectiveness. Results support earlier diagnosis and more personalised treatment planning.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages12

Reading Time30 min

AccessFree to download

Introduction

Background and motivation

Class imbalance problem

Study aim and contributions

Comparative evaluation of techniques

Methodology and evaluation

Resampling, ensemble, and algorithm-specific approaches

Performance metrics

Association for Information Systems AIS Electronic Library (AISeL) | ACIS 2023 Proceedings | Australasian (ACIS) | | --- | --- | | 12-2-2023
An In-Depth Comparative Analysis of Machine Learning Techniques for Addressing Class Imbalance in Mental Health Prediction.
Tsholofelo Mokheleli
University of Johannesburg, South Africa, [mokhelelitsholo48@gmail.com](mokhelelitsholo48@gmail.com)
Tebogo Bokaba
University of Johannesburg, South Africa, [tbokaba@uj.ac.za](tbokaba@uj.ac.za)
Tinofirei Museba
University of Johannesburg, South Africa, [tmuseba@uj.ac.za](tmuseba@uj.ac.za)
Follow this and additional works at: [https://aisel.aisnet.org/acis2023](https://aisel.aisnet.org/acis2023) | | Recommended Citation Mokheleli, Tsholofelo; Bokaba, Tebogo; and Museba, Tinofirei, "An In-Depth Comparative Analysis of Machine Learning Techniques for Addressing Class Imbalance in Mental Health Prediction." (2023) . ACIS 2023 Proceedings. 15. [https://aisel.aisnet.org/acis2023/15](https://aisel.aisnet.org/acis2023/15) This material is brought to you by the Australasian (ACIS) at AIS Electronic Library (AISeL) . It has been accepted for inclusion in ACIS 2023 Proceedings by an authorized administrator of AIS Electronic Library (AISeL) . For more information, please [contact](contact elibrary@aisnet.org)[ elibrary@aisnet.org](contact elibrary@aisnet.org). An In-Depth Comparative Analysis of Machine Learning Techniques for Addressing Class Imbalance in Mental Health Prediction Full research paper Tsholofelo Mokheleli Department of Applied Information Systems University of Johannesburg Johannesburg, South Africa [Email: mokhelelitsholo48@gmail.com](Email: mokhelelitsholo48@gmail.com) Tebogo Bokaba Department of Applied Information Systems University of Johannesburg Johannesburg, South Africa Email: [tbokaba@uj.ac.za](tbokaba@uj.ac.za) Tinofirei Museba Department of Applied Information Systems University of Johannesburg Johannesburg, South Africa Email: [tmuseba@uj.ac.za](tmuseba@uj.ac.za) Abstract The application of machine learning (ML) in predicting mental healthcare faces a challenge due to imbalanced datasets. ML techniques analyse extensive datasets to make predictions; however, the unequal distribution of samples, with the majority belonging to diagnosed mental disorders, can lead to biased model training and limited generalisation. To mitigate the issue of class imbalance in mental health datasets, this study employed diverse ML techniques, namely, resampling, ensemble, and algorithm-specific approaches and metrics such as accuracy, precision, recall and F1 score. The dataset used was collected from the Open Sourcing Mental Illness website, spanning 2016 to 2021. The findings indicate that ensemble techniques, particularly Random Forest, excelled in managing class imbalance compared to other methods. Beyond conventional performance metrics, the study introduced Kappa, balanced accuracy, and geometric mean to evaluate model effectiveness. These findings provide valuable insights for improving mental health predictions, enabling early diagnosis and personalised treatment strategies. Keywords Class imbalance, Cross-validation, Machine learning, Mental health prediction, Resampling methods. 1 Introduction In recent years, the application of machine learning (ML) for mental health prediction has gained increasing attention, driven by its potential to facilitate the early detection of mental disorders (Awal et al., 2021) . ML algorithms have demonstrated remarkable capabilities in analysing vast datasets encompassing medical records, physiological measurements, and patient-reported symptoms, enabling the prediction of the likelihood of developing mental health disorders and evaluating the efficacy of various treatment options (Vahdat et al. 2016). Accurately predicting mental health conditions can playa significant role in timely intervention, personalised treatment planning, and improve

FAQs

Why does class imbalance affect machine learning for mental health prediction?

Imbalanced datasets have disproportionate sample distributions across disorders, causing biased training and weaker generalisation. Underrepresented disorders may be harder for models to recognise and differentiate accurately.

Which machine learning approaches were used to address class imbalance in the study?

The study applied resampling, ensemble techniques, and algorithm-specific approaches. It also assessed performance using multiple evaluation metrics.

What were the main findings and which method performed best?

Ensemble techniques, particularly Random Forest, performed best for managing class imbalance. The paper also used metrics beyond accuracy, including Kappa, balanced accuracy, and geometric mean, to confirm model effectiveness.

Aug 6, 2026

Upload by Patrick