Extreme Gradient Boosting Algorithm to Improve Machine Learning Model Performance on Multiclass Imbalanced Dataset

Aug 4, 2026

Upload by Hazel

Class imbalance in real-world datasets arises when minority classes have far fewer samples than majority classes, causing models to learn dominant patterns and misclassify minority instances. While many existing solutions target binary imbalance, multiclass imbalance is more complex due to multiple classes and more challenging decision boundaries. This study proposes using Xtreme Gradient Boosting within an ensemble approach to adjust the learning difficulty. Results on eight datasets show improved performance across five evaluation metrics, and future work recommends combining data-level methods with the proposed boosting strategy.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages6

Reading Time15 min

AccessFree to download

Introduction

Background on class imbalance

Approaches to handle imbalanced data

Challenges of multiclass imbalance

Related work on multiclass imbalance datasets

INTERNATIONAL JOURNAL ON INFORMATICS VISUALIZATION [journal homepage : www.joiv.org/index.php/joiv](journal homepage : www.joiv.org/index.php/joiv) Extreme Gradient Boosting Algorithm to Improve Machine Learning Model Performance on Multiclass Imbalanced Dataset Yoga Pristyantoa,*, Zulfikar Mukarabimana, Anggit Ferdita Nugrahaa a Faculty of Computer Science, Universitas Amikom Yogyakarta, Yogyakarta, Indonesia Corresponding author:*[yoga.pristyanto@amikom.ac.id](yoga.pristyanto@amikom.ac.id) Abstract—Unbalanced conditions in the dataset often become a real-world problem, especially in machine learning. Class imbalance in the dataset is a condition where the number of minority classes is much smaller than the majority class, or the number is insufficient. Machine learning models tend to recognize patterns in the majority class more than in the minority class. This problem is one of the most critical challenges in machine learning research, so several methods have been developed to overcome it. However, most of these methods only focus on binary datasets, so few methods still focus on multiclass datasets. Handling unbalanced multiclass is more complex than handling unbalanced binary because it involves more classes than binary class datasets. With these problems, we need an algorithm with features that can support adjustments to the difficulties that arise in multiclass unbalanced datasets. One of the algorithms that have features for adjustment is the ensemble algorithm, namely Xtreme Gradient Boosting. Based on the research, our proposed method with Xtreme Gradient Boosting showed better results than the other classification and ensemble algorithms on eight datasets with five evaluation metrics indicators such as balanced accuracy, the geometric-mean, multiclass area under the curve, true positive rate, and true negative rate. In future research, we suggest combining methods at the data level and Xtreme Gradient Boosting. With the performance increase in Xtreme Gradient Boosting, it can be a solution and reference in the case of handling multiclass imbalanced problems. Besides, we also recommended testing with datasets in the form of categorical and continuous data. Keywords—Class imbalanced; ensemble algorithm; XGBoost; classification; multiclass. Manuscript received 12 Aug. 2022; revised 31 Dec. 2022; accepted 12 Jan. 2023. Date of publication 10 Sep. 2023. International Journal on Informatics Visualization is licensed under a Creative Commons Attribution-Share Alike 4.0 International License. I. INTRODUCTION Class imbalance in the dataset is a situation or condition where the value of the minority class is much smaller than that of the majority class or so inadequate that the model recognizes patterns in the majority class more than the minority class. For example, in the medical world, the amount of data from patients with diabetes is less than from patients who do not have diabetes. This problem can result in an inaccurate classification and be fatal if implemented in the real world [1]–[3]. The problem of imbalanced datasets is oneof the most critical challenges in the machine learning research community. Various methods have been developed to overcome these problems, such as resampling methods, cost-sensitive approaches, ensemble learning algorithms, kernel-based methods, and active learning methods. These techniques can be categorized into several approaches based on how to overcome the problem of imbalanced data. The first approach at the algorithm level is to create or modify an algorithm to consider the significance of the positive class. Algorithm-based approaches include cost-sensitive methods and recognition-based approaches [4], [5] . The second approach, namely the data level, is carried out at the pre-data processing stage. The class distribution in the data is rebalanced to reduce the effect of the number of majority classes being too dominant in the learning process. One alternative t

FAQs

What is class imbalance in a dataset and why is it a problem in machine learning?

Class imbalance occurs when minority classes have far fewer samples than majority classes, or too few samples to learn reliable patterns. Models then recognize majority-class patterns more strongly, leading to inaccurate classification.

Why is handling multiclass imbalance harder than handling binary imbalance?

Multiclass imbalance requires distinguishing among more classes, making decision boundaries more complex. Multiclass data also introduces additional difficulties such as overlapping classes, small sample sizes, and uneven class distribution slopes.

How does the proposed Extreme Gradient Boosting approach help improve performance on multiclass imbalanced datasets?

The method uses an ensemble boosting algorithm (Xtreme Gradient Boosting) designed for adjusting learning toward imbalanced difficulties. Experiments on eight datasets show better results than other classification and ensemble approaches across multiple metrics.

Aug 4, 2026

Upload by Hazel