Machine Learning and SHAP Interpretability for Chronic Disease Understanding - Diabetes Prediction - Results and Deployment

Aug 3, 2026

Upload by Riley

Non-communicable diseases, especially diabetes, remain a major global health challenge due to the complexity of medical data and the limitations of traditional approaches for prediction and management. This study applies machine learning to diabetes risk estimation and uses hyperparameter tuning for model development. SHAP is incorporated to interpret feature contributions and improve clinical insight. The workflow includes SMOTE for class imbalance, model evaluation with accuracy/precision/recall/F1, and a Streamlit web interface enabling real-time, explainable predictions for healthcare providers.

LanguageEnglish

CategoryHealthcare

FormatPDF

Pages1

Reading Time3 min

AccessFree to download

Introduction

Problem Statement

Project Goals

Specifications

Methodology

Data Preparation

Model Training and Evaluation

Deployment and Interpretability

Results

Discussion

Conclusion

Reference

| Machine Learning and SHAP Interpretability for Chronic Disease Understanding
Nnaemeka Charles Igwe, Khandaker Mamun Ahmed
DAKOTA STATE UNIVERSITY | | | --- | --- | Introduction • Non-communicable diseases (NCDs), such as diabetes, are major global health concerns influenced by various health parameters and lifestyle choices. • Traditional methods struggle to efficiently predict and manage these conditions due to the complexity and diversity of medical data. • There is a need to leverage machine learning algorithms and modern computational tools to accurately predict diabetes, improve diagnosis, and provide actionable insights for better healthcare outcomes. • There are various health parameters and lifestyle choices responsible for diabetes. • Key health parameters include age, insulin level, body mass index (BMI), and family history. Problem Statement • Our study predicts non-communicable diseases (NCDs), such as diabetes, by leveraging machine learning algorithms. • We leverage hyperparameter tuning techniques for model development and SHapley Additive exPlanation (SHAP) for results interpretations. Project Goals • Train machine learning models for diabetes prediction • Utilize SHAP for feature importance and model interpretability. • Handle class imbalance using SMOTE. • Identify key factors affecting diabetes risk. • Develop a user-friendly web interface for healthcare providers using STREAMLIT. Specifications • Data: We use the Pima Indian dataset that includes raw (diabetes.csv) and processed datasets (updated_diabetes_dataset.csv) and SMOTE to handle class imbalance. • Key Tools: Python for scripting, Streamlit for building a web app interface, and SHAP for explaining black box AI model predictions. Methodology • Data Preparation: We use the PIMA Indian diabetes dataset, preprocess data and select feature using a correlation matrix. • Model Training and Evaluation: We train different machine learning models (SVM, KNN, Logistic Regression(LR)), fine-tune the hyperparameters, perform cross-validation, and evaluate the models’ performance using metrics like accuracy, precision, recall, and F1-score. • Deployment and Interpretability: We deploy a Streamlit-based web app for real-time predictions and utilize SHAP to explain feature importance and model decisions. • • • • Fig 1: The overall architecture of our proposed method Results Fig 3. F-1 score, Recall, Accuracy, Precision of SVM, KNN, and LR models Fig 5: Individual feature’s contribution to the prediction. Fig 4: SHAP interpretation of features importance in predicting diabetes using the SVM model Fig 6: Compares the performance of LinearSVC, KNN, and LR after applying SMOTE • • • • • • Discussion Figure 2 and 3 show the comparison of SVM, KNN, and LR models shows that SVM and KNN consistently outperform LR in key metrics like accuracy, precision, recall, and F1-score. SVM displayed the most robust performance overall, while KNN was competitive in precision and recall. LR, although less effective, demonstrated acceptable results in simpler scenarios. Glucose is the most influential feature, with higher values strongly contributing to a positive diabetes prediction. Age and BMI are also significant predictors, where higher values generally indicate an increased risk. The plot visually distinguishes high (pink) and low (blue) feature values and their corresponding SHAP values, showing how individual features influence the model's predictions. LinearSVC performs the best, with the highest Accuracy (0 .76) and F1-Score (0 .63) . Conclusion This study highlights the importance of explainability and user-centric tools in deploying AI for healthcare cyber-physical systems. ML models (SVM, KNN, and LR) were applied to predict diabetes using the Pima Indian Diabetes dataset. SVM performed best, achieving 76% accuracy and the highest F1-score, while KNN followed with 75% accurac

FAQs

How does the study predict diabetes risk?

It trains multiple machine learning models (SVM, KNN, and Logistic Regression) using the Pima Indian diabetes dataset and evaluates them with standard classification metrics.

Why is SHAP used in this project?

SHAP explains model decisions by identifying individual feature contributions to diabetes predictions, improving interpretability of the otherwise black-box outputs.

How is class imbalance handled and how is the system deployed?

SMOTE is applied during dataset preparation to balance classes. A Streamlit-based web app is deployed to provide real-time predictions with SHAP-based explanations.

Aug 3, 2026

Upload by Riley