Evaluation of the effectiveness of the ChatGPT artificial intelligence application in the diagnosis of spontaneous pneumothorax on chest radiograph interpretation

Sep 30, 2026

Uploaded by Kurz

Evaluation of the effectiveness of the ChatGPT artificial intelligence application in diagnosing spontaneous pneumothorax on chest radiographs targets a clinically critical problem where image interpretation is difficult due to anatomical overlap and observer variability. A retrospective dataset of 220 radiographs was classified by expert consensus into 110 pneumothorax and 110 non-pneumothorax cases, then analyzed by GPT-4o without clinical information. Diagnostic accuracy, sensitivity, specificity, predictive values, and AUC were computed, with performance assessed by pneumothorax size and compared with expert evaluations. Results indicate overall strong specificity and accuracy, better detection for large pneumothoraces, and limited sensitivity for small cases, supporting cautious use with expert oversight rather than stand-alone triage.

LanguageEnglish

CategoryHealthcare

FormatPDF

Pages8

Reading Time20 min

AccessFree to download

Abstract

Background

Methods

Results

Conclusions

Keywords

Introduction

Akçay etal. BMC Pulmonary Medicine (2026) 26:8 [https://doi.org/10.1186/s12890-025-04041-w](https://doi.org/10.1186/s12890-025-04041-w) BMC Pulmonary Medicine RESEARCH Open Access Evaluation of the effectiveness of the ChatGPT artificial intelligence application in the diagnosis of spontaneous pneumothorax on chest radiograph interpretation Onur Akçay 1*, Azat Özel 1, Özgür Öztürk 1*, Tuba Acar 1, Ahmet Kayahan Tekneci2, Tevfik İlker Akçam2 and Soner Gürsoy 1 Abstract Background Spontaneous pneumothorax is a potentially life-threatening condition commonly diagnosed using chest radiographs. However, interpreting chest X-rays can be challenging due to anatomical overlap and observer variability. This study aimed to evaluate the diagnostic accuracy of ChatGPT, a large language model (LLM), in detecting pneumothorax on chest radiographs compared to expert thoracic surgeons. Methods In this retrospective study, 220 chest radiographs were assessed. Expert consensus classified 110 cases with pneumothorax and 110 without. The images were uploaded to the GPT-4o model without any clinical information, and ChatGPT was asked to identify the presence or absence of pneumothorax. Diagnostic performance was evaluated by calculating sensitivity, specificity, accuracy, positive and negative predictive values, and area under the receiver operating characteristic curve (AUC) . Subgroup analyses were performed based on pneumothorax size. Results ChatGPT demonstrated an overall diagnostic accuracy of 83 . 7%, sensitivity of 70 . 9%, specificity of 96 .4%, positive predictive value of 95 . 1%, and negative predictive value of 76. 8% . The AUC was 0.836 (95% CI: 0 .780-0. 893) . Diagnostic performance was higher for large pneumothoraces (AUC: 0 . 894) compared to small pneumothoraces (AUC: 0 .439) . Cohen’s kappa coefficient indicated substantial agreement (κ=0 . 673; 95%CI: 0 .575-0. 771) with expert evaluations. Conclusions ChatGPT demonstrates potential in detecting pneumothorax on chest radiographs, particularly in cases of large pneumothorax. However, its limited sensitivity for small pneumothoraces raises significant concerns about its reliability in clinical decision-making. Any use of ChatGPT in diagnostic workflows should be approached with caution, as unverified outputs may lead to inappropriate interventions or under-triaging. Therefore, the model is not suitable as a standalone diagnostic or triage tool. Its potential utility may lie in exploratory or supervised settings where expert oversight is available, but further validation is required before clinical implementation can be considered. *Correspondence: Onur Akçay [onur_akcay@yahoo.com](onur_akcay@yahoo.com)[ ](onur_akcay@yahoo.com)Özgür Öztürk[ozgur.ozturk54@hotmail.com](ozgur.ozturk54@hotmail.com) Full list of author information is available at the end of the article © The Author(s) 2025. Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit [http://creati](http://creati)[vecommons.org/l](vecommons.org/l)icenses/by-nc-nd/4.0/. Akçay et al. BMC Pulmonary

FAQs

What clinical challenge does the study address?

Spontaneous pneumothorax is potentially life-threatening, and chest radiograph interpretation can be challenging due to anatomical overlap and observer variability.

How was ChatGPT evaluated for pneumothorax detection?

A retrospective set of 220 chest radiographs was uploaded to GPT-4o without clinical information, and ChatGPT was asked to identify the presence or absence of pneumothorax. Performance was assessed with sensitivity, specificity, accuracy, predictive values, and AUC.

What were the main findings about diagnostic performance?

ChatGPT achieved overall diagnostic accuracy of 83.7% with high specificity (96.4%). Performance was stronger for large pneumothoraces than for small pneumothoraces, reflecting limited sensitivity for small cases.

Sep 30, 2026

Uploaded by Kurz