Planet-wide performance of a skin disease AI algorithm validated in Korea

Sep 23, 2026

Uploaded by Mary Man

Curated a large Korea hospital dataset covering 70 skin diseases with 152,443 images and paired it with real-world web application data totaling 1,691,032 requests to evaluate an AI algorithm for dermatologic surveillance. Proposed a conservative assessment using sensitivity in hospitals and specificity in real-world use while assuming malignancy predictions were false positives. Reported Korea skin cancer sensitivity of 78.2% (NIA) and specificity of 88.0% (webapp), with Top-1/Top-3 accuracies of 43.3% and 66.6%. Web data supported global prevalence insights across 228 countries.

LanguageEnglish

CategoryHealthcare

FormatPDF

Pages11

Reading Time28 min

AccessFree to download

Background

Data sources

Evaluation method

Results

Sensitivity and specificity

Classification accuracy across diseases

Global prevalence insights

Implications for dermatologic surveillance

npj | digital medicine Article | Published in partnership with Seoul National University Bundang Hospital | | | | --- | --- | --- | | [https://doi.org/10.1038/s41746-025-01980-w](https://doi.org/10.1038/s41746-025-01980-w) | | | | Planet-wide performanceof a skindisease AI algorithm validated in Korea
Check for updates | | | | Seung Seog Han1,14, Soo Ick Cho2,14, Gröger Fabian3, Alexander A. Navarini3, Myoung Shin Kim4, Dong Hun Lee5, Ju Hee Lee6, Jihee Kim7, Chong Hyun Won7, Kyung-Nam Bae8, Jee-Bum Lee9, Hyun-Sun Yoon10, Sung Eun Chang7, Seong Hwan Kim11 , Jung Im Na12 &
Cristian Navarrete-Dechent13 | | | | To address the diversity of skin conditions and the low prevalence of skin cancers, we curated a large hospital dataset (National Information Society Agency, Seoul, Korea [NIA] dataset; 70 diseases, 152,443 images) and collected real-world webapp data ([https://modelderm.com](https://modelderm.com) ; 1,691,032 requests) . We propose a conservative evaluation method by assessing sensitivity in hospitals and speciﬁcity in real-world use, assuming all malignancy predictions were false positives. Based on three differential diagnoses, skin cancer sensitivity in Korea was 78.2%(NIA) and speciﬁcity was 88.0%(webapp). Top-1 and Top-3 accuracies for 70 diseases (NIA) were 43.3% and 66.6%, respectively. Analysis of webapp data provides insights into disease prevalence and public interest across 228 countries. Malignancy predictions were highest in North America (2.6%) and lowest in Africa (0.9%), while benign tumors were most common in Asia(55.5%), and infectious diseases were most prevalent in Africa (17.1%) . These ﬁndings suggest that AI can aid global dermatologic surveillance. | | | | Artiﬁcial intelligence (AI) has demonstrated remarkable performance in dermatology, often surpassing dermatologists in certain controlled conditions1. Those studies, mostly encompassing skin cancer and utilizing clinical photographs and dermoscopy images, have shown that AI can outperform dermatologists when diagnoses are based solely on image data2 –5. The performance of various AI models has been validated using a variety of publicly available dermatology datasets6–14.
Unlike in radiology and ophthalmology, vision AI models in dermatology have not been yet clinically successful for several reasons. First, AI performance in dermatological conditions tends to be overestimated in the reader test when conducted with limited information. Unlike radiologists who detect speciﬁc ﬁndings from X-ray images or ophthalmologists who interpret fundus imaging to determine diabetic retinopathy, dermatologists rely on a variety of contextual and patient-speciﬁc information outside of imaging to make a diagnosis. Second, AI performance can decrease due to | out-of-distribution (OOD) cases in real-world situations. While images utilized in radiology or ophthalmology are taken with specialized medical equipment or by medical staffs following DICOM standards, resulting in more standardized samples, most images used in dermatology outside research settings are taken with non-medical equipment such as nonprofessional consumer cameras or smartphone cameras in a nonstandardized manner and are often acquired by non-medical personnel, resulting in frequent OOD scenarios15.
Consequently, while AI may outperform dermatologists in purely image-based diagnoses using preprocessed in-distribution datasets, clinicians in real practice tend to be more accurate16. Historical examples, such as the MelaFind algorithm, which received FDA approval but ultimately disappeared from the market due to high false-positive rates, highlight how AI algorithms that perform well under controlled conditions can struggle in real-world applications17. In addition, the results may be distorted because | | 1I Dermatology Clinic, Seoul, South Korea. 2InSkin Lab Inc. , Seoul, South Korea. 3Department of Dermatology, University of Basel, Basel, Switzerland. 4Department of De

FAQs

What data sources were used to evaluate the skin disease AI algorithm?

A large hospital dataset from the Korea NIA source (70 diseases, 152,443 images) and real-world web app usage data from modelderm.com (1,691,032 requests) were used.

How was conservative performance evaluation designed in the study?

Sensitivity was assessed in hospitals while specificity was assessed in real-world use, assuming malignancy predictions were false positives to prevent overestimation.

What were the main performance metrics reported for Korea?

Skin cancer sensitivity in Korea was 78.2% (NIA) and specificity was 88.0% (webapp). Top-1 and Top-3 accuracies for 70 diseases were 43.3% and 66.6%.

Sep 23, 2026

Uploaded by Mary Man