Improving image-retrieval performance of foundation models in gastrointestinal endoscopic images - article

Sep 28, 2026

Uploaded by Mia

Gastrointestinal endoscopy quality control requires selecting specific landmark images among large volumes of procedure photos, a task that is slow and burdens manual review. A dual-backbone content-based image-retrieval framework is introduced, combining a general vision foundation model (DINOv2) with a domain-specific endoscopic model (GastroNet). Parameter-efficient metric learning trains discriminative embeddings for efficient similarity search, enabling automated identification of relevant endoscopic images. Evaluation on 3,500 public images and validation on unseen real-world and synthetic data show state-of-the-art retrieval performance and improved generalization.

LanguageEnglish

CategoryHealthcare

FormatPDF

Pages12

Reading Time30 min

AccessFree to download

Introduction

Materials and methods

Results

Discussion

TYPE Original Research PUBLISHED 18 December 2025 DOI 10.3389/fmed.2025.1727884 OPEN ACCESS EDITED BY Palak Handa, Danube Private University, Austria REVIEWED BY Kai Zhang, Improving image-retrieval performance of foundation models in gastrointestinal endoscopic images Chongqing Chang’an Industrial Co., Ltd., China Imran Iqbal, Helmholtz Association of German Research Centres (HZ), Germany Nima Torbati, Danube Private University, Austria *CORRESPONDENCE Youngbae Hwang [ybhwang@cbnu.ac.kr](ybhwang@cbnu.ac.kr) †These authors have contributed equally to this work and share first authorship RECEIVED 18 October 2025 REVISED 19 November 2025 ACCEPTED 26 November 2025 PUBLISHED 18 December 2025 CITATION Kim K, Park J, Kim SH and Hwang Y (2025) Improving image-retrieval performance of foundation models in gastrointestinal endoscopic images. Front. Med. 12:1727884 . doi: 10.3389/fmed.2025.1727884 COPYRIGHT © 2025 Kim, Park, Kim and Hwang. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY) . The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms. Kangsan Kim 1†, Junseok Park 2†, Sang Hyun Kim3 and Youngbae Hwang 1* 1Department of Intelligent Systems and Robotics, Chungbuk National University, Cheongju, Republic of Korea, 2Department of Internal Medicine, Soonchunhyang University College of Medicine, Seoul, Republic of Korea, 3Department of Surgery, Soonchunhyang University College of Medicine, Seoul, Republic of Korea Introduction: The quality of gastrointestinal endoscopy is verified by documenting specific required images, but identifying these images from the numerous photographs captured during a procedure is tedious. Conventional deep-learning approaches that aim to automate this process are often limited by subjective assessments and poor interpretability. Methods: We introduce a novel content-based image-retrieval framework that employs a dual-backbone architecture, integrating a general-purpose vision foundation model (DINOv2) and a domain-specific endoscopic model (GastroNet) . The system is trained using parameter-efficient metric learning to generate discriminative embeddings for efficient similarity searches. The framework is evaluated on 3,500 public endoscopic images (from the Kvasir and HyperKvasir datasets) and validated on entirely unseen real-world and synthetic data. Results: Our model achieves state-of-the-art performance (97.71% Recall@1, 99.14% Recall@5, and 96.74% mean average precision), which is significantly superior to those of single-backbone baseline models. Ablation studies confirm that this improvement is primarily due to the two backbones capturing complementary features. Discussion: These findings demonstrate that the proposed dual-backbone framework offers an accurate and automated tool for assessing the procedural quality of gastrointestinal endoscopy and may facilitate more reliable quality control in clinical practice. KEYWORDS gastrointestinal endoscope, artificial intelligence, deep learning, image retrieval, foundation model 1 Introduction Endoscopy is a critical diagnostic tool for gastrointestinal diseases, which enables the direct visual inspection of internal organs. For instance, during esophagogastroduodenoscopy (EGD), the entire stomach cannot be visualized in a single field of view; therefore, images of specific anatomical segments must be captured ( 1) . These landmark images serve as crucial quality control indicators (2, 3). Currently, manually verifying these images from the numerous photographs taken is a tedious task, prompting the automation of the process (4, 5). However, tradi

FAQs

What problem does the study address in gastrointestinal endoscopy quality control?

It targets the difficulty of identifying required landmark images from thousands of endoscopic photographs captured during a procedure.

How does the proposed method retrieve images?

It maps a query image and database images into discriminative embeddings using a dual-backbone architecture, then ranks database items by similarity (e.g., cosine similarity).

What performance does the model achieve compared with single-backbone baselines?

It reports state-of-the-art results with 97.71% Recall@1, 99.14% Recall@5, and 96.74% mean average precision, with ablations indicating complementary features from both backbones.

Sep 28, 2026

Uploaded by Mia