Dermacen Analytica - A Novel Methodology Integrating Multi-Modal Large Language Models with Machine Learning in tele-dermatology

Aug 4, 2026

Upload by Hazel

The rise of Artificial Intelligence creates great promise in medical discovery, diagnostics, and patient management, yet the complexity of medical domains requires an integrated approach. Dermacen Analytica presents an AI-empowered system and methodology that assists diagnosis of skin lesions and related dermatological conditions by combining machine learning algorithms, segmentation tools, and large language models. The workflow integrates large language models with transformer-based vision models and a multi-stage evaluation pipeline using publicly available medical case studies and images. Performance is scored through machine learning and natural language processing techniques emphasizing similarity comparison and natural language inference, supported by expert evaluation with a structured checklist, achieving weighted scores of about 0.87 for contextual understanding and diagnostic accuracy.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages46

Reading Time1 h 56 min

AccessFree to download

Introduction

AI-enabled dermatology diagnostics

Proposed Dermacen Analytica workflow

Key workflow components

arXiv :2403 . 14243v1 [ cs .CL] 21 Mar 2024 Dermacen Analytica: A Novel Methodology Integrating Multi-Modal Large Language Models with Machine Learning in tele-dermatology Dimitrios P. Panagouliasa (panagoulias [d@unipi.gr](d@unipi.gr)), Evridiki Tsoureli-Nikitab ([evinikita@gmail.com](evinikita@gmail.com)), Maria Virvoua ([mvirvou@unipi.gr](mvirvou@unipi.gr)), George A. Tsihrintzisa ([geoatsi@unipi.gr](geoatsi@unipi.gr)) a Department of Informatics, University of Piraeus 185 34, Greece b Athens University Medical School, Athens 115 27, Greece Corresponding Author: George A. Tsihrintzis Department of Informatics, University of Piraeus 185 34, Greece Tel: (+30) 697 2882168 Email: [geoatsi@unipi.gr](geoatsi@unipi.gr) Dermacen Analytica: A Novel Methodology Integrating Multi-Modal Large Language Models with Machine Learning in tele-dermatology Dimitrios P. Panagouliasa , Evridiki Tsoureli-Nikitab , Maria Virvoua , George A. Tsihrintzisa,1 a Department of Informatics, University of Piraeus, Piraeus 185 34, Greece b Athens University Medical School, Athens, Greece Abstract The rise of Artificial Intelligence creates great promise in the field of medical discovery, diagnostics and patient management. However, the vast complexity of all medical domains require a more complex approach that combines machine learning algorithms, classifiers, segmentation algorithms and, lately, large language models. In this paper, we describe, implement and assess an Artificial Intelligence-empowered system and methodology aimed at assisting the diagnosis process of skin lesions and other skin conditions within the field of dermatology that aims to holistically address the diagnostic process in this domain. The workflow integrates large language, transformer-based vision models and sophisticated machine learning tools. This holistic approach achieves a nuanced interpretation of dermatological conditions that simulates and facilitatesa dermatologist’s workflow. We assess our proposed methodology through a thorough cross-model validation technique embedded in an evaluation pipeline that utilizes publicly available medical case studies of skin conditions and relevant images. To quantitatively score the system performance, advanced machine learning and natural language processing tools are employed which focus on similarity comparison and natural language inference. Additionally, we incorporate ∗ Corresponding author. Email addresses: [panagoulias_d@unipi.gr](panagoulias_d@unipi.gr) (Dimitrios P. Panagoulias), [evinikita@gmail.com](evinikita@gmail.com) (Evridiki Tsoureli-Nikita), [mvirvou@unipi.gr](mvirvou@unipi.gr) (Maria Virvou), [geoatsi@unipi.gr](geoatsi@unipi.gr) (George A. Tsihrintzis) Preprint submitted to arXiv March 22, 2024 a human expert evaluation process based on a structured checklist to further validate our results. We implemented the proposed methodology in a system which achieved approximate (weighted) scores of 0.87 for both contextual understanding and diagnostic accuracy, demonstrating the efficacy of our approach in enhancing dermatological analysis. The proposed methodology is expected to prove useful in the development of next-generation tele-dermatology applications, enhancing remote consultation capabilities and access to care, especially in underserved areas. Keywords: Artificial Intelligence-empowered software engineering, Multimodal Large Language Models, GPT-4V, Telehealth 1. Introduction The rise of Artificial Intelligence (AI) and its transformative effect on many fields and domains provides significant opportunities for the revamping and reengineering of the diagnostic process. However, in order to navigate through the complexity of the medical field effectively, a multifaceted approach is essential that integrates several AI technologies including explainability (Panagoulias et al. , 2024b), trustworthiness and holistic evaluation properties (Virvou, 2023) . Machine learning alg

FAQs

How is the methodology evaluated and validated?

It uses a thorough cross-model validation technique with an evaluation pipeline based on publicly available medical case studies and images, plus quantitative scoring via similarity comparison and natural language inference and expert evaluation using a structured checklist.

Aug 4, 2026

Upload by Hazel