[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-117382-en":3,"doc-seo-117382-105":30,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":4,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":27,"seo_description":14,"update_tm":28,"read_time":29},117382,7971461740886,"Theodore","https://ap-avatar.wpscdn.com/davatar_3d24733baf745e90a7e4bdd5f77d97b2",8,"Research & Report","Automated Ensemble Multimodal Machine Learning for Healthcare","Machine learning in medicine has produced many diagnostic and prognostic models, yet most methods rely on single-modality inputs, unlike clinical decisions that combine information from multiple sources. This paper presents AutoPrognosis-M, an automated machine learning framework that integrates structured tabular clinical data with medical imaging. The system includes 17 imaging models and three multimodal fusion strategies, and ensemble learning is evaluated on a multimodal skin lesion dataset to demonstrate performance and adoption potential for healthcare informatics.","This article has been accepted for publication in IEEE Journal of Biomedical and Health Informatics. This is the author's version which has not been fully edited and content may change prior to final publication. Citation information: DOI 10. 1109/JBHI.2025.3530156  \n IEEE TRANSACTIONS AND JOURNALS TEMPLATE 1  \nAutomated Ensemble Multimodal Machine Learning for Healthcare  \nFergus Imrie* , Stefan Denner* , Lucas S. Brunschwig, Klaus Maier-Hein and Mihaela van der Schaar,  \nFellow, IEEE  \nAbstract—The application of machine learning in medicine and healthcare has led to the creation of numerous diagnostic and prognostic models. However, despite their success, current approaches generally issue predictions using data from a single modality. This stands in stark contrast with clinician decision-making which employs diverse information from multiple sources. While several multimodal machine learning approaches exist, significant challenges in developing multimodal systems remain that are hindering clinical adoption. In this paper, we introduce a multimodal framework, AutoPrognosis-M, that enables the integration of structured clinical (tabular) data and medical imaging using automated machine learning. AutoPrognosis-M incorporates 17 imaging models, including convolutional neural networks and vision transformers, and three distinct multimodal fusion strategies. In an illustrative application using a multimodal skin lesion dataset, we highlight the importance of multimodal machine learning and the power of combining multiple fusion strategies using ensemble learning. We have open-sourced our framework as a tool for the community and hope it will accelerate the uptake of multimodal machine learning in healthcare and spur further innovation.  \nIndex Terms—Multimodal Machine Learning, Medical Imaging, Machine Learning, Deep Learning, Automated Machine Learning, Cancer, Biomedicine, Healthcare Informatics  \nI. INTRODUCTION  \nHealthcare data is increasingly diverse in origin and nature, encompassing patient records and imaging to genetic information and real-time biometrics. Machine learning (ML) can learn complex relationships from data and has shown promise for diagnostic and prognostic modeling [1],[2] . However, such  \n*  \nFergus Imrie and Stefan Denner contributed equally.  \nFergus Imrie is with the Department of Statistics, University of Oxford, Oxford, UK ([e-mail: fergus.imrie@stats.ox.ac.uk](e-mail: fergus.imrie@stats.ox.ac.uk)).  \nStefan Denner is with the Division of Medical Image Computing, German Cancer Research Center (DKFZ), Germany, and Faculty of Mathematics and Computer Science, Heidelberg University, Germany (e-mail: stefan.denner@dkfz-heidelberg.de) .  \nLucas S. Brunschwig is with ´Ecole Polytechnique Fdrale de Lausanne, Switzerland (e-mail: lucas.brunschwig@epfl.ch) .  \nKlaus Maier-Hein is with the Division of Medical Image Computing, German Cancer Research Center (DKFZ), Germany, Pattern Analysis and Learning Group, Department of Radiation Oncology, Heidelberg University Hospital, Germany, and National Center for Tumor Diseases (NCT) Heidelberg, Germany (e-mail: k.maier-hein@dkfz-heidelberg.de) .  \nMihaela van der Schaar is with Department of Applied Mathematics and Theoretical Physics, University of Cambridge, UK (e-mail: [mv472@cam.ac.uk](mv472@cam.ac.uk)) .  \napproaches typically only use one type or modality of data [3], limiting their ability to consider the broader clinical context. In contrast, clinicians make decisions based on the synthesis of information from multiple sources [4], with numerous studies showing the absence of such information can result in lower performance and decreased clinical utility [5], [6] .  \nThis is perhaps particularly the case in medical imaging. For example, almost 90% of radiologists reported that additional clinical information was important and could change diagnoses compared to imaging alone [7] . Numerous other examples exist across many specialties such as ophthalmology [","cbCaiqbZ66XVZdAu","https://ap.wps.com/l/cbCaiqbZ66XVZdAu","pdf",1434292,1,14,"English","en",105,"# Introduction\n## Motivation for multimodal learning in healthcare\n## Automated machine learning and ensemble-based prognostic modeling\n## AutoPrognosis-M framework overview","[{\"question\":\"Why do current machine learning models in healthcare limit clinical adoption?\",\"answer\":\"Many approaches generate predictions from a single data modality, while clinical practice synthesizes multiple sources of information, which can reduce performance and clinical utility when multimodal context is missing.\"},{\"question\":\"What does AutoPrognosis-M integrate in its multimodal framework?\",\"answer\":\"It combines structured clinical tabular data with medical imaging, using automated machine learning to build multimodal models.\"},{\"question\":\"How is ensemble learning used in the paper’s illustrative application?\",\"answer\":\"Using a multimodal skin lesion dataset, the paper highlights the importance of multimodal learning and demonstrates the benefit of combining multiple fusion strategies through ensemble learning.\"}]","Automated Ensemble Multimodal Machine Learning for Healthcare | PDF",1785675499,35,{"code":4,"msg":31,"data":32},"ok",{"site_id":24,"language":23,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":28},"automated-ensemble-multimodal-machine-learning-for-healthcare","",{"@graph":36,"@context":85},[37,54,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":53},"https://docshare.wps.com/document/automated-ensemble-multimodal-machine-learning-for-healthcare/117382/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":23,"description":14,"dateModified":62,"datePublished":62,"encodingFormat":61,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":41,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-02",true,{"@type":65,"interactionType":66,"userInteractionCount":4},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"Why do current machine learning models in healthcare limit clinical adoption?","Question",{"text":75,"@type":76},"Many approaches generate predictions from a single data modality, while clinical practice synthesizes multiple sources of information, which can reduce performance and clinical utility when multimodal context is missing.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"What does AutoPrognosis-M integrate in its multimodal framework?",{"text":80,"@type":76},"It combines structured clinical tabular data with medical imaging, using automated machine learning to build multimodal models.",{"name":82,"@type":73,"acceptedAnswer":83},"How is ensemble learning used in the paper’s illustrative application?",{"text":84,"@type":76},"Using a multimodal skin lesion dataset, the paper highlights the importance of multimodal learning and demonstrates the benefit of combining multiple fusion strategies through ensemble learning.","https://schema.org",{"og:url":52,"og:type":87,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":89,"canonical":52},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":20,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":46,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":46,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":46,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":46,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":46,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":46,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]