[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-124608-en":3,"doc-seo-124608-105":30,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":4,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":27,"seo_description":14,"update_tm":28,"read_time":29},124608,687197100911,"Himbo","https://ap-avatar.wpscdn.com/avatar/a000239b6f1da00475?x-image-process=image/resize,m_fixed,w_180,h_180&k=1785132997149421697",7,"Healthcare","Generalist Vision Foundation Models for Medical Imaging - A Case Study of Segment Anything Model on Zero-shot Medical Segmentation","This study evaluates the Segment Anything Model (SAM) on medical images through both quantitative and qualitative zero-shot segmentation experiments. Results are reported across nine medical segmentation benchmarks spanning multiple modalities including OCT, MRI, and CT, as well as applications in dermatology, ophthalmology, and radiology. SAM shows strong performance within general-domain images, but limited zero-shot capability for out-of-distribution medical images, with inconsistent results across unseen domains and complete failure for structured targets such as blood vessels. Limited fine-tuning with small datasets markedly improves segmentation quality, demonstrating the feasibility of adapting generalist vision foundation models for precision diagnostics when large, diverse medical data are hard to access.","Generalist Vision Foundation Models for Medical Imaging: A Case Study of Segment Anything Model on Zero-Shot Medical Segmentation  \narXiv :2304 . 12637v2 [ cs .CV] 5 Jun 2023  \nPeilun Shi† Jianing Qiu†,‡ Sai Mu Dalike Abaxi† Hao Wei†  \nFrank P.-W. Lo‡  \n†The Chinese University of Hong Kong  \n{peilunshi, tariqabaxi, [wyuan](wyuan}@cuhk.edu.hk)[}](wyuan}@cuhk.edu.hk)[@cuhk.edu.hk](wyuan}@cuhk.edu.hk)[haowei@link.cuhk.edu.hk](haowei@link.cuhk.edu.hk)  \nWu Yuan†, *  \n‡Imperial College London  \n{jianing.qiu17, [po.lo15](po.lo15}@imperial.ac.uk)[}](po.lo15}@imperial.ac.uk)[@imperial.ac.uk](po.lo15}@imperial.ac.uk)  \nAbstract  \nIn this paper, we examine the recent Segment Anything Model (SAM) on medical images, and report both quantitative and qualitative zero-shot segmentation results on nine medical image segmentation benchmarks, covering various imaging modalities, such as optical coherence tomography (OCT), magnetic resonance imaging (MRI), and computed tomography (CT), as well as different applications including dermatology, ophthalmology, and radiology. Those benchmarks are representative and commonly used in model development. Our experimental results indicate that while SAM presents remarkable segmentation performance on images from the general domain, its zero-shot segmentation ability remains restricted for out-of-distribution images, e.g., medical images. In addition, SAM exhibits inconsistent zero-shot segmentation performance across different unseen medical domains. For certain structured targets, e.g., blood vessels, the zero-shot segmentation of SAM completely failed. In contrast, a simple fine-tuning of it with a small amount of data could lead to remarkable improvement of the segmentation quality, showing the great potential and feasibility of using fine-tuned SAM to achieve accurate medical image segmentation for a precision diagnostics. Our study indicates the versatility of generalist vision foundation models on medical imaging, and their great potential to achieve desired performance through fine-turning and eventually address the challenges associated with accessing large and diverse medical datasets in support of clinical diagnostics.  \n1. Introduction  \nRecently, large AI models (LAMs) have been actively researched as they manifest impressive performance on var-  \n*Corresponding author  \nious downstream tasks and offer a foundation to advance and foster future research in manifold AI areas, such as computer vision and natural language processing [4, 25] . In medical and healthcare domains, LAMs are also transforming methodological designs and paradigms, and establishing new state-of-the-arts and breakthroughs in various sectors including medical informatics and decision-making [31] . Despite the active development, advances in medical LAMs often lag behind their counterparts in general domains. To identify current discrepancies and guide the future development of medical LAMs, we select one of these LAMs in the general domain, i.e., Segment Anything Model (SAM) [22], which is a foundational vision model recently proposed for image segmentation and has shown stunning performance on tasks ranging from edge detection to instance segmentation, and thoroughly evaluate its zero-shot segmentation performance on medical images. Although there are few studies out there that tested SAM on medical imaging, they either only focus on one imaging modality, i.e., pathology [11], or only showcase a few qualitative segmentation samples [20] without reporting quantitative results. To provide a comprehensive and objective evaluation of SAM on medical image segmentation, this work conducted extensive experiments on nine benchmarks using the zero-shot segmentation feature of SAM. The selected datasets contain a wide diversity of medical imaging modalities and organs.  \nOur key findings include:  \n1. SAM demonstrated better performance on endoscopic and dermoscopic images than other medical modalities, which is conjectured as SAM w","cbCaiip3q34CcfNT","https://ap.wps.com/l/cbCaiip3q34CcfNT","pdf",3183657,1,12,"English","en",105,"# Introduction\n# Segment Anything Model\n# Zero-shot Medical Segmentation Evaluation\n# Fine-tuning Strategy and Results\n# Key Findings","[{\"question\":\"What evaluation approach is used to test SAM on medical images?\",\"answer\":\"The study performs extensive experiments using SAM’s zero-shot segmentation capability across nine medical image segmentation benchmarks, reporting both quantitative and qualitative results.\"},{\"question\":\"How does SAM perform on out-of-distribution medical images compared with general-domain images?\",\"answer\":\"SAM achieves remarkable performance on general-domain images, but its zero-shot segmentation ability is restricted on out-of-distribution medical images, with inconsistent results across unseen medical domains.\"},{\"question\":\"What happens when SAM is applied to structured targets like blood vessels?\",\"answer\":\"For structured targets such as blood vessels, SAM’s zero-shot segmentation completely fails, and it also struggles with continuous branching structures like tree branches.\"}]","Generalist Vision Foundation Models for Medical Imaging - A Case Study of Segment Anything Model on Zero-shot Medical Segmentation | PDF",1785893292,30,{"code":4,"msg":31,"data":32},"ok",{"site_id":24,"language":23,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":28},"generalist-vision-foundation-models-for-medical-imaging-a-case-study-of-segment-anything-model-on-zero-shot-medical-segmentation","",{"@graph":36,"@context":85},[37,54,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/healthcare/",3,{"item":52,"name":13,"@type":43,"position":53},"https://docshare.wps.com/document/generalist-vision-foundation-models-for-medical-imaging-a-case-study-of-segment-anything-model-on-zero-shot-medical-segmentation/124608/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":23,"description":14,"dateModified":62,"datePublished":62,"encodingFormat":61,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":41,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-05",true,{"@type":65,"interactionType":66,"userInteractionCount":4},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"What evaluation approach is used to test SAM on medical images?","Question",{"text":75,"@type":76},"The study performs extensive experiments using SAM’s zero-shot segmentation capability across nine medical image segmentation benchmarks, reporting both quantitative and qualitative results.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"How does SAM perform on out-of-distribution medical images compared with general-domain images?",{"text":80,"@type":76},"SAM achieves remarkable performance on general-domain images, but its zero-shot segmentation ability is restricted on out-of-distribution medical images, with inconsistent results across unseen medical domains.",{"name":82,"@type":73,"acceptedAnswer":83},"What happens when SAM is applied to structured targets like blood vessels?",{"text":84,"@type":76},"For structured targets such as blood vessels, SAM’s zero-shot segmentation completely fails, and it also struggles with continuous branching structures like tree branches.","https://schema.org",{"og:url":52,"og:type":87,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":89,"canonical":52},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,118,122,127,130,134],{"id":20,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":46,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":116,"slug":117},40,"healthcare",{"id":119,"doc_module":4,"doc_module_name":46,"category_name":120,"show_sort_weight":29,"slug":121},8,"Research & Report","research-report",{"id":123,"doc_module":4,"doc_module_name":46,"category_name":124,"show_sort_weight":125,"slug":126},9,"Religion & Spirituality",20,"religion-spirituality",{"id":125,"doc_module":4,"doc_module_name":46,"category_name":128,"show_sort_weight":125,"slug":129},"World Cup","world-cup",{"id":131,"doc_module":4,"doc_module_name":46,"category_name":132,"show_sort_weight":131,"slug":133},10,"Lifestyle","lifestyle",{"id":135,"doc_module":4,"doc_module_name":46,"category_name":136,"show_sort_weight":106,"slug":137},19,"General","general"]