[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-82327-en":3,"doc-seo-82327-105":29,"detail-sidebar-cat-0-en-105":90},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":4,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":13,"seo_description":14,"update_tm":27,"read_time":28},82327,1374391974564,"Clementine","https://ap-avatar.wpscdn.com/avatar/14000253aa45c000a9e?x-image-process=image/resize,m_fixed,w_180,h_180&k=1779874745381141002",6,"Technology","CtrlVTON: Controllable Virtual Try-On via Visual-Instance-Prompt Segmentation","Virtual try-on (VTO) enables photorealistic garment transfer, yet most methods restrict user control over how a garment is worn, including size (loose vs. fitted), style (tucked/untucked, open/closed, zipped/unzipped), and exact spatial placement and layering. CtrlVTON addresses this gap with two parts: VIP-SAM performs instance-level Visual-Instance-Prompt Segmentation from flatlay to on-body images; CtrlVTON reframes try-on as controllable image editing using pixel-level segmentation masks. Results achieve state-of-the-art fidelity and layout adherence.","CtrlVTON: Controllable Virtual Try-On via Visual-Instance-Prompt Segmentation  \narXiv :2607 .09362v1 [ cs .CV] 10 Jul 2026  \nSeungyong Lee 1†, Hyun Jun Jang 1 , Sangoh Kim 1 ,2‡, and Sungjoon Park 1†,*  \n1 NXN Labs  \n2 KAIST  \n[seungyong@nxn.ai](seungyong@nxn.ai) , [hjang@nxn.ai](hjang@nxn.ai) , [tkddh1109@kaist.ac.kr](tkddh1109@kaist.ac.kr) , [sungjoon@nxn.ai](sungjoon@nxn.ai)  \n[https://github.com/nxnai/CtrlVTON](https://github.com/nxnai/CtrlVTON)  \nFig. 1: (a) CtrlVTON-base is a baseline image-editing model that enables semantic control via task tokens (full _swap / partial _swap / add) over multiple garment classes. (b) CtrlVTON enables fine-grained spatial control through hand-drawn masks (yellow), supporting both single-and multi-garment try-on.  \nAbstract. Virtual try-on (VTO) has made significant progress in realistically transferring garments onto a target person. Yet most systems give the user little control over how a garment should be worn—its size (loose or fitted), style (e.g ., tucked in or untucked, open or closed), and spatial placement on the body. We address this gap with two complementary contributions. First, we define and solve Visual-Instance-Prompt Segmentation via VIP-SAM: given a flatlay image of a garment, segment that specific instance in a photograph of a person wearing it. This is an instance-level task, distinct from the typically studied category-level segmentation. Second, we introduce CtrlVTON, a controllable VTO framework that recasts try-on as an image editing problem and adds segmentation masks as pixellevel control over garment layout, including style, size, and spatial placement on the body. VIP-SAM and CtrlVTON each achieve state-of-the-art results on their respective tasks. In particular, CtrlVTON generates images that follow user-provided layouts far more faithfully than the strongest proprietary editing systems while matching them on garment fidelity.  \nKeywords: Virtual Try-On · Controllable Image Generation · Visual-Prompt Segmentation  \n1 Introduction  \nThe fashion and e-commerce industries have long sought to bridge the gap between how a garment appears online and how it looks when worn. Virtual try-on (VTO) addresses this need by synthesizing a photorealistic  \n† Equal contribution.  \n‡ Work done during internship at NXN Labs.  \n∗ Corresponding author.  \n2 S. Lee et al.  \nimage of a person wearing the garment, allowing customers to visualize how they look without physically trying it on. Recent diffusion-based methods have substantially improved photorealism and garment fidelity [10, 12 , 31 , 37 , 43 , 74], making VTO commercially viable. Despite this progress, current VTO methods share a fundamental limitation: they allow users limited control over how a garment should be worn, including size (e.g. loose or fitted), style (e.g. tucked in or untucked, zipped or unzipped), and spatial placement (e.g. spatial position, layering) .  \nTo enable controllability, we start by recasting VTO as an image-editing problem rather than an inpainting problem. This reformulation avoids the well-known failure modes of inpainting-based VTO methods (Sec. B of Supp.) . The resulting model, CtrlVTON-base, handles diverse garment categories (tops, bottoms, dresses, shoes, bags) and garment display formats (flatlay, on-person, in-the-wild) . It also supports two scenarios that conventional VTO methods struggle with: garment layering (adding an item over the outfit) and selective garment switching (replacing only a specific item) . We then notice that even in the editing framework, segmentation masks can be used a pixel-level interface for spatial control. By extending CtrlVTON-base with this capability, we obtain CtrlVTON (Sec. 4), which not only matches the strongest VTO systems on image quality, but also enables fine-grained control over garment style, size, and placement via segmentation masks.  \nThe models rely critically on our data preparation pipeline, which requires an automatic, scalable way ","cbCaimLfpU0vFb3z","https://ap.wps.com/l/cbCaimLfpU0vFb3z","pdf",23322177,1,33,"English","en",105,"# Introduction\n## Virtual Try-On\n## Visual-Instance-Prompt Segmentation (VIP-Seg)\n## CtrlVTON Framework and Pixel-Level Control\n# Related Work\n## Virtual Try-On","[{\"question\":\"What limitation of existing virtual try-on does CtrlVTON focus on?\",\"answer\":\"Existing VTO systems provide limited control over garment size, style, and spatial placement. CtrlVTON targets controllability for how garments are worn, not just realistic transfer.\"},{\"question\":\"How does VIP-SAM contribute to controllable try-on?\",\"answer\":\"VIP-SAM performs visual-instance-prompt segmentation by locating the same garment instance from a flatlay support image into a query person photo. This instance-level mask is essential for precise layout control.\"},{\"question\":\"What makes CtrlVTON different from inpainting-based virtual try-on?\",\"answer\":\"CtrlVTON reframes try-on as an image-editing problem and uses segmentation masks as pixel-level control. This improves faithfulness to user-provided layouts across style, size, and placement.\"}]",1784179672,83,{"code":4,"msg":30,"data":31},"ok",{"site_id":24,"language":23,"slug":32,"title":13,"keywords":33,"description":14,"schema_data":34,"social_meta":85,"head_meta":87,"extra_data":89,"updated_unix":27},"ctrlvton-controllable-virtual-try-on-via-visual-instance-prompt-segmentation","",{"@graph":35,"@context":84},[36,53,67],{"@type":37,"itemListElement":38},"BreadcrumbList",[39,43,47,50],{"item":40,"name":41,"@type":42,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":44,"name":45,"@type":42,"position":46},"https://docshare.wps.com/document/","Document",2,{"item":48,"name":12,"@type":42,"position":49},"https://docshare.wps.com/document/technology/",3,{"item":51,"name":13,"@type":42,"position":52},"https://docshare.wps.com/document/ctrlvton-controllable-virtual-try-on-via-visual-instance-prompt-segmentation/82327/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":23,"description":14,"dateModified":61,"datePublished":61,"encodingFormat":60,"isAccessibleForFree":62,"interactionStatistic":63},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":40,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-16",true,{"@type":64,"interactionType":65,"userInteractionCount":4},"InteractionCounter",{"@type":66},"ViewAction",{"@type":68,"mainEntity":69},"FAQPage",[70,76,80],{"name":71,"@type":72,"acceptedAnswer":73},"What limitation of existing virtual try-on does CtrlVTON focus on?","Question",{"text":74,"@type":75},"Existing VTO systems provide limited control over garment size, style, and spatial placement. CtrlVTON targets controllability for how garments are worn, not just realistic transfer.","Answer",{"name":77,"@type":72,"acceptedAnswer":78},"How does VIP-SAM contribute to controllable try-on?",{"text":79,"@type":75},"VIP-SAM performs visual-instance-prompt segmentation by locating the same garment instance from a flatlay support image into a query person photo. This instance-level mask is essential for precise layout control.",{"name":81,"@type":72,"acceptedAnswer":82},"What makes CtrlVTON different from inpainting-based virtual try-on?",{"text":83,"@type":75},"CtrlVTON reframes try-on as an image-editing problem and uses segmentation masks as pixel-level control. This improves faithfulness to user-provided layouts across style, size, and placement.","https://schema.org",{"og:url":51,"og:type":86,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":88,"canonical":51},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":91},[92,96,100,104,109,112,117,122,127,130,134],{"id":20,"doc_module":4,"doc_module_name":45,"category_name":93,"show_sort_weight":94,"slug":95},"Story & Novel",90,"story-novel",{"id":46,"doc_module":4,"doc_module_name":45,"category_name":97,"show_sort_weight":98,"slug":99},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":45,"category_name":101,"show_sort_weight":102,"slug":103},"Exam",70,"exam",{"id":105,"doc_module":4,"doc_module_name":45,"category_name":106,"show_sort_weight":107,"slug":108},5,"Comic",60,"comic",{"id":11,"doc_module":4,"doc_module_name":45,"category_name":12,"show_sort_weight":110,"slug":111},50,"technology",{"id":113,"doc_module":4,"doc_module_name":45,"category_name":114,"show_sort_weight":115,"slug":116},7,"Healthcare",40,"healthcare",{"id":118,"doc_module":4,"doc_module_name":45,"category_name":119,"show_sort_weight":120,"slug":121},8,"Research & Report",30,"research-report",{"id":123,"doc_module":4,"doc_module_name":45,"category_name":124,"show_sort_weight":125,"slug":126},9,"Religion & Spirituality",20,"religion-spirituality",{"id":125,"doc_module":4,"doc_module_name":45,"category_name":128,"show_sort_weight":125,"slug":129},"World Cup","world-cup",{"id":131,"doc_module":4,"doc_module_name":45,"category_name":132,"show_sort_weight":131,"slug":133},10,"Lifestyle","lifestyle",{"id":135,"doc_module":4,"doc_module_name":45,"category_name":136,"show_sort_weight":105,"slug":137},19,"General","general"]