[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-84558-en":3,"doc-seo-84558-105":29,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":13,"seo_description":14,"update_tm":27,"read_time":28},84558,34359740700684,"Finn","https://ap-avatar.wpscdn.com/avatar/1f400023980c374ae676?_k=1777273430885731487",8,"Research & Report","Learning to Compose Revisiting Proxy Task Design for Zero-Shot Composed Image Retrieval","Composed Image Retrieval (CIR) retrieves a target image using a reference image plus a textual modification, while Zero-Shot CIR (ZS-CIR) avoids costly triplet supervision by training proxy tasks on image–text pairs. Existing proxy designs largely enhance representations to fit predefined, non-learnable composition rules, leaving the composition function itself unlearned and limiting fine-grained semantic edits. FoCo introduces a learnable two-stage composition: focus on modification-relevant visual content and then complete target semantics via context-conditioned completion, trained jointly with cross-instance contrastive learning. Experiments on four ZS-CIR benchmarks show state-of-the-art results and improved generalization.","arXiv :2607 .00374v 1 [ cs .CV] 1 Jul 2026  \nLearning to Compose: Revisiting Proxy Task Design for Zero-Shot Composed Image Retrieval  \nJingjing Zhang, Lei Zhang⋆ , Zheren Fu, and Zhendong Mao  \nUniversity of Science and Technology of China, Hefei, China  \n[zjj1029@mail.ustc.edu.cn](zjj1029@mail.ustc.edu.cn) , {leizh23, fzr, [zdmao}@ustc.edu.cn](zdmao}@ustc.edu.cn)  \n[Abstract.](Abstract. Composed Image Retrieval)[ Composed Image Retrieval](Abstract. Composed Image Retrieval) ([CIR](CIR)) [retrieves a target image](retrieves a target image)[ ](retrieves a target image)[from a reference image and a textual modification. While supervised](from a reference image and a textual modification. While supervised)[ ](from a reference image and a textual modification. While supervised)CIR relies on costly triplets, Zero-Shot CIR (ZS-CIR) alleviates this reliance through proxy tasks trained on image–text pairs. However, existing proxy tasks primarily enhance visual and textual representations to accommodate a predefined composition mechanism such as pseudoword injection into a frozen text encoder or linear feature arithmetic.  \nAs a result, the composition function itself remains unlearned, limiting the model’s ability to express diverse and fine-grained semantic modifications. To address this, we propose FoCo, which models composition as two coordinated stages: focusing on modification-relevant visual content, and then completing the target semantics. We realize these through two proxy tasks: text-anchored visual aggregation to selectively gather visual content guided by localized textual semantics, and contextconditioned semantic completion to transform these aggregated visuals with the remaining scene context into a coherent composed representation. The tasks are trained jointly with a cross-instance contrastive objective, encouraging semantic diversity and discouraging shortcut composition strategies. Extensive experiments on four ZS-CIR benchmarks show FoCo’s state-of-the-art performance and improved generalization.  \nKeywords: Zero-Shot Composed Image Retrieval · Learnable Composition · Proxy Tasks  \n1 Introduction  \nComposed Image Retrieval (CIR) [4, 32] aims to retrieve a target image using a reference image and a textual modification, enabling users to specify how the reference should be changed rather than describing the target from scratch. Recently, Zero-Shot CIR (ZS-CIR) [23] has emerged as an important paradigm that eliminates the need for triplet annotations (reference image, textual modification, target image) during training, making the task both practical and challenging.  \nMost existing ZS-CIR approaches design proxy tasks on pretrained vision– language models to avoid triplet supervision. However, these methods mainly  \n⋆ Corresponding author.  \n2 J. Zhang et al.  \n\n|  |  |\n| --- | --- |\n| (b) Existing ZS-CIR Methods\u003Cbr>image\u003Cbr>text\u003Cbr>\u003Cbr> Proxy  Training\u003Cbr>\u003Cbr>composed feature\u003Cbr> |  |\n\nFig. 1: (a) The CIR task retrieves a target image from a reference image and a modification text. (b) Existing Zero-Shot CIR uses proxy tasks to avoid supervised triplets, but relies on fixed, non-learnable composition rules, causing the proxy learner to mainly adapt features to these rules. (c) Our learnable composition framework is inspired by human-like selective composition: first focus on the visual semantics relevant to the modification, and then integrate them with the text to form the target representation.  \nfocus on learning feature representations that conform to a predefined, nonlearnable composition mechanism, leaving the composition process itself unmodeled. Two prevalent paradigms illustrate this limitation. Some methods project image features into pseudo-words and compose them with a frozen text encoder [1, 8, 16, 17, 27] . Such encoders are pretrained to maximize global image–text semantic alignment, rendering them ill-suited to capturing the localized semantic changes central to CIR. Others rely on simple vector ar","cbCaigzijxeCPF0R","https://ap.wps.com/l/cbCaigzijxeCPF0R","pdf",3685816,1,17,"English","en",105,"# Introduction\n# Focus-then-Complete (FoCo) Framework\n## Two-Stage Learnable Composition\n## Proxy Tasks: Text-Anchored Visual Aggregation and Context-Conditioned Semantic Completion","[{\"question\":\"What problem does FoCo address in existing zero-shot composed image retrieval methods?\",\"answer\":\"Existing ZS-CIR proxy tasks mainly adapt representations to fixed, non-learnable composition rules, so the composition function remains unlearned, restricting diverse and fine-grained semantic modifications.\"},{\"question\":\"How does FoCo model composition in its proposed framework?\",\"answer\":\"FoCo treats composition as two coordinated stages: focusing on modification-relevant visual content and then completing the target semantics using remaining scene context.\"},{\"question\":\"What are the two proxy tasks used to enable FoCo’s two-stage composition?\",\"answer\":\"FoCo uses text-anchored visual aggregation to selectively gather localized visual–semantic evidence guided by local captions, and context-conditioned semantic completion to transform these localized representations with a contextual caption into a coherent composed representation.\"}]",1784196725,43,{"code":4,"msg":30,"data":31},"ok",{"site_id":24,"language":23,"slug":32,"title":13,"keywords":33,"description":14,"schema_data":34,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":27},"learning-to-compose-revisiting-proxy-task-design-for-zero-shot-composed-image-retrieval","",{"@graph":35,"@context":85},[36,53,68],{"@type":37,"itemListElement":38},"BreadcrumbList",[39,43,47,50],{"item":40,"name":41,"@type":42,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":44,"name":45,"@type":42,"position":46},"https://docshare.wps.com/document/","Document",2,{"item":48,"name":12,"@type":42,"position":49},"https://docshare.wps.com/document/research-report/",3,{"item":51,"name":13,"@type":42,"position":52},"https://docshare.wps.com/document/learning-to-compose-revisiting-proxy-task-design-for-zero-shot-composed-image-retrieval/84558/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":23,"description":14,"dateModified":61,"datePublished":62,"encodingFormat":60,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":40,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-17","2026-07-16",true,{"@type":65,"interactionType":66,"userInteractionCount":20},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"What problem does FoCo address in existing zero-shot composed image retrieval methods?","Question",{"text":75,"@type":76},"Existing ZS-CIR proxy tasks mainly adapt representations to fixed, non-learnable composition rules, so the composition function remains unlearned, restricting diverse and fine-grained semantic modifications.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"How does FoCo model composition in its proposed framework?",{"text":80,"@type":76},"FoCo treats composition as two coordinated stages: focusing on modification-relevant visual content and then completing the target semantics using remaining scene context.",{"name":82,"@type":73,"acceptedAnswer":83},"What are the two proxy tasks used to enable FoCo’s two-stage composition?",{"text":84,"@type":76},"FoCo uses text-anchored visual aggregation to selectively gather localized visual–semantic evidence guided by local captions, and context-conditioned semantic completion to transform these localized representations with a contextual caption into a coherent composed representation.","https://schema.org",{"og:url":51,"og:type":87,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":89,"canonical":51},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":20,"doc_module":4,"doc_module_name":45,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":46,"doc_module":4,"doc_module_name":45,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":45,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":45,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":45,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":45,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":45,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":45,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":45,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":45,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":45,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]