[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"detail-sidebar-cat-0-en-105":3,"doc-seo-501560-105":59,"doc-detail-501560-en":130},{"code":4,"msg":5,"data":6},0,"success",[7,13,18,23,28,33,38,43,48,51,55],{"id":8,"doc_module":4,"doc_module_name":9,"category_name":10,"show_sort_weight":11,"slug":12},1,"Document","Story & Novel",90,"story-novel",{"id":14,"doc_module":4,"doc_module_name":9,"category_name":15,"show_sort_weight":16,"slug":17},2,"Literature",80,"literature",{"id":19,"doc_module":4,"doc_module_name":9,"category_name":20,"show_sort_weight":21,"slug":22},4,"Exam",70,"exam",{"id":24,"doc_module":4,"doc_module_name":9,"category_name":25,"show_sort_weight":26,"slug":27},5,"Comic",60,"comic",{"id":29,"doc_module":4,"doc_module_name":9,"category_name":30,"show_sort_weight":31,"slug":32},6,"Technology",50,"technology",{"id":34,"doc_module":4,"doc_module_name":9,"category_name":35,"show_sort_weight":36,"slug":37},7,"Healthcare",40,"healthcare",{"id":39,"doc_module":4,"doc_module_name":9,"category_name":40,"show_sort_weight":41,"slug":42},8,"Research & Report",30,"research-report",{"id":44,"doc_module":4,"doc_module_name":9,"category_name":45,"show_sort_weight":46,"slug":47},9,"Religion & Spirituality",20,"religion-spirituality",{"id":46,"doc_module":4,"doc_module_name":9,"category_name":49,"show_sort_weight":46,"slug":50},"World Cup","world-cup",{"id":52,"doc_module":4,"doc_module_name":9,"category_name":53,"show_sort_weight":52,"slug":54},10,"Lifestyle","lifestyle",{"id":56,"doc_module":4,"doc_module_name":9,"category_name":57,"show_sort_weight":24,"slug":58},19,"General","general",{"code":4,"msg":60,"data":61},"ok",{"site_id":62,"language":63,"slug":64,"title":65,"keywords":66,"description":67,"schema_data":68,"social_meta":123,"head_meta":125,"extra_data":127,"updated_unix":129},105,"en","token-merging-for-training-free-semantic-binding-in-text-to-image-synthesis","Token Merging for Training-Free Semantic Binding in Text-to-Image Synthesis","","Text-to-image (T2I) models often fail at semantic binding, meaning objects and their related attributes or sub-objects are not consistently linked to the prompt. Prior work relies on full model fine-tuning or requires explicit layouts from users or large language models, increasing complexity. This paper defines attribute binding and object binding, then proposes Token Merging (ToMe) that aggregates relevant tokens into composite tokens for shared cross-attention. An end token substitution complements ToMe, and two auxiliary losses refine composite tokens during early generation, validated on benchmarks including T2I-CompBench and a GPT-4o object binding benchmark.",{"@graph":69,"@context":122},[70,84,105],{"@type":71,"itemListElement":72},"BreadcrumbList",[73,77,79,82],{"item":74,"name":75,"@type":76,"position":8},"https://docshare.wps.com","Home","ListItem",{"item":78,"name":9,"@type":76,"position":14},"https://docshare.wps.com/document/",{"item":80,"name":40,"@type":76,"position":81},"https://docshare.wps.com/document/research-report/",3,{"item":83,"name":65,"@type":76,"position":19},"https://docshare.wps.com/document/token-merging-for-training-free-semantic-binding-in-text-to-image-synthesis/501560/",{"url":83,"name":65,"@type":85,"image":86,"author":91,"headline":65,"publisher":94,"fileFormat":97,"inLanguage":63,"description":67,"dateModified":98,"datePublished":99,"encodingFormat":97,"isAccessibleForFree":100,"interactionStatistic":101},"DigitalDocument",{"url":87,"@type":88,"width":89,"height":90},"https://docshare.wps.com/thumbnails/token-merging-for-training-free-semantic-binding-in-text-to-image-synthesis/501560.png","ImageObject",300,407,{"name":92,"@type":93},"EvelynWrites","Person",{"url":74,"name":95,"@type":96},"DocShare","Organization","application/pdf","2026-10-19","2026-10-03",true,{"@type":102,"interactionType":103,"userInteractionCount":24},"InteractionCounter",{"@type":104},"ViewAction",{"@type":106,"mainEntity":107},"FAQPage",[108,114,118],{"name":109,"@type":110,"acceptedAnswer":111},"What problem does the paper call semantic binding in text-to-image synthesis?","Question",{"text":112,"@type":113},"Semantic binding is the task of associating an object with its attribute (attribute binding) or linking it to related sub-objects (object binding) so the generated image matches the prompt semantics.","Answer",{"name":115,"@type":110,"acceptedAnswer":116},"How does Token Merging (ToMe) improve semantic binding?",{"text":117,"@type":113},"ToMe aggregates relevant prompt tokens into a single composite token so the object, attributes, and sub-objects share the same cross-attention map, improving semantic alignment.",{"name":119,"@type":110,"acceptedAnswer":120},"What additional strategies does the paper introduce beyond ToMe?",{"text":121,"@type":113},"It proposes end token substitution (ETS) to mitigate confusion in end tokens for long prompts and uses two auxiliary losses, entropy loss and semantic binding loss, to iteratively refine the composite token early in generation.","https://schema.org",{"og:url":83,"og:type":124,"og:title":65,"og:site_name":95,"og:description":67},"article",{"robots":126,"canonical":83},"index,follow",{"doc_id":128,"site_id":62},501560,1791518089,{"code":4,"msg":5,"data":131},{"doc_id":128,"user_id":132,"nickname":92,"user_avatar":133,"doc_module":4,"category_id":39,"category_name":40,"doc_title":65,"doc_description":67,"doc_content":134,"file_id":135,"file_url":136,"file_type":137,"file_size":138,"view_count":24,"is_deleted":4,"is_public":8,"is_downloadable":8,"audit_status":8,"page_count":139,"language":140,"language_code":63,"site_id":62,"html_lang":63,"table_of_contents":141,"faqs":142,"seo_title":143,"seo_description":67,"update_tm":144,"read_time":145},687212989558,"https://ap-avatar.wpscdn.com/davatar_a8503ba1806abce46bf441b54a3ca4cd","Token Merging for Training-Free Semantic Binding in Text-to-Image Synthesis  \nTaihang Hu 1 , Linxuan Li 1 , Joost van de Weijer3 , Hongcheng Gao4 Fahad Shahbaz Khan5 ,6 , Jian Yang 1 , Ming-Ming Cheng 1 ,2 , Kai Wang3∗ , Yaxing Wang 1 ,2 ∗  \n1VCIP, College of Computer Science, Nankai University, 2NKIARI, Shenzhen Futian  \n3 Computer Vision Center, Universitat Autònoma de Barcelona  \n4University of Chinese Academy of Sciences  \n5Mohamed bin Zayed University of AI, 6Linkoping University  \n{hutaihang00, linxuanli520, [gaohongcheng2000}@gmail.com](gaohongcheng2000}@gmail.com)[ ](gaohongcheng2000}@gmail.com){joost, [kwang}@cvc.uab.es](kwang}@cvc.uab.es), [fahad.khan@liu.se](fahad.khan@liu.se)[ ](fahad.khan@liu.se){csjyang,cmm,[yaxing}@nankai.edu.cn](yaxing}@nankai.edu.cn)  \nAbstract  \nAlthough text-to-image (T2I) models exhibit remarkable generation capabilities, they frequently fail to accurately bind semantically related objects or attributes in the input prompts; a challenge termed semantic binding. Previous approaches either involve intensive fine-tuning of the entire T2I model or require users or large language models to specify generation layouts, adding complexity. In this paper, we define semantic binding as the task of associating a given object with its attribute, termed attribute binding, or linking it to other related sub-objects, referred to as object binding. We introduce a novel method called Token Merging (ToMe), which enhances semantic binding by aggregating relevant tokens into a single composite token. This ensures that the object, its attributes and sub-objects all share the same cross-attention map. Additionally, to address potential confusion among main objects with complex textual prompts, we propose end token substitution asa complementary strategy. To further refine our approach in the initial stages of T2I generation, where layouts are determined, we incorporate two auxiliary losses, an entropy loss and a semantic binding loss, to iteratively update the composite token to improve the generation integrity. We conducted extensive experiments to validate the effectiveness of ToMe, comparing it against various existing methodson the T2I-CompBench and our proposed GPT-4o object binding benchmark. Our method is particularly effective in complex scenarios that involve multiple objects and attributes, which previous methods often fail to address. The code will be publicly available at [https://github.com/hutaihang/ToMe](https://github.com/hutaihang/ToMe).  \n1 Introduction  \nText-to-image generation has seen significant advancements with the recent introduction of diffusion models [57, 59, 62], with their capabilities of generating high-fidelity images from text prompts. Despite these achievements, aligning the generated images with the text prompts, which is referred to as semantic alignment [30, 43], remains a notable challenge. One of the most common issues observed in existing text-to-image (T2I) generation models is the lack of proper semantic binding, where a given object is not properly binding to its attributes or related objects. For example, as illustrated in Fig. 1, even a state-of-the-art T2I model such as SDXL [53] can struggle to generate content that accurately reflects the intended nuances of text prompts. To address the persistent challenges of aligning T2I diffusion models with the intricate semantics of text prompts, a variety of enhancement  \n∗ : Co-corresponding authors  \n38th Conference on Neural Information Processing Systems (NeurIPS 2024) .  \nFigure 1: Current state-of-the-art T2I models often struggle with semantic binding in generated images according to textual prompts. For example, hats and sunglasses are placed on incorrect objects. We introduce a novel method ToMe to address these challenges.  \nstrategies [35, 46, 87] are proposed, either by optimizing the latent representations [69, 82, 83], guiding the generation by layout priors [54, 71, 85] or fine-tuning the T2I models [21, 34] .","cbCairTqqBhqYNSh","https://ap.wps.com/l/cbCairTqqBhqYNSh","pdf",16055982,27,"English","# Abstract\n# Introduction","[{\"question\":\"What problem does the paper call semantic binding in text-to-image synthesis?\",\"answer\":\"Semantic binding is the task of associating an object with its attribute (attribute binding) or linking it to related sub-objects (object binding) so the generated image matches the prompt semantics.\"},{\"question\":\"How does Token Merging (ToMe) improve semantic binding?\",\"answer\":\"ToMe aggregates relevant prompt tokens into a single composite token so the object, attributes, and sub-objects share the same cross-attention map, improving semantic alignment.\"},{\"question\":\"What additional strategies does the paper introduce beyond ToMe?\",\"answer\":\"It proposes end token substitution (ETS) to mitigate confusion in end tokens for long prompts and uses two auxiliary losses, entropy loss and semantic binding loss, to iteratively refine the composite token early in generation.\"}]","Token Merging for Training-Free Semantic Binding in Text-to-Image Synthesis | PDF",1791020566,68]