[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"detail-sidebar-cat-0-en-105":3,"doc-seo-153318-105":59,"doc-detail-153318-en":131},{"code":4,"msg":5,"data":6},0,"success",[7,13,18,23,28,33,38,43,48,51,55],{"id":8,"doc_module":4,"doc_module_name":9,"category_name":10,"show_sort_weight":11,"slug":12},1,"Document","Story & Novel",90,"story-novel",{"id":14,"doc_module":4,"doc_module_name":9,"category_name":15,"show_sort_weight":16,"slug":17},2,"Literature",80,"literature",{"id":19,"doc_module":4,"doc_module_name":9,"category_name":20,"show_sort_weight":21,"slug":22},4,"Exam",70,"exam",{"id":24,"doc_module":4,"doc_module_name":9,"category_name":25,"show_sort_weight":26,"slug":27},5,"Comic",60,"comic",{"id":29,"doc_module":4,"doc_module_name":9,"category_name":30,"show_sort_weight":31,"slug":32},6,"Technology",50,"technology",{"id":34,"doc_module":4,"doc_module_name":9,"category_name":35,"show_sort_weight":36,"slug":37},7,"Healthcare",40,"healthcare",{"id":39,"doc_module":4,"doc_module_name":9,"category_name":40,"show_sort_weight":41,"slug":42},8,"Research & Report",30,"research-report",{"id":44,"doc_module":4,"doc_module_name":9,"category_name":45,"show_sort_weight":46,"slug":47},9,"Religion & Spirituality",20,"religion-spirituality",{"id":46,"doc_module":4,"doc_module_name":9,"category_name":49,"show_sort_weight":46,"slug":50},"World Cup","world-cup",{"id":52,"doc_module":4,"doc_module_name":9,"category_name":53,"show_sort_weight":52,"slug":54},10,"Lifestyle","lifestyle",{"id":56,"doc_module":4,"doc_module_name":9,"category_name":57,"show_sort_weight":24,"slug":58},19,"General","general",{"code":4,"msg":60,"data":61},"ok",{"site_id":62,"language":63,"slug":64,"title":65,"keywords":66,"description":67,"schema_data":68,"social_meta":124,"head_meta":126,"extra_data":128,"updated_unix":130},105,"en","devil-is-in-the-detail-towards-injecting-fine-details-of-image-prompt-in-image-generation","Devil is in the Detail - Towards Injecting Fine Details of Image Prompt in Image Generation","","Large-scale text-to-image diffusion models can generate diverse, high-quality images from prompts, yet detailed user intent often fails to transfer—especially textures and other fine-grained attributes. Image prompts address this by conditioning generation on user-provided images, typically by modifying self-attention through key/value integration. This paper identifies two recurring flaws that prevent faithful image-prompt reflection, then introduces a conflict-free guidance strategy and a stratified attention mechanism to balance realism with prompt fidelity, validated across multiple tasks.",{"@graph":69,"@context":123},[70,84,106],{"@type":71,"itemListElement":72},"BreadcrumbList",[73,77,79,82],{"item":74,"name":75,"@type":76,"position":8},"https://docshare.wps.com","Home","ListItem",{"item":78,"name":9,"@type":76,"position":14},"https://docshare.wps.com/document/",{"item":80,"name":40,"@type":76,"position":81},"https://docshare.wps.com/document/research-report/",3,{"item":83,"name":65,"@type":76,"position":19},"https://docshare.wps.com/document/devil-is-in-the-detail-towards-injecting-fine-details-of-image-prompt-in-image-generation/153318/",{"url":83,"name":65,"@type":85,"image":86,"author":91,"headline":65,"publisher":94,"fileFormat":97,"inLanguage":63,"description":67,"dateModified":98,"datePublished":99,"encodingFormat":97,"isAccessibleForFree":100,"interactionStatistic":101},"DigitalDocument",{"url":87,"@type":88,"width":89,"height":90},"https://docshare.wps.com/thumbnails/devil-is-in-the-detail-towards-injecting-fine-details-of-image-prompt-in-image-generation/153318.png","ImageObject",300,407,{"name":92,"@type":93},"Felix Montgomery","Person",{"url":74,"name":95,"@type":96},"DocShare","Organization","application/pdf","2026-11-07","2026-08-27",true,{"@type":102,"interactionType":103,"userInteractionCount":105},"InteractionCounter",{"@type":104},"ViewAction",11,{"@type":107,"mainEntity":108},"FAQPage",[109,115,119],{"name":110,"@type":111,"acceptedAnswer":112},"为什么仅靠文本提示难以捕捉细节？","Question",{"text":113,"@type":114},"文本提示难以描述纹理等细粒度属性，导致用户意图无法完整反映到生成结果中。图像提示能提供这些精细信息。","Answer",{"name":116,"@type":111,"acceptedAnswer":117},"现有自注意力修改方法存在哪些主要问题？",{"text":118,"@type":114},"文中指出两类常见问题会阻碍扩散模型对图像提示的反映，尤其是在与分类器无关引导相关的机制上产生冲突信号。",{"name":120,"@type":111,"acceptedAnswer":121},"本文提出的关键方法是什么？",{"text":122,"@type":114},"通过冲突无关引导（只将图像提示用于期望条件）消除负向冲突，并提出分层注意力（Stratified Attention）联合使用来自两路的 keys/values，以在真实感与提示一致性之间取得平衡。","https://schema.org",{"og:url":83,"og:type":125,"og:title":65,"og:site_name":95,"og:description":67},"article",{"robots":127,"canonical":83},"index,follow",{"doc_id":129,"site_id":62},153318,1787873821,{"code":4,"msg":5,"data":132},{"doc_id":129,"user_id":133,"nickname":92,"user_avatar":134,"doc_module":4,"category_id":39,"category_name":40,"doc_title":65,"doc_description":67,"doc_content":135,"file_id":136,"file_url":137,"file_type":138,"file_size":139,"view_count":105,"is_deleted":4,"is_public":8,"is_downloadable":8,"audit_status":8,"page_count":44,"language":140,"language_code":63,"site_id":62,"html_lang":63,"table_of_contents":141,"faqs":142,"seo_title":143,"seo_description":67,"update_tm":130,"read_time":144,"preview_page_count":4},549768064778,"https://ap-avatar.wpscdn.com/davatar_6f874abed73319feea01a86fa6f0fab8","This CVPR paper is the Open Access version, provided by the Computer Vision Foundation.  \nExcept for this watermark, it is identical to the accepted version; the final published version of the proceedings is available on IEEE Xplore.  \nDevil is in the Detail: Towards Injecting Fine Details of Image Prompt in Image Generation via Conflict-free Guidance and Stratified Attention  \nKyungmin Jo Jooyeol Yun Jaegul Choo  \nKorea Advanced Institute of Science and Technology (KAIST)  \nDaejeon, Korea  \n{bttkm, blizzard072, [jchoo](jchoo}@kaist.ac.kr)[}](jchoo}@kaist.ac.kr)[@kaist.ac.kr](jchoo}@kaist.ac.kr)  \nFigure 1 . Our method faithfully reflects the details of image prompts (e.g., eye makeup) for diverse image generation across various tasks (Tasks A-C) in a training-free manner. In contrast, existing methods often fail to incorporate fine details of the image prompt. Even with training, IP-Adapter [19] and SSR-Encoder [21] struggle to properly represent the color and texture of the image prompt. RIVAL [22] and StyleAligned [4] also fail to generate high-fidelity images that faithfully reflect the image or text prompts due to a trade-off between the realism of the generated results and alignment with the image prompt. Extra results are in the Supplementary (Suppl.) .  \nAbstract  \nWhile large-scale text-to-image diffusion models enable the generation of high-quality, diverse images from text prompts, these prompts struggle to capture intricate details, such as textures, preventing the user intent from being reflected. This limitation has led to efforts to generate images conditioned on user-provided images, referred to as image prompts. Recent work modifies the self-attention mechanism to impose image conditions in generated images by replacing or concatenating the keys and values from the image prompt. This enables the self-attention layer to work like a cross-attention layer, generally used to incorporate text prompts. In this paper, we identify two common issues in existing methods of modifying self-attention that hinder diffusion models from reflecting the image prompt. By addressing these issues, we propose a novel method that generates images that properly reflect the details of image prompts.  \nFirst, existing approaches often neglect the importance of image prompts in classifier-free guidance, which directs the model towards the intended conditions and away from those undesirable. Specifically, current methods use image prompts as both desired and undesired conditions, causing conflicting signals. To resolve this, we propose conflict-free guidance by using image prompts only as desired conditions, ensuring that the generated image faithfully reflectsthe image prompt. In addition, we observe that the two most common self-attention modifications involve a tradeoff between the realism of the generated image and alignment with the image prompt, achieved by selectively using keys and values from both images. Specifically, selecting more keys and values from the image prompt improves alignment, while selecting more from the generated image enhances realism. To balance both, we propose an alternative self-attention modification method, Stratified Attention, which jointly uses keys and values from both images  \nrather than selecting between them. Through extensive experiments across three distinct image generation tasks, we demonstrate that the proposed method outperforms existing image-prompting models in faithfully reflecting the image prompt.  \n1. Introduction  \nThe emergence of large-scale text-to-image diffusion models [11, 13] has enabled users to generate diverse images from text. However, describing complex scenes using only text prompts can be complicated, leading to ongoing efforts to incorporate user-provided image conditions to better reflect user intent [14, 17, 19, 21] . As part of these efforts, modifying the self-attention mechanism in pre-trained textto-image diffusion models has recently attracted interest in areas such as image editing ","cbCaitBXhd7ZvPdq","https://ap.wps.com/l/cbCaitBXhd7ZvPdq","pdf",3318625,"English","# Abstract\n# Introduction\n## Motivation: image prompts for fine details\n## Self-attention modification for conditioning\n## Classifier-free guidance conflicts\n# Method overview\n## Conflict-free guidance\n## Stratified attention","[{\"question\":\"为什么仅靠文本提示难以捕捉细节？\",\"answer\":\"文本提示难以描述纹理等细粒度属性，导致用户意图无法完整反映到生成结果中。图像提示能提供这些精细信息。\"},{\"question\":\"现有自注意力修改方法存在哪些主要问题？\",\"answer\":\"文中指出两类常见问题会阻碍扩散模型对图像提示的反映，尤其是在与分类器无关引导相关的机制上产生冲突信号。\"},{\"question\":\"本文提出的关键方法是什么？\",\"answer\":\"通过冲突无关引导（只将图像提示用于期望条件）消除负向冲突，并提出分层注意力（Stratified Attention）联合使用来自两路的 keys/values，以在真实感与提示一致性之间取得平衡。\"}]","Devil is in the Detail - Towards Injecting Fine Details of Image Prompt in Image Generation | PDF",23]