[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-82523-en":3,"doc-seo-82523-105":29,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":13,"seo_description":14,"update_tm":27,"read_time":28},82523,687197100911,"Himbo","https://ap-avatar.wpscdn.com/avatar/a000239b6f1da00475?x-image-process=image/resize,m_fixed,w_180,h_180&k=1782698725881665579",8,"Research & Report","HyFL-CLIP Hyperbolic Fine-Tuning of CLIP for Robust Long-Context Understanding","HyFL-CLIP introduces a hyperbolic fine-tuning framework for CLIP-style vision-language alignment under long-context descriptions. CLIP struggles with text over 77 tokens because absolute positional encoding and short-caption pretraining limit robustness to sentence reordering, summarization, and omission. The method addresses this by distilling Euclidean CLIP alignment into hyperbolic space using cross-manifold similarity distillation, enabling hierarchical and entailment modeling via hyperbolic entailment and Einstein midpoint aggregation. Experiments across long-context retrieval and perturbation benchmarks show up to 19.5% improvement, and the approach integrates into SDXL.","arXiv :2607 .00428v 1 [ cs .CV] 1 Jul 2026  \nHyFL-CLIP: Hyperbolic Fine-Tuning of CLIP for Robust Long-Context Understanding  \nJi Ha Jang 1 ,∗, Hayeon Kim 1 ,∗, Chulwon Lee2,  \nJunghun James Kim2, Se Young Chun 1 ,2 ,3 ,†  \n1 Dept. of Electrical and Computer Engineering, 2 IPAI, 3 INMC & AIIS,  \nSeoul National University, Republic of Korea {jeeit17, khy5630, chul0e, jonghean12, [sychun}@snu.ac.kr](sychun}@snu.ac.kr)  \n[Abstract.](Abstract. CLIP)[ CLIP](Abstract. CLIP) ([Contrastive Language-Image Pre-training](Contrastive Language-Image Pre-training)) [has be](has be)come a de facto paradigm for image-text alignment, but it struggles with long-context descriptions (> 77 tokens) due to absolute positional encoding and pretraining on short captions. In long contexts, sentences are often reordered, summarized, or partially omitted. Although prior works extend CLIP with longer positional encodings, they often suffer from degraded image-text alignment under such text perturbations. We attribute this limitation to the Euclidean contrastive objective, which enforces strict one-to-one matching and lacks explicit mechanisms for modeling hierarchical relationships between global context and its constituent elements. To address this issue, we propose HyFL-CLIP, a hyperbolic fine-tuning framework that distills the well-established textimage alignment learned in Euclidean CLIP into hyperbolic space viacross-manifold similarity distillation, leveraging its geometry to capture hierarchical and entailment relations. Our method models hierarchical semantics by linking summarized token-wise features, long-context descriptions, constituent short textual components, and images, capturing part–whole relationships via hyperbolic entailment with Einstein midpoint aggregation. Experiments on diverse benchmarks, including longcontext cross-modal retrieval, cross-modal retrieval with caption perturbations, intra-modality retrieval, and short-text cross-modal retrieval, show that HyFL-CLIP achieves more robust long-context understanding. In particular, it yields up to 19 .5% improvement in long-text crossmodal retrieval under textual perturbations over the best prior method.  \nWe also show HyFL-CLIP can be seamlessly integrated into other model frameworks by applying it to Stable Diffusion XL (SDXL) . The project page is available at [https://janeyeon.github.io/hyflclip](https://janeyeon.github.io/hyflclip).  \nKeywords: hyperbolic representation learning · long-context vision-language alignment · cross-manifold distillation  \n1 Introduction  \nVision-language contrastive pre-training has laid the foundation for a wide range of vision-language learning tasks. Models such as CLIP [59], ALIGN [29] have  \n* Authors contributed equally. † Corresponding author.  \n2 JH Jang, H Kim et al.  \nRobust Long-Context Retrieval under Caption Perturbation  \nLong text query  \nIn the foreground, there's a pedestrian crossing the street marked black car in the crosswalk.  \nThe traffic light for pedestrians is visible, displaying a red hand signal.  \nTo the left, another pedestrian appears to be wearing a white and red outfit crossing the street.  \nBuildings with varying facades line the street, and clear blue skies with scattered clouds are above.  \nRandom subsampling  \nTo the left, another pedestrian appears to be wearing a white and red outfit crossing the street.  \nThe traffic light for pedestrians is visible, displaying a red hand signal.  \nOrder shuffling  \nThe traffic light for pedestrians is visible, displaying a red hand signal.  \nTo the left, another pedestrian appears to be wearing a white and red outfit crossing the street.  \nIn the foreground, there's a pedestrian crossing the street marked black car in the crosswalk.  \nBuildings with varying facades line the street, and clear blue skies with scattered clouds are above.  \nWord dropout  \nIn the foreground, there's a  the street marked black car in the crosswalk.  \nThe traffic light for is visible, displaying a re","cbCaiesvI9Fr9WSz","https://ap.wps.com/l/cbCaiesvI9Fr9WSz","pdf",29720468,1,48,"English","en",105,"# Abstract\n# Introduction\n## Robust Long-Context Retrieval under Caption Perturbation","[{\"question\":\"Why does CLIP perform poorly on long-context image-text alignment?\",\"answer\":\"CLIP is primarily trained on short captions and relies on absolute positional encoding, which becomes unstable for text longer than 77 tokens. Long-context descriptions often get reordered, summarized, or partially omitted, breaking strict one-to-one alignment.\"},{\"question\":\"What is HyFL-CLIP and how does it improve robustness?\",\"answer\":\"HyFL-CLIP distills Euclidean CLIP alignment into hyperbolic space via cross-manifold similarity distillation. In hyperbolic geometry, the model captures hierarchical and entailment relations and part–whole structure using hyperbolic entailment with Einstein midpoint aggregation.\"},{\"question\":\"How effective is HyFL-CLIP and can it be used with other frameworks?\",\"answer\":\"Experiments on multiple long-context and perturbation benchmarks show improved robustness, including up to 19.5% gains in long-text cross-modal retrieval under textual perturbations. It can also be applied to Stable Diffusion XL (SDXL) for seamless integration.\"}]",1784181196,121,{"code":4,"msg":30,"data":31},"ok",{"site_id":24,"language":23,"slug":32,"title":13,"keywords":33,"description":14,"schema_data":34,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":27},"hyfl-clip-hyperbolic-fine-tuning-of-clip-for-robust-long-context-understanding","",{"@graph":35,"@context":85},[36,53,68],{"@type":37,"itemListElement":38},"BreadcrumbList",[39,43,47,50],{"item":40,"name":41,"@type":42,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":44,"name":45,"@type":42,"position":46},"https://docshare.wps.com/document/","Document",2,{"item":48,"name":12,"@type":42,"position":49},"https://docshare.wps.com/document/research-report/",3,{"item":51,"name":13,"@type":42,"position":52},"https://docshare.wps.com/document/hyfl-clip-hyperbolic-fine-tuning-of-clip-for-robust-long-context-understanding/82523/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":23,"description":14,"dateModified":61,"datePublished":62,"encodingFormat":60,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":40,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-17","2026-07-16",true,{"@type":65,"interactionType":66,"userInteractionCount":20},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"Why does CLIP perform poorly on long-context image-text alignment?","Question",{"text":75,"@type":76},"CLIP is primarily trained on short captions and relies on absolute positional encoding, which becomes unstable for text longer than 77 tokens. Long-context descriptions often get reordered, summarized, or partially omitted, breaking strict one-to-one alignment.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"What is HyFL-CLIP and how does it improve robustness?",{"text":80,"@type":76},"HyFL-CLIP distills Euclidean CLIP alignment into hyperbolic space via cross-manifold similarity distillation. In hyperbolic geometry, the model captures hierarchical and entailment relations and part–whole structure using hyperbolic entailment with Einstein midpoint aggregation.",{"name":82,"@type":73,"acceptedAnswer":83},"How effective is HyFL-CLIP and can it be used with other frameworks?",{"text":84,"@type":76},"Experiments on multiple long-context and perturbation benchmarks show improved robustness, including up to 19.5% gains in long-text cross-modal retrieval under textual perturbations. It can also be applied to Stable Diffusion XL (SDXL) for seamless integration.","https://schema.org",{"og:url":51,"og:type":87,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":89,"canonical":51},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":20,"doc_module":4,"doc_module_name":45,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":46,"doc_module":4,"doc_module_name":45,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":45,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":45,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":45,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":45,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":45,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":45,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":45,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":45,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":45,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]