Uncertainty-Guided Compositional Hyperbolic Alignment for Part-to-Whole Semantic Representativeness in Vision-Language Models

Jul 16, 2026

Vision-Language Models achieve strong image–text performance, yet Euclidean embeddings often struggle to represent hierarchical part-to-whole (parent–child) relations, especially in complex multi-object compositions. Hyperbolic VLMs better preserve hierarchical structure through entailment, but they do not differentiate how representative each part is to the whole scene. UNCHA introduces uncertainty-guided compositional hyperbolic alignment, assigning lower uncertainty to more representative parts and higher uncertainty to less representative ones, then calibrating this representativeness within contrastive and entailment objectives. Results improve part–whole ordering and understanding of complex scenes, reaching state-of-the-art zero-shot and retrieval performance.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages25

Reading Time1 h 3 min

Abstract

Introduction

Background on VLM Hierarchies and Limitations

Hyperbolic Geometry for Hierarchical Representation

Problem: Unequal Part Representativeness

UNCHA Approach and Key Losses

Uncertainty-guided Compositional Alignment with Part-to-Whole Semantic Representativeness in Hyperbolic Vision-Language Models Hayeon Kim 1 ,∗ Ji Ha Jang 1 ,∗ Junghun James Kim2 Se Young Chun 1 ,2 ,† 1 Dept. of Electrical and Computer Engineering, 2 INMC & IPAI Seoul National University, Republic of Korea {khy5630, jeeit17, jonghean12, [sychun](sychun}@snu.ac.kr)[}](sychun}@snu.ac.kr)[@snu.ac.kr](sychun}@snu.ac.kr) ∗Authors contributed equally. †Corresponding author. arXiv :2603 .22042v3 [ cs .CV] 13 Jul 2026 Abstract While Vision-Language Models (VLMs) have achieved remarkable performance, their Euclidean embeddings remain limited in capturing hierarchical relationships such as partto-whole or parent-child structures, and often face challenges in multi-object compositional scenarios. Hyperbolic VLMs mitigate this issue by better preserving hierarchical structures and modeling part-whole relations (i.e., whole scene and its part images) through entailment. However, existing approaches do not model that each part has a different level of semantic representativeness to the whole. We propose UNcertainty-guided Compositional Hyperbolic Alignment (UNCHA) for enhancing hyperbolic VLMs. UNCHA models part-to-whole semantic representativeness with hyperbolic uncertainty, by assigning lower uncertainty to more representative parts and higher uncertainty to less representative ones for the whole scene. This representativeness is then incorporated into the contrastive objective with uncertainty-guided weights. Finally, the uncertainty is further calibrated with an entailment loss regularized by entropy-based term. With the proposed losses, UNCHA learns hyperbolic embeddings with more accurate part-whole ordering, capturing the underlying compositional structure in an image and improving its understanding of complex multi-object scenes. UNCHA achieves state-of-the-art performance on zero-shot classification, retrieval, and multi-label classification benchmarks. Our code and models are available at: [https:](https:)// [github.com/jeeit17/UNCHA.git](github.com/jeeit17/UNCHA.git). 1. Introduction Understanding hierarchical structures is essential for capturing complex compositional information efficiently. As well established in cognitive science, human perception relies on part-whole hierarchies [25, 26], enabling general- ization by interpreting new inputs through known relational structures [26, 30, 67] . Such hierarchical representations also improve information compression, classification, and inference efficiency [8, 16, 48, 69] . Vision-Language Models (VLMs) such as CLIP [53], ALIGN [31], and ALBEF [39] have demonstrated remarkable performance in image-text matching and shown strong versatility across various downstream tasks. However, owing to their reliance on Euclidean geometry, these models often face distortion of hierarchical structure and dimensionality tradeoffs in capturing hierarchical or complex relational structures [21, 48, 65] . Moreover, CLIP has been reported to exhibit bias and difficulty with compositional relations in complex multi-object scenes [1], which is partly due to the lack of modeling part-whole relations. Hyperbolic space, characterized by constant negative curvature and exponential volume growth, provides an efficient geometric foundation for embedding hierarchical and fine-grained relational structures. Motivated by these properties, recent studies [6, 10, 11, 35, 49, 54, 58] have explored hyperbolic geometry in vision-language learning. MERU [10] extended contrastive vision-language learning into hyperbolic space by explicitly modeling entailment relations between text and image pairs. ATMG [54] later demonstrated that proximity-based contrastive losses can hinder hierarchical structure learning and proposed an angle-based alternative. HyCoCLIP [49] extended entailment modeling beyond inter-modal image-text relations by including intra-modal part-whole relationships. Although h

FAQs

How does UNCHA improve hyperbolic embeddings for understanding complex scenes?

UNCHA calibrates uncertainty with losses that incorporate uncertainty-guided weighting in the contrastive objective and regularize entailment using an entropy-based term. This yields more accurate part–whole ordering and better comprehension of multi-object compositional structure.

Jul 16, 2026

Upload by Olivia Brown