H3Former 基于双曲线分层对比损失的超图语义感知聚合用于细粒度视觉分类

Jul 16, 2026

Upload by Patrick

Fine-Grained Visual Classification (FGVC) is difficult because classes differ subtly while intra-class variation is large. Prior feature-selection and region-proposal methods may miss discriminative cues and introduce category-agnostic redundancy. H3Former introduces a token-to-region framework with a Semantic-Aware Aggregation Module (SAAM) that builds a weighted hypergraph among tokens using multi-scale context and hypergraph convolution for structured region representations. It further proposes Hyperbolic Hierarchical Contrastive Loss (HHCL) to enforce hierarchical semantic constraints in non-Euclidean space. Experiments on four FGVC benchmarks validate superior performance.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages15

Reading Time38 min

Introduction

Fine-Grained Visual Classification challenges

Feature-selection based FGVC methods

Region-proposal based FGVC methods

Proposed H3Former approach overview

H3Former: Hypergraph-based Semantic-Aware Aggregation via Hyperbolic Hierarchical Contrastive Loss for Fine-Grained Visual Classification Yongji Zhang†, Siqi Li†, Kuiyang Huang, Yue Gao, Senior Member, IEEE and Yu Jiang* arXiv :2511 . 10260v2 [ cs .CV] 10 Jul 2026 Abstract—Fine-Grained Visual Classification (FGVC) remains a challenging task due to subtle inter-class differences and large intra-class variations. Existing approaches typically rely on feature-selection mechanisms or region-proposal strategies to localize discriminative regions for semantic analysis. However, these methods often fail to capture discriminative cues comprehensively while introducing substantial category-agnostic redundancy. To address these limitations, we propose H3Former, a novel token-to-region framework that leverages high-order semantic relations to aggregate local fine-grained representations with structured region-level modeling. Specifically, we propose the Semantic-Aware Aggregation Module (SAAM), which exploits multi-scale contextual cues to dynamically construct a weighted hypergraph among tokens. By applying hypergraph convolution, SAAM captures high-order semantic dependencies and progressively aggregates token features into compact regionlevel representations. Furthermore, we introduce the Hyperbolic Hierarchical Contrastive Loss (HHCL), which enforces hierarchical semantic constraints in a non-Euclidean embedding space. The HHCL enhances inter-class separability and intra-class consistency while preserving the intrinsic hierarchical relationships among fine-grained categories. Comprehensive experiments conducted on four standard FGVC benchmarks validate the superiority of our H3Former framework. Code is available at [https://github.com/xiaozhangfangyang/H3Former](https://github.com/xiaozhangfangyang/H3Former). I. INTRODUCTION Fine-Grained Visual Classification (FGVC) aims to distinguish subordinate categories within a general class, e.g., distinguishing the Black-footed Albatross in Fig. 1 (a) from its close relative, the Sooty Albatross. Unlike generic object classification, FGVC heavily relies on capturing subtle visual differences typically localized in structural or textural cues. This task requires models to possess fine-grained spatial sensitivity and robust detail modeling capabilities. Additionally, This work was supported by Brain Science and Brain-like Intelligence Technology National Science and Technology Major Project (2025ZD0217300), National Natural Science Foundation of China (Nos. U25A20532 and 62501358), the Beijing Natural Science Foundation under Grant No. L242167, the Open Project Program of State Key Laboratory of Virtual Reality Technology and Systems, Beihang University (No.VRLAB2025A01), and the National Natural Science Foundation of China under Grant 62072211 . (Corresponding author: Yu Jiang.) † These authors contributed equally to this work. Yongji Zhang and Yu Jiang are with the College of Computer Science and Technology, Jilin University, Changchun 130012, China (zhangy[ongji1998@gmail.com](ongji1998@gmail.com); [jiangyu2011@jlu.edu.cn](jiangyu2011@jlu.edu.cn)). Kuiyang Huang is with the College of Software, Jilin University, Changchun 130012, China ([huangky24@mails.jlu.edu.cn](huangky24@mails.jlu.edu.cn)). Siqi Li and Yue Gao are with BNRist, THUIBCS, BLBCI, School of Software, Tsinghua University, Beijing 100084, China ([lisiqi19971013@gmail.com](lisiqi19971013@gmail.com); [kevin.gaoy@gmail.com](kevin.gaoy@gmail.com)). Fig. 1. Hyperedges (E1–E4) of hypergraph H = (V , E) generated by our H3Former. Distinct hyperedges correspond to meaningful semantic regions, e.g., tail feathers, wing, beak, and eye. The learned hypergraphs automatically highlight key discriminative parts without any part-level supervision. H3Former adaptively constructs coherent semantic regions through its hypergraph construction mechanism, bridging local token cues and global structural representation for FGVC.

FAQs

H3Former解决细粒度视觉分类中的核心难点是什么？

针对类间差异细微、类内变化大以及背景干扰导致的判别困难，H3Former通过语义感知的token-to-region聚合来更完整捕获判别线索。

SAAM（Semantic-Aware Aggregation Module）如何构建语义区域并聚合特征？

SAAM利用多尺度上下文为tokens动态构建带权超图，并通过超图卷积将token特征逐步聚合为紧凑的区域级表示。

HHCL（Hyperbolic Hierarchical Contrastive Loss）的作用是什么？

HHCL在非欧几里得嵌入空间中施加层级语义约束，增强类间可分性与类内一致性，同时保留细粒度类别之间的内在层级关系。

Jul 16, 2026

Upload by Patrick