Beyond Looking Up, Try Looking Around Harmonizing Global Structure and Local Consistency in Optimal Transport for Short Text Clustering

Jul 16, 2026

Upload by Miles

Pseudo-labeling based on Optimal Transport (OT) enhances short text clustering, yet existing OT approaches under-model semantic consistency among samples, which can give different pseudo-labels to semantically similar items and degrade cluster quality. The proposed consistency-aware adaptive optimal transport (CAOT) introduces instance-level attention to capture semantic relationships and embeds this consistency into the OT formulation. Solving CAOT yields pseudo-labels aligned with sample-to-sample semantics and sample-to-cluster global structure, then guides accurate clustering. Extensive experiments show state-of-the-art performance.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages19

Reading Time48 min

Abstract

Introduction

Motivation and background

Optimal transport-based pseudo-labeling

Semantic consistency limitation

Proposed CAOT approach

Beyond Looking Up, Try Looking Around: Harmonizing Global Structure and Local Consistency in Optimal Transport for Short Text Clustering Zhihao Yao 1 Yuxuan Gu 2 Jixuan Yin 1 Bo Li 1 arXiv :2607 . 10548v1 [ stat .ML] 12 Jul 2026 Abstract Pseudo-labeling based on Optimal Transport (OT) has become an effective mechanism for enhancing short text clustering. Existing OT methods are short in modeling semantic consistencies between samples, which may assign different pseudo-labels to semantically similar samples. These erroneous pseudo-labels can cause the model to produce inferior clusters. This paper proposes a novel short text clustering framework, which remedies the neglect of semantic consistency in existing OT methods, generating reliable pseudo-labels to facilitate clustering. Specifically, the proposed approach first designs an instancelevel attention mechanism to capture semantic relationships between samples, which are then integrated into the OT formulation to endow the transport process with neighborhood semantic awareness. By solving the proposed OT formulation, reliable pseudo-labels are obtained that simultaneously account for sample-to-sample semantic consistency and sample-to-cluster global structure information. These pseudo-labels are then used as supervisory signals to guide the model to achieve accurate clustering. Extensive experiments demonstrate that the proposed approach outperforms state-of-the-art methods. The code is available at: [https://github.com/YZH0905/CAOT](https://github.com/YZH0905/CAOT)STC. 1. Introduction In the era of massive digital communication, short texts such as messages, queries, and comments are generated on an unprecedented scale. Effectively clustering and organizing these short texts is crucial for numerous applications 1Harbin Engineering University, China. 2Harbin Institute of Technology, China. Correspondence to: Bo Li <boli@hrbeu.ed[u.cn](u.cn) > . Proceedings of the 43 rd International Conference on Machine Learning, Seoul, South Korea. PMLR 306, 2026 . Copyright 2026 by the author(s) . (a)Existing OT Semantic consistency (b)CAOT : Samples : Pseudo-labels : Decision boundary : Misidentified samples Figure 1. Schematic illustration of the motivation. Existing methods rely solely on the transport cost from samples to candidate pseudolabels (hexagons), where the cost is visualized as the distances from triangles to hexagons, causing semantically similar neighbors to be assigned different pseudo-labels (red boxes) . The proposed CAOT addresses this issue by incorporating sample-to-sample semantic consistency, thereby correcting these misassignments. (Ma & Zheng, 2025) . This urgency is further amplified by the prevalence of Large Language Models (LLMs), since LLM-driven applications, such as chatbots (Kang & Ki, 2025) and virtual assistants (Kweon et al., 2025), have further fueled the proliferation of short texts. Existing methods typically adopt an EM-like optimization framework to achieve clustering (Zheng et al., 2023 ; Wu et al., 2025), which alternates between estimating pseudo-labels (E-step) and updating model parameters based on the generated pseudo-labels (M-step) . Under this framework, early methods often adopt a greedy strategy to estimate pseudo-labels directly from model predictions (Sohn et al., 2020) . However, they ignore both global and local information of the data distribution (Qin et al., 2026), resulting in unreliable pseudo-labels that may misguide model optimization in the subsequent M-step (Yin et al., 2025 ; Cui et al., 2025) . Instead of assigning pseudo-labels greedily, recent studies introduce Optimal Transport (OT) to estimate pseudo-labels by minimizing the overall cost of transporting the sample distribution to the cluster distribution (Zheng et al., 2023 ; Cui et al., 2025), integrating sample-cluster relationships from a global perspective (Zhang et al., 2024 ; Zheng et al., 2023), i.e., the sample-to-cluster global stru

FAQs

What problem does existing OT-based pseudo-labeling have for short text clustering?

It often fails to model semantic consistency between semantically similar samples, so ambiguous transport costs can lead to different or incorrect pseudo-label assignments. Noisy pseudo-labels then misguide subsequent clustering updates.

How does CAOT improve pseudo-label estimation?

CAOT integrates neighborhood semantic awareness by using an instance-level attention mechanism to capture sample-to-sample semantic relationships. This semantic consistency constraint is incorporated into the OT formulation to correct ambiguous assignments.

How are the generated pseudo-labels used in CAOT?

The pseudo-labels produced by solving the proposed OT formulation are used as supervisory signals. They guide EM iterations so the model learns accurate clustering assignments that reflect both local semantic consistency and global cluster structure.

Jul 16, 2026

Upload by Miles