Topic Model Based on Co-occurrence Word Networks for Unbalanced Short Text Datasets

Jul 16, 2026

Upload by Jiven

A straightforward solution is presented for detecting scarce topics in unbalanced short-text datasets. The proposed method, CWUTM, mitigates sparse and unbalanced topic behavior by reducing the influence of incidental word co-occurrence. CWUTM models topic distribution for each word through co-occurrence word networks, adjusts node activity computation, and partially normalizes representations across scarce and abundant topics to improve low-frequency topic sensitivity. Gibbs sampling, similar to LDA, supports flexible deployment. Experiments show superior scarce-topic discovery and effective early detection of emerging topics or unexpected events on social platforms.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages7

Reading Time18 min

Introduction

Topic modeling background and motivations

Limitations of LDA and co-occurrence-based variants

Short-text topic modeling challenges

Related approaches (pseudo-documents, external knowledge, modified LDA, neural methods, BITERM)

Topic Model Based on Co-occurrence Word Networks for Unbalanced Short Text Datasets Chengjie Ma [macj163@163.com](macj163@163.com) Beijing Key Laboratory of Intelligent Telecommunication Software and Multimedia, Beijing University of Posts and Telecommunications Beijing, China Junping Du∗ [junpingdu@126.com](junpingdu@126.com) Beijing Key Laboratory of Intelligent Telecommunication Software and Multimedia, Beijing University of Posts and Telecommunications Beijing, China Meiyu Liang [meiyu1210@bupt.edu.cn](meiyu1210@bupt.edu.cn) Beijing Key Laboratory of Intelligent Telecommunication Software and Multimedia, Beijing University of Posts and Telecommunications Beijing, China Zeli Guan [guanzeli@bupt.edu.cn](guanzeli@bupt.edu.cn) Beijing Key Laboratory of Intelligent Telecommunication Software and Multimedia, Beijing University of Posts and Telecommunications Beijing, China arXiv :2311 .02566v2 [ cs .CL] 10 Jul 2026 Abstract We propose a straightforward solution for detecting scarce topics in unbalanced short-text datasets. Our approach, named CWUTM (Topic model based on co-occurrence word networks for unbalanced short text datasets), addresses the challenge of sparse and unbalanced short text topics by mitigating the effects of incidental word co-occurrence. This allows our model to prioritize the identification of scarce topics (low-frequency topics) . Unlike previous methods, CWUTM leverages co-occurrence word networks to capture the topic distribution of each word, and enhances the sensitivity in identifying scarce topics by redefining the calculation of node activity and normalizing the representation of both scarce and abundant topics to some extent. Moreover, CWUTM adopts Gibbs sampling, similar to LDA, making it easily adaptable to various application scenarios. Extensive experimental validation on unbalanced short-text datasets demonstrates the superiority of CWUTM compared to baseline approaches in discovering scarce topics. According to the experimental results, the proposed model is effective in early and accurate detection of emerging topics or unexpected events on social platforms. Keywords Scarce topic, co-occurrence network, unbalanced datasets 1 Introduction Topic models serve as statistical tools used to uncover concealed semantic structures within document collections [1]. These models, along with their extensions, have found applications in diverse fields including marketing, sociology, political science, among others [2] . In applied intelligent decision scenarios, interpretable machine learning also emphasizes that learned semantic features should remain transparent and actionable [3] . ∗ Corresponding author. This work was supported by the Program of the National Natural Science Foundation of China (62192784, U22B2038, 62172056) and by Young Elite Scientists Sponsorship Program by CAST (2022QNRC001) . Most topic models are advancements built upon the latent Dirichlet allocation (LDA) technique [1]. LDA, being the canonical form of existing topic models, employs a hierarchical parametric Bayesian approach to uncover topics within extensive corpora. It represents documents as a mixture of topics, with each topic being a probability distribution of words from the corpus vocabulary. Through statistical inference, LDA learns the probability distribution of words associated with each topic and the topic distribution for each document. However, LDA-like models, which leverage document-level word co-occurrence information [4], tend to consolidate semantically related words into a single topic. This characteristic makes them highly sensitive to the length and quantity of documents attributed to each topic. Consequently, when working with short texts that contain a limited number of words, these models fail to capture the relationships among words. With the rapid evolution of the World Wide Web and the emergence of various web applications, short texts have become p

FAQs

What problem does CWUTM address in unbalanced short-text datasets?

CWUTM focuses on detecting scarce topics in settings where topics are sparse and unevenly distributed, which makes low-frequency topics hard to learn and distinguish in short texts.

How does CWUTM differ from LDA-like topic models?

CWUTM uses co-occurrence word networks to characterize the topic distribution of each word and redesigns node activity calculation, aiming to reduce the impact of incidental co-occurrence on scarce topic detection.

What inference method does CWUTM use and why is it useful?

CWUTM adopts Gibbs sampling, similar to LDA, which makes the model easier to adapt to different application scenarios.

Jul 16, 2026

Upload by Jiven