Echoes Across Vietnam’s Highlands, Delta, and Coast Multilingual Corpus for Cham, Khmer, and Tay-Nung

Jul 16, 2026

Vietnam’s ethnic minority languages remain largely absent from Natural Language Processing, and the barrier goes beyond data scarcity. Cham, Khmer, and Tay-Nung differ in script, contact with Vietnamese, and standardization, causing multilingual adaptation to learn incorrect signals. CKTN is presented as the first corpus and benchmark for these languages, with 44,367 documents and 24M subword tokens, enabling continued pretraining, category classification, and summary-document retrieval.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages18

Reading Time45 min

Introduction

Related Work

Low-Resource Corpus for Vietnamese Ethnic Languages

Echoes Across Vietnam’s Highlands, Delta, and Coast: A Multilingual Corpus for Cham, Khmer, and Tay-Nung Anh Trac Duc Dinh1*†, Khang Nhat Hoang Vo2*†, Vinh Cong Doan1 , Tai Tien Ta 1 , Khoa Duc Anh Lam 1 1Faculty of Computer Science and Engineering, Ho Chi Minh City University of Technology (HCMUT), VNU-HCM, Ho Chi Minh City, Vietnam 2Mohamed bin Zayed University of Artificial Intelligence, Abu Dhabi, UAE *Corresponding author: [anh.dinhtracduc@hcmut.edu.vn](anh.dinhtracduc@hcmut.edu.vn); [Khang.Vo@mbzuai.ac.ae](Khang.Vo@mbzuai.ac.ae) arXiv :2607 .08362v 1 [ cs .CL] 9 Jul 2026 Abstract Vietnam’s ethnic minority languages are almost absent from the field of Natural Lan guage Processing (NLP), and the challenge goes beyond data scarcity: Cham, Khmer, and Tay-Nung differ sharply in script, Vietnamese contact, and standardization, conditions under which standard multilingual adaptation can learn the wrong signals. We introduce CKTN, the first corpus and benchmark for these languages (44,367 documents, 24M subword tokens), spanning continued pretraining, category classification, and summary-document retrieval. We show that existing multilingual encoders severely fragment these languages, and that common adaptation metrics can mislead: models may lower language-modeling loss or excel at lexical-overlap retrieval while still failing at semantic generalization across documents. We address this with a script-aware adaptation recipe-vocabulary augmentation combined with calibrated replaced-token pretraining-that prevents the discriminator from exploiting trivial script mismatches. The result is an encoder with substantially less fragmentation and the strongest classification performance among evaluated models, exposing the limits of lexical-overlap retrieval as an evaluation signal. 1 Introduction Low-resource NLP is often framed as a problem of missing data. For many minority languages, however, scarcity is only one part of the difficulty. Languages spoken under sustained contact with a dominant national language may diverge from their higher-resource genealogical relatives, develop unstable or mixed writing practices, and appear in digital text through scripts that are poorly represented by multilingual tokenizers. In such settings, adaptation can fail even when some text is available: models may learn local subword regularities, †These authors contributed equally to this work, and are project leaders. Names are ordered alphabetically. lexical overlap, or script cues rather than transferable language structure. Vietnam’s ethnic minority languages offer a compelling testbed for this problem. Vietnam recognizes 54 ethnic groups (Van, 2002 ; Tappe, 2015); beyond the Kinh majority, over 14 million people speak minority languages spanning the Austroasiatic, Tai-Kadai, Austronesian, and Sino-Tibetan families (Edmondson and Gregerson, 2007 ; Liu et al., 2020) . We focus on three languages actively published in digital media yet almost absent from NLP: Cham, Khmer, and Tay-Nung. Together they form a natural contrast set: Cham (Austronesian, south-central coast) is traditionally written ina Brahmic-derived abugida but often appears in Latin script online; Khmer (Austroasiatic, Mekong Delta) uses a script without explicit word boundaries; and Tay-Nung (Tai-Kadai varieties of the northern highlands) uses Latin-based orthography but has limited standardized digital text. These languages thus span different regions, scripts, language families, and degrees of visible contact with Vietnamese. This diversity is not merely sociolinguistic background; it creates concrete modeling failures. Existing multilingual encoders such as mBERT, XLM-R, and RemBERT (Pires et al., 2019 ; Conneau et al., 2020 ; Chung et al., 2021) allocate little vocabulary capacity to these languages, fragmenting words into continuation pieces and weakening lexical representations. Moreover, script heterogeneity creates a specific hazard for ELECTRAsty

FAQs

What problem does the paper address in NLP for Vietnam’s ethnic minority languages?

It addresses the gap where Cham, Khmer, and Tay-Nung are largely absent from NLP, and explains that adaptation can fail due to script and standardization differences, not just data scarcity.

What is CKTN and what tasks does it support?

CKTN is the first multilingual corpus and benchmark for Cham, Khmer, and Tay-Nung. It includes 44,367 documents and 24M subword tokens, supporting continued pretraining, 28-way category classification, and summary-document retrieval.

Why can common adaptation metrics give misleading results?

The paper shows that models may reduce language-modeling loss or perform well on lexical-overlap retrieval without achieving semantic generalization across documents, so evaluation based on lexical overlap can overestimate real representation quality.

Jul 16, 2026

Upload by River Wang