A Speech Corpus for Mizo Automatic Speech Recognition - Whisper and SraVaani 1.0 Fine-Tuning with Morphology-Aware Evaluation

Sep 15, 2026

Uploaded by Miles

This study reports the development of an Automatic Speech Recognition (ASR) system for Mizo, a low-resource Tibeto-Burman language. The work includes collecting and curating 17.62 hours of speech data, then fine-tuning multilingual Whisper models and the SraVaani 1.0 Indic model. Whisper-large-v3 yields the lowest conventional WER (18.08%), while morphology-aware evaluation gives 7.22%. SraVaani 1.0 zero-shot performance reaches 58.27% WER, with fine-tuning reducing conventional WER to 29.45% and morphology-aware WER to 17.93%, showing large gains from language-adaptive training.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages6

Reading Time15 min

AccessFree to download

Introduction

Mizo speech ASR development background

Mizo Speech corpus

Data collection

A Speech Corpus for Mizo Automatic Speech Recognition: Whisper and SraVaani 1.0 Fine-Tuning with Morphology-Aware Evaluation Priyankoo Sarmah , Sanasam Ranbir Singh , Lalhmingmawia Center for Linguistic Science and Technology Indian Institute of Technology Guwahati, India {priyankoo,[ranbir](ranbir}@iitg.ac.in)[}](ranbir}@iitg.ac.in)[@iitg.ac.in](ranbir}@iitg.ac.in) , [mapuia442@gmail.com](mapuia442@gmail.com) arXiv : 2608 . 1936 1v 1 [cs .CL] 19 Aug 2026 Abstract This study reports the development of an Automatic Speech Recognition (ASR) system in Mizo, a low-resource language. The development included collecting 17.62 hours of speech data, curating it, and fine-tuning the Mizo ASR system with three Whisper multilingual models and with the SraVaani 1.0 Indic multilingual model. Whisper-large-v3 achieved the lowest conventional WER (18 .08%), while morphology-aware evaluation yielded a WER of 7.22% . Zero-shot evaluation of the SraVaani 1.0 Indic multilingual model yielded a WER of 58.27%, while Mizo-specific fine-tuning reduced the conventional WER to 29.45% and the morphology-aware WER to 17.93% . The results demonstrate that the Whisper model can achieve a substantially low WER, even when adapted to an unseen language. In contrast, SraVaani 1.0 supports the Mizo language in its multilingual model; however, fine-tuning with carefully curated Mizo speech data substantially improves its performance. Index Terms: ASR, Mizo, Whisper fine tuning,SraVaani 1.0 1. Introduction Mizo is a low-resource Tibeto-Burman language spoken in the Mizoram province of India, and in parts of Myanmar and Bangladesh. This paper reports the development of a spoken data resource for Mizo Automatic Speech Recognition (ASR) system and the subsequent fine-tuning of publicly available Mizo ASR models to build an ASR system for the language. Specifically, two multilingual models are fine-tuned on our data: Whisper [1] and SraVaani 1.0 [2] . Mizo is not included in Whisper’s original language set, whereas it is supported by SraVaani 1.0. We therefore investigate whether pretrained multilingual Whisper models can be effectively adapted to Mizo using the available training data. It has been reported that Whisper models can be successfully adapted to previously unseen languages [3] . A study on Kildin-Sami showed that fine-tuning with only 30 minutes of transcribed Kildin-Sami speech using the Russian-based Whisper model yielded a modest WER of 68.55%[4] . These findings motivate the investigation of Whisper-based adaptation to Mizo. In contrast, SraVaani 1.0 [2] supports Mizo; hence, our first goal is to conduct a zero-shot test on our Mizo test set using the SraVaani 1.0 multilingual model. The model was subsequently fine-tuned on the Mizo training data to assess the effect of language-specific adaptation. The development of Mizo continuous ASR has progressed in stages over the last 10 years. Starting with simple phone and digit recognition systems for Mizo [5, 6], it has progressed to domain-specific, limited-vocabulary speech recognition and continuous speech recognition [7, 8] . Recently, studies Table 1: Data split for training, validation and testing. | | Speakers | Sentences | Hours | | --- | --- | --- | --- | | Training | 184 | 7656 | 16.18 | | Validation | 11 | 426 | 1.02 | | Testing | 05 | 192 | 0.42 | have shown the effectiveness of fine-tuning with Wav2vec 2.0 and XLS-R models, resulting in substantially improved performance in Mizo ASR [9] . It is reported that the Wav2vec-BaseMizo-Lus and the XLS-R-300M-Mizo-Lus models’WERs are 16.59% and 11.84%, respectively [9] . A major limitation in the development of Mizo ASR systems has been the limited availability of publicly accessible speech data. Hence, in this work, we report the collection and curation of speech data, which is released to support further development of Mizo language technologies. The development of the corpus is reported in Section 2 . Following t

FAQs

What datasets and models are used to build the Mizo ASR system?

The study collects and curates 17.62 hours of Mizo speech data and fine-tunes three multilingual Whisper models as well as the SraVaani 1.0 Indic multilingual model on the Mizo data.

How does morphology-aware evaluation change the reported error rates?

Morphology-aware evaluation produces a lower WER than conventional evaluation for the best-performing Whisper model, yielding 7.22% WER with Whisper-large-v3, compared with 18.08% conventional WER.

What effect does fine-tuning have on SraVaani 1.0 for Mizo?

SraVaani 1.0 achieves 58.27% WER in zero-shot evaluation on the Mizo test set, while fine-tuning on the curated Mizo training data reduces conventional WER to 29.45% and morphology-aware WER to 17.93%.

Sep 15, 2026

Uploaded by Miles