Efficiently Adapting Spoken Language Models for the Singaporean Context

Jul 16, 2026

Uploaded by Liam

Spoken language models (SLMs) combine speech perception with reasoning, yet adapting them to sensitive, multilingual domains remains insufficiently studied. The work adapts an open-source SLM for Singapore’s Home Team context across five speech tasks and four official languages, using LoRA fine-tuning, a surrogate text-QA dataset to reduce catastrophic forgetting, and a multi-task objective extending CoBa reweighting to speech. It introduces HTD-multilingualQA (504,853 samples) and evaluates HTMoonstone (5B), which matches or exceeds larger SLMs by up to 7× on most tasks while degrading original speech QA by under 2%.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages10

Reading Time25 min

AccessFree to download

Introduction

Background and motivation

Key contributions

Related Work

Rise of Spoken Language Models

Singaporean Audio-Text Datasets

Efficiently Adapting Spoken Language Models for the Singaporean Context Language AI R&D, xData Home Team Science & Technology Agency (HTX), Singapore Ng Jia Sheng Jason arXiv :2607 . 10092v 1 [ cs .CL] 11 Jul 2026 Abstract Spoken language models (SLMs) unify speech perception and reasoning, but adapting them to sensitive domains is underexplored, especially when the original training data is inaccessible and the use case demands multilingual, spokenquery interaction. We adapt an open-source SLM to the Singaporean Home Team context across five speech tasks in Singapore’s four official languages, combining LoRA fine-tuning, a surrogate text-QA dataset that guards against catastrophic forgetting, and a multi-task objective that adapts the CoBa reweighting scheme to speech. We also build HTD-multilingualQA, a 504,853 sample multilingual QA dataset in text and spoken form. The resulting HTMoonstone (5B) matches or outperforms SLMs up to 7× its size on most tasks, attains the best accent and gender recognition among all models evaluated, and loses under 2% of its original speech QA ability. 1 Introduction The Home Team Science and Technology Agency (HTX) is a statutory board under Singapore’s Ministry of Home Affairs (MHA) . It serves as a “force multiplier” for the Home Team departments, applying science and technology to strengthen homeland security and public safety. Among the use cases HTX receives from these departments are speech tasks such as automatic speech recognition (ASR) and spoken question answering (QA), the latter forming the core of speech-to-speech conversational AI applications. Because much of this work involves highly sensitive data, closed-source or proprietary models are often unsuitable, which is why we turn to open-source alternatives. HTX’s Language AI R&D team has been developing and adapting open-source foundational speech models, such as those for ASR. However, the field is moving towards spoken language models (SLMs) (Arora et al., 2025), which handle a wide range of cognitive and perception speech tasks (Peng et al., 2024) . Several open-source SLMs already exist, including Kimi-Audio (Kimi Team, 2025) and Audio-Flamingo-Next (Ghosh et al., 2026), as well as models built for the Singaporean context, such as MERaLiON-2 (MERaLiON Team, 2024) . However, because our operating environment and use cases are specific to the Home Team context, a suitable model must either be built from scratch or adapted from an existing open-source SLM. Training from scratch demands far more audio data, which is costly to acquire, so we favour adaptation. The SLM domain is still nascent, and few studies document how to adapt these models effectively for new tasks while preserving their original performance, particularly when the full dataset used to build them is unavailable. As we have use cases involving the use of text context and spoken queries and no Singaporean multilingual spoken QA training dataset currently exists, the development of a resource that covers Singaporean English, Mandarin, Bahasa Melayu, and Tamil is essential for adapting models to handle such spoken QA task effectively. Furthermore, adapting models without full access to their original training data often risks catastrophic forgetting. Our adaptation strategy is designed to mitigate this risk, ensuring we maximise performance on both perceptive and cognitive speech tasks in the Singaporean domain while preserving the model’s foundational capabilities. In this paper, we address these challenges and introduce: 1. a strategy to effectively adapt SLMs to the Singaporean context for multiple speech tasks; 2. HTD-multilingual-QA: a multilingual, multi-turn QA training dataset in both text and spoken form, grounded in Singaporean Home Team context that covers the 4 Singaporean languages-English, Mandarin, Bahasa Melayu, and Tamil for training SLMson spoken QA tasks; and 3. HT-Moonstone, an adapted 5B-parameter SLM that performs

FAQs

Why is adapting spoken language models to the Singaporean Home Team context challenging?

The domain involves highly sensitive data and typically requires multilingual spoken-query interaction. Additionally, the original training datasets for existing models are often inaccessible, making effective adaptation without losing prior abilities difficult.

What strategy is used to adapt an open-source SLM while mitigating catastrophic forgetting?

The approach combines LoRA fine-tuning, a surrogate text-QA dataset designed to guard against catastrophic forgetting, and a multi-task objective that adapts the CoBa reweighting scheme to speech.

What are HTD-multilingualQA and HTMoonstone, and how do they perform?

HTD-multilingualQA is a 504,853-sample multilingual QA dataset provided in both text and spoken form. HTMoonstone is an adapted 5B SLM that matches or outperforms SLMs up to 7× larger on most tasks, achieves the best accent and gender recognition among evaluated models, and shows under 2% degradation in original speech QA ability.

Jul 16, 2026

Uploaded by Liam