StatLLaMA - Multi-Stage Training for Domain-Optimized Statistical Large Language Models - Abstract

Sep 19, 2026

This study investigates how to efficiently build a domain-specialized large language model for statistics using the lightweight LLaMA-3.2-3B family as the foundation model. It systematically compares three multi-stage training pipelines across continual pretraining, supervised fine-tuning, and RLHF preference alignment. Pipelines starting from a base model without instruction-following fail to develop meaningful statistical reasoning. In contrast, starting from LLaMA-3.2-3B-Instruct enables effective domain specialization with clear trade-offs. Direct preference optimization yields stable RLHF alignment, and StatLLaMA attains strong balanced performance on reasoning and statistical benchmarks.

LanguageEnglish

CategoryGeneral

FormatPDF

Pages39

Reading Time14 min

AccessFree to download

Abstract

Introduction

Motivation and background on LLMs

Supervised fine-tuning and parameter-efficient approaches

Continual pretraining vs instruction tuning

Preference alignment: RLHF, DPO, and GRPO

June 2026, Volume VI, Issue IV. doi: 10.52933/jdssv.v6i4.171 StatLLaMA: Multi-Stage Training for Domain-Optimized Statistical Large Language Models Jing-Yi Zeng National Yang Ming Chiao Tung University Guan-Hua Huang National Yang Ming Chiao Tung University Abstract This study investigates how to efficiently build a domain-specialized large language model (LLM) for statistics using the lightweight LLaMA-3.2-3B family as the foundation model (FM) . We systematically compare three multi-stage training pipelines—starting from a base FM with no instruction-following capability, a base FM augmented with post-hoc instruction tuning, and an instructiontuned FM with strong general reasoning abilities—across continual pretraining, supervised fine-tuning (SFT), and reinforcement learning from human feedback (RLHF) preference alignment. Results show that pipelines beginning with abase FM fail to develop meaningful statistical reasoning, even after extensive instruction tuning, SFT, or RLHF alignment. In contrast, starting from LLaMA- 3.2-3B-Instruct enables effective domain specialization. A comprehensive evaluation of SFT variants reveals clear trade-offs between domain expertise and general reasoning ability. We further demonstrate that direct preference optimization provides stable and effective RLHF preference alignment. The final model, StatLLaMA, achieves strong and balanced performance on benchmarks of mathematical reasoning, common-sense reasoning, and statistical expertise, offering a practical blueprint for developing resource-efficient statistical LLMs. The code is available at [https://github.com/HuangDLab/StatLLaMA](https://github.com/HuangDLab/StatLLaMA). 2 StatLLaMA, Version 1 Keywords: Foundation model, supervised fine-tuning, continual pretraining, instruction tuning, reinforcement learning from human feedback. 1 . Introduction Large language models (LLMs) based on Transformer architectures have become a central tool in modern natural language processing. Models such as BERT (Devlin et al. 2019), GPT (Radford et al. 2018 , 2019 ; Brown et al. 2020), and the LLaMA family (Touvron et al. 2023a,b; Grattafiori et al. 2024) have demonstrated strong performance across a wide range of general-purpose language tasks, including reasoning, summarization, and question answering. Trained on massive text corpora, these foundation models exhibit broad linguistic competence and extensive general knowledge. However, their effectiveness in specialized technical domains—particularly statistics—remains limited. Statistical reasoning requires precise use of domain-specific terminology, adherence to formal definitions, and careful multi-step analytical reasoning. In practice, generalpurpose LLMs often produce explanations that are shallow, imprecise, or conceptually incorrect when applied to statistical tasks. This gap raises an important applied question: how can statistical knowledge and reasoning be systematically integrated into LLMs without degrading their general language and reasoning abilities? This study addresses this question by developing and evaluating a resource-efficient training framework for constructing a statistical LLM. We focus on the lightweight LLaMA-3 .2-3B model, motivated by the practical need for deployable models under constrained computational resources. Our objective is to build a model that achieves strong performance on statistical reasoning tasks while maintaining robust general-purpose capabilities. Supervised fine-tuning (SFT) is the standard approach for adapting pretrained language models to downstream tasks. However, full fine-tuning becomes prohibitively expensive for large models and is inefficient when multiple domain-specific variants are required. Parameter-efficient fine-tuning (PEFT) methods, particularly low-rank adaptation (LoRA) (Hu et al. 2021), address this limitation by updating only a small subset of parameters while keeping the base model fixed. These techniques s

FAQs

What is the main goal of StatLLaMA?

To build an efficient statistical large language model by leveraging multi-stage training that improves domain expertise while preserving general reasoning and language capabilities.

How do the three training pipelines differ in their effectiveness?

Pipelines that start from a base foundation model without instruction-following fail to develop meaningful statistical reasoning, while starting from LLaMA-3.2-3B-Instruct enables effective domain specialization with measurable trade-offs.

Why is preference alignment included, and what methods are compared?

Preference alignment addresses challenges in matching human judgments for nuanced reasoning. The document discusses RLHF and also compares more efficient alternatives such as direct preference optimization (DPO) and group relative policy optimization (GRPO).

Sep 19, 2026

Upload by Lucas Martin