PSK at WMT 2026 MIST - Task-Specialized QLoRA Adapters for Multilingual Summarization and Question Answering

Sep 15, 2026

PSK submission to the WMT 2026 Multilingual Instruction Shared Task (MIST) presents a system built on the 3.35B-parameter Tiny Aya Global backbone using three task-specific QLoRA adapters. Summarization and question answering use multilingual document-summary pairs, passage-based QA, and filtered standalone QA, with scientific summarization including author-written abstracts. On a held-out split, context and summarization adapters outperform the multitask adapter, while open QA results remain mixed by answer length and evaluation setting. Three systems share identical context and summarization adapters but differ for open QA.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages5

Reading Time13 min

AccessFree to download

Introduction

WMT 2026 Multilingual Instruction Shared Task (MIST)

One multilingual backbone with three task-specific adapters

Task and System Overview

Base Model and Initial Adaptation

Training Data

Summarization

Context Question Answering

Open-Ended Question Answering

Training and Inference

Development Results

PSK at WMT 2026 MIST: Task-Specialized QLoRA Adapters for Multilingual Summarization and Question Answering Srikar Kashyap Pulipaka Independent Researcher [srikar.kashyap@gmail.com](srikar.kashyap@gmail.com) arXiv : 2608 .20757v 1 [cs .CL] 2 1 Aug 2026 Abstract We describe the PSK submission to the WMT 2026 Multilingual Instruction Shared Task. Our system uses the 3.35B-parameter Tiny Aya Global model with three QLoRA adapters, one for each task. The adapters are trained on multilingual document–summary pairs, passagebased question answering, and filtered standalone question answering. The summarization data also includes scientific papers with their author-written abstracts. On our held-out split, the context and summarization adapters perform better than our multitask adapter, which was trained only on data supplied by the organizers. Results for open QA are mixed and vary with answer length and evaluation method. We therefore submit three systems with the same context and summarization adapters but different open-QA adapters. 1 Introduction The WMT 2026 Multilingual Instruction Shared Task (MIST) evaluates models under a 10Bparameter limit on three tasks: context-based question answering, summarization from a document in language X to language Y , and open-ended generation (WMT 2026 MIST Organizers, 2026) . The test set covers 24 languages and contains both same-language and cross-lingual generation. MIST follows the broader multilingual evaluation effort introduced in WMT 2025, which found substantial variation across tasks and languages and also showed that automatic metrics are imperfect for open generation (Kocmi et al., 2025) . We use one multilingual backbone with three task-specific adapters. Tiny Aya Global (Salamanca et al., 2026) is first adapted to the provided data and then continued separately for each task. This design preserves a single 3.35B backbone while allowing the supervision and decoding policy to follow the output structure of each task. 2 Task and System Overview MIST provides a task label at inference time. We use it to select one of three LoRA adapters and the corresponding decoding configuration (Figure 1); there is no language-specific routing. 2.1 Base Model and Initial Adaptation We use CohereLabs/tiny-aya-global, a 3 .35Bparameter model covering 70 languages (Salamanca et al., 2026) . Its 8K context window supports the longer task inputs. An initial multitask adapter is trained on 20,293 provided examples using a deterministic 85/15 split; the resulting 3,626 held-out examples are reused for model selection. The task-specific adapters are initialized from this model. 3 Training Data Table 1 summarizes the task-specific training data. Validation examples and exact test prompts are excluded from training. 3.1 Summarization The 12,000-example mixture is divided evenly between general and scientific text. General examples combine provided data with CrossSum (Bhattacharjee et al., 2023), WikiLingua (Ladhak et al., 2020), and UPDESH (Chitale et al., 2026) . The scientific portion uses full ACL Anthology papers (Bird et al., 2008) with author-written abstracts. No Language Left Behind (NLLB) (NLLB Team, 2024) supplies non-English targets, which are filtered for language, length, repetition, and semantic consistency. We remove test overlaps and placeholderheavy papers. We also screen the provided data for likely misalignment by comparing each summary with chunks of its source document and discardinga pair only when both its best LaBSE similarity and best chrF score fall below fixed thresholds. Figure 1: The three routes share one frozen Tiny Aya Global backbone. The known task label selects an adapter and its decoding policy; language does not change the route. | Adapter | Rows | Context | Main sources and construction | | --- | --- | --- | --- | | Multitask adapter Summarization
Context QA Open QA | 20,293
12,000
8,512
6,400 | 8,192
8,192
4,0

FAQs

What is the core model and how is it adapted in this submission?

The system uses the 3.35B-parameter Tiny Aya Global model and applies an initial multitask QLoRA adapter, then continues training separately to create three task-specific adapters.

How does the submission handle the three MIST tasks during inference?

At inference time, the task label selects one of the three LoRA adapters and its corresponding decoding configuration, with no language-specific routing.

What training data is used for summarization and how are scientific abstracts included?

Summarization training mixes general sources with CrossSum, WikiLingua, and UPDESH, and includes full ACL Anthology papers paired with author-written abstracts, while filtering targets using language, length, repetition, and semantic consistency checks.

Sep 15, 2026

Uploaded by Genevieve