[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-84841-en":3,"doc-seo-84841-105":29,"detail-sidebar-cat-0-en-105":83},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":13,"seo_description":14,"update_tm":27,"read_time":28},84841,8796095462418,"Noah","https://ap-avatar.wpscdn.com/avatar/80000253c1241d02b47?x-image-process=image/resize,m_fixed,w_180,h_180&k=1778826106357471780",8,"Research & Report","NAVER LABS System Re-implementation for the IWSLT 2026 Instruction-Following Task","Re-implements the NAVER LABS IWSLT 2025 instruction-following pipeline for the IWSLT 2026 Shared Task under constrained, short-audio conditions, replacing mandated components with SeamlessM4Tv2-large as the speech encoder and Qwen3-4B-Instruct as the LLM backbone. Retains the three-stage strategy of projector alignment, text-only LoRA pretraining, and multimodal merging. Builds 100k synthetic instruction-following examples across ten speech-centric task types for further Stage 3 fine-tuning, and reports COMET 0.781 for EN–ZH speech translation plus BERTScore-F1 0.346 for English SQA on MCIF.","NAVER LABS System Re-implementation for the IWSLT 2026 Instruction-Following Task  \nAnand Kamble  \nFlorida State University [amk23j@fsu.edu](amk23j@fsu.edu)  \nAniket Tathe  \nUniversity of Illinois Urbana-Champaign [atathe@illinois.edu](atathe@illinois.edu)  \narXiv :2607 .05623v 1 [ cs .CL] 6 Jul 2026  \nAbstract  \nWe re-implement the NAVER LABS IWSLT 2025 instruction-following pipeline (Lee et al., 2025) for the IWSLT 2026 Shared Task (constrained condition, short audio track), adapting it to the mandated components: SeamlessM4Tv2-large (Barrault et al., 2023) as the speech encoder and Qwen3-4B-Instruct (Team, 2025) asthe LLM backbone. The three-stage approach—projector alignment, text-only LoRA pretraining, and multimodal merging—is preserved from the original design. We additionally construct 100k synthetic instructionfollowing examples across ten speech-centric task types (10k per task) from the provided corpora, suitable for further Stage 3 fine-tuning. Our primary model achieves COMET 0.781 on EN–ZH speech translation and BERTScore-F1 0.346 on English SQA on theMCIF benchmark.  \nCode, training scripts, and generated data are publicly available.1  \n1 Introduction  \nMultimodal speech LLMs such as SALMONN (Tang et al., 2024), Qwen-Audio (Chu et al., 2023), SpeechGPT (Zhang et al., 2023), and WavLLM (Hu et al., 2024) couple a frozen speech encoder with an instruction-tuned LLM (Ouyang et al., 2022) via a lightweight connector, enabling flexible multi-task inference through natural language prompts. The IWSLT 2026 Instruction-Following Shared Task (Organizers, 2026) formalizes this paradigm with the MCIF benchmark, evaluating unified models on ASR, multilingual ST (EN→{DE,IT,ZH}), and SQA. The NAVER LABS 2025 system (Lee et al., 2025) demonstrated a competitive three-stage pipeline in the IWSLT 2025 constrained setting (Abdulmuminet al., 2025), but was not publicly released.  \nWe provide the first open-source reimplementation, adapted to the IWSLT 2026  \n1 [https://github.com/anand-kamble/](https://github.com/anand-kamble/)[ ](https://github.com/anand-kamble/)iwslt2026-instruction-following  \n\n| Dataset | Task | Lang. | Stage(s) |\n| --- | --- | --- | --- |\n| CoVoST 2 | ASR, ST | EN→{DE,IT,ZH} | 1, 2∗ , 3 |\n| EuroParlST | ASR, ST | EN→{DE,IT,ZH} | 1, 2∗ , 3 |\n| LibriSQA | SQA | EN (+ DE,IT,ZH‡) | 1†, 2∗ , 3 |\n| NUTSHELL | S2TSum | EN | 3 |\n| YTSeg | AChap | EN | 3 |\n\nTable 1: Training corpora per stage. ∗Text-only. †A.2 variant only. ‡Machine-translated.  \nconstraints (SeamlessM4T-v2-large encoder, Qwen3-4B-Instruct LLM—replacing the LLaMA- 3.1-8B (Dubey et al., 2024) backbone used in 2025) . We further construct 100k synthetic instruction-following examples across ten speechcentric task types (Section 2.1) and ablate LoRA rank and learning rate configurations for Stage 2 text pre-training.  \n2 Task and Data  \nShared Task. We participate in the constrained condition, short audio track (Organizers, 2026) . Evaluation on MCIF uses WER (↓) for ASR, COMET (Rei et al., 2020) (↑) for ST, and BERTScore-F1 (Zhang et al., 2020) (↑) for SQA. Task instructions follow the natural-language prompt format of Lee et al. (2025) .  \nTraining Corpora. Core speech data is from CoVoST 2 (Wang et al., 2021) and EuroParlST (Iranzo-Sánchez et al., 2020) (ASR/ST) and LibriSQA (Huang et al., 2024) (SQA); multilingual SQA pairs in DE, IT, ZH are obtained by machine-translating LibriSQA via SeamlessM4Tv2 . Stage 3 additionally draws on NUTSHELL (Maikezu et al., 2024) for speech summarization and YTSeg (Retkowski et al., 2024) for audio chapter detection. Table 1 summarizes corpora per stage.  \nFigure 1: Three-stage training pipeline. Frozen modules: dashed border. Trainable: solid. Stage 3 jointly fine-tunes both projector and LoRA adapters.  \n2.1 Synthetic Instruction-Following Data  \nWe construct 100k synthetic examples (10k per task) from the provided corpora using open-weight Gemma models (Gemma Team, 2025) . Seven text-grounded tasks are generated by ","cbCaip4p8cHcDhsM","https://ap.wps.com/l/cbCaip4p8cHcDhsM","pdf",326192,1,5,"English","en",105,"# Introduction\n# Task and Data\n## Synthetic Instruction-Following Data\n# System\n## Architecture\n## Stage 1—Projector Alignment\n## Stage 2—Text-Only LoRA\n## Stage 3—Multimodal Merge","[{\"question\":\"How does the model train across the three stages?\",\"answer\":\"Stage 1 trains only the projector with the encoder and LLM frozen; Stage 2 performs text-only LoRA adaptation with the projector frozen; Stage 3 fine-tunes the projector together with the LoRA adapters under multimodal merging.\"}]",1784198703,13,{"code":4,"msg":30,"data":31},"ok",{"site_id":24,"language":23,"slug":32,"title":13,"keywords":33,"description":14,"schema_data":34,"social_meta":78,"head_meta":80,"extra_data":82,"updated_unix":27},"naver-labs-system-re-implementation-for-the-iwslt-2026-instruction-following-task","",{"@graph":35,"@context":77},[36,53,68],{"@type":37,"itemListElement":38},"BreadcrumbList",[39,43,47,50],{"item":40,"name":41,"@type":42,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":44,"name":45,"@type":42,"position":46},"https://docshare.wps.com/document/","Document",2,{"item":48,"name":12,"@type":42,"position":49},"https://docshare.wps.com/document/research-report/",3,{"item":51,"name":13,"@type":42,"position":52},"https://docshare.wps.com/document/naver-labs-system-re-implementation-for-the-iwslt-2026-instruction-following-task/84841/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":23,"description":14,"dateModified":61,"datePublished":62,"encodingFormat":60,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":40,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-17","2026-07-16",true,{"@type":65,"interactionType":66,"userInteractionCount":20},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71],{"name":72,"@type":73,"acceptedAnswer":74},"How does the model train across the three stages?","Question",{"text":75,"@type":76},"Stage 1 trains only the projector with the encoder and LLM frozen; Stage 2 performs text-only LoRA adaptation with the projector frozen; Stage 3 fine-tunes the projector together with the LoRA adapters under multimodal merging.","Answer","https://schema.org",{"og:url":51,"og:type":79,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":81,"canonical":51},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":84},[85,89,93,97,101,106,111,114,119,122,126],{"id":20,"doc_module":4,"doc_module_name":45,"category_name":86,"show_sort_weight":87,"slug":88},"Story & Novel",90,"story-novel",{"id":46,"doc_module":4,"doc_module_name":45,"category_name":90,"show_sort_weight":91,"slug":92},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":45,"category_name":94,"show_sort_weight":95,"slug":96},"Exam",70,"exam",{"id":21,"doc_module":4,"doc_module_name":45,"category_name":98,"show_sort_weight":99,"slug":100},"Comic",60,"comic",{"id":102,"doc_module":4,"doc_module_name":45,"category_name":103,"show_sort_weight":104,"slug":105},6,"Technology",50,"technology",{"id":107,"doc_module":4,"doc_module_name":45,"category_name":108,"show_sort_weight":109,"slug":110},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":45,"category_name":12,"show_sort_weight":112,"slug":113},30,"research-report",{"id":115,"doc_module":4,"doc_module_name":45,"category_name":116,"show_sort_weight":117,"slug":118},9,"Religion & Spirituality",20,"religion-spirituality",{"id":117,"doc_module":4,"doc_module_name":45,"category_name":120,"show_sort_weight":117,"slug":121},"World Cup","world-cup",{"id":123,"doc_module":4,"doc_module_name":45,"category_name":124,"show_sort_weight":123,"slug":125},10,"Lifestyle","lifestyle",{"id":127,"doc_module":4,"doc_module_name":45,"category_name":128,"show_sort_weight":21,"slug":129},19,"General","general"]