[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"detail-sidebar-cat-0-en-105":3,"doc-seo-273697-105":59,"doc-detail-273697-en":131},{"code":4,"msg":5,"data":6},0,"success",[7,13,18,23,28,33,38,43,48,51,55],{"id":8,"doc_module":4,"doc_module_name":9,"category_name":10,"show_sort_weight":11,"slug":12},1,"Document","Story & Novel",90,"story-novel",{"id":14,"doc_module":4,"doc_module_name":9,"category_name":15,"show_sort_weight":16,"slug":17},2,"Literature",80,"literature",{"id":19,"doc_module":4,"doc_module_name":9,"category_name":20,"show_sort_weight":21,"slug":22},4,"Exam",70,"exam",{"id":24,"doc_module":4,"doc_module_name":9,"category_name":25,"show_sort_weight":26,"slug":27},5,"Comic",60,"comic",{"id":29,"doc_module":4,"doc_module_name":9,"category_name":30,"show_sort_weight":31,"slug":32},6,"Technology",50,"technology",{"id":34,"doc_module":4,"doc_module_name":9,"category_name":35,"show_sort_weight":36,"slug":37},7,"Healthcare",40,"healthcare",{"id":39,"doc_module":4,"doc_module_name":9,"category_name":40,"show_sort_weight":41,"slug":42},8,"Research & Report",30,"research-report",{"id":44,"doc_module":4,"doc_module_name":9,"category_name":45,"show_sort_weight":46,"slug":47},9,"Religion & Spirituality",20,"religion-spirituality",{"id":46,"doc_module":4,"doc_module_name":9,"category_name":49,"show_sort_weight":46,"slug":50},"World Cup","world-cup",{"id":52,"doc_module":4,"doc_module_name":9,"category_name":53,"show_sort_weight":52,"slug":54},10,"Lifestyle","lifestyle",{"id":56,"doc_module":4,"doc_module_name":9,"category_name":57,"show_sort_weight":24,"slug":58},19,"General","general",{"code":4,"msg":60,"data":61},"ok",{"site_id":62,"language":63,"slug":64,"title":65,"keywords":66,"description":67,"schema_data":68,"social_meta":124,"head_meta":126,"extra_data":128,"updated_unix":130},105,"en","inject-align-recover-staged-post-training-for-retrieval-free-document-knowledge-internalization","Inject, Align, Recover - Staged Post-Training for Retrieval-Free Document Knowledge Internalization","","Large language models often struggle with question answering over a fixed, bounded document collection when retrieval is unavailable at inference time. The work studies document knowledge internalization: transforming a static corpus into usable parametric knowledge for retrieval-free QA. It introduces IAR, a three-stage post-training method that injects structured document knowledge, aligns QA behavior with answer-only supervision, and recovers general capabilities via post-hoc model merging across datasets and multiple model families.",{"@graph":69,"@context":123},[70,84,106],{"@type":71,"itemListElement":72},"BreadcrumbList",[73,77,79,82],{"item":74,"name":75,"@type":76,"position":8},"https://docshare.wps.com","Home","ListItem",{"item":78,"name":9,"@type":76,"position":14},"https://docshare.wps.com/document/",{"item":80,"name":40,"@type":76,"position":81},"https://docshare.wps.com/document/research-report/",3,{"item":83,"name":65,"@type":76,"position":19},"https://docshare.wps.com/document/inject-align-recover-staged-post-training-for-retrieval-free-document-knowledge-internalization/273697/",{"url":83,"name":65,"@type":85,"image":86,"author":91,"headline":65,"publisher":94,"fileFormat":97,"inLanguage":63,"description":67,"dateModified":98,"datePublished":99,"encodingFormat":97,"isAccessibleForFree":100,"interactionStatistic":101},"DigitalDocument",{"url":87,"@type":88,"width":89,"height":90},"https://docshare.wps.com/thumbnails/inject-align-recover-staged-post-training-for-retrieval-free-document-knowledge-internalization/273697.png","ImageObject",300,407,{"name":92,"@type":93},"Sage","Person",{"url":74,"name":95,"@type":96},"DocShare","Organization","application/pdf","2026-11-06","2026-09-15",true,{"@type":102,"interactionType":103,"userInteractionCount":105},"InteractionCounter",{"@type":104},"ViewAction",13,{"@type":107,"mainEntity":108},"FAQPage",[109,115,119],{"name":110,"@type":111,"acceptedAnswer":112},"What problem does the document address?","Question",{"text":113,"@type":114},"It studies retrieval-free question answering over a fixed document corpus, where the model cannot access the source documents at inference time.","Answer",{"name":116,"@type":111,"acceptedAnswer":117},"How does IAR differ from conventional continued pretraining and SFT?",{"text":118,"@type":114},"IAR uses three stages: Inject converts documents into instruction-conditioned supervised reconstruction tasks, Align adapts the model with answer-only QA supervision, and Recover merges the domain-adapted model with the base instruction model to restore general capability.",{"name":120,"@type":111,"acceptedAnswer":121},"What components does the paper suggest should be measured separately?",{"text":122,"@type":114},"It argues that document exposure, answer alignment, data recipe, token budget, and recovery should be evaluated independently rather than collapsed into a single fine-tuning comparison.","https://schema.org",{"og:url":83,"og:type":125,"og:title":65,"og:site_name":95,"og:description":67},"article",{"robots":127,"canonical":83},"index,follow",{"doc_id":129,"site_id":62},273697,1789472527,{"code":4,"msg":5,"data":132},{"doc_id":129,"user_id":133,"nickname":92,"user_avatar":134,"doc_module":4,"category_id":39,"category_name":40,"doc_title":65,"doc_description":67,"doc_content":135,"file_id":136,"file_url":137,"file_type":138,"file_size":139,"view_count":105,"is_deleted":4,"is_public":8,"is_downloadable":8,"audit_status":8,"page_count":140,"language":141,"language_code":63,"site_id":62,"html_lang":63,"table_of_contents":142,"faqs":143,"seo_title":144,"seo_description":67,"update_tm":130,"read_time":145,"preview_page_count":4},687197207057,"https://ap-avatar.wpscdn.com/davatar_29158cc5080c5b710cf443261637dec0","Inject, Align, Recover: Staged Post-Training for Retrieval-Free Document  \nKnowledge Internalization  \nQian Kou1∗†, Xiaofeng Shi1∗†, Xiaosong Qiu1 , Hua Zhou1‡  \n1Beijing Academy of Artificial Intelligence (BAAI)  \narXiv : 2608 .2028 1v 1 [cs .CL] 20 Aug 2026  \nAbstract  \nLarge language models often fail to answer questions about a bounded document collection when the source documents are not retrieved at inference time. We study this setting as document knowledge internalization: converting a fixed corpus into usable parametric knowledge for retrieval-free question answering. We propose IAR (Inject, Align, and Recover), a three-stage post-training framework that separates structured document knowledge injection, QA behavior alignment, and general ability recovery. Unlike conventional continued pretraining, Inject converts source documents into continuation, rewrite, and instruction-conditioned reconstruction objectives. Align then adapts the injected model with answeronly QA supervision, while Recover merges the domainadapted model with the base instruction model to recover general capabilities. Across Common Corpus (CC) and CCI, and across Llama, Phi, Qwen, and SmolLM model families, IAR improves the domain-primary domain-general frontier for retrieval-free document internalization. In the main comparison, IAR improves over Vanilla SFT on all four reported metrics in 7 of 8 dataset-model settings, with average gains of 3.6 percentage points in domain QA accuracy and 12.1 percentage points in mean general performance across IFEval, MMLU, and MSBench. Extended CC baselines show that LoRA and FAPM can win individual general metrics, but among methods that also reach leading or near-leading domain internalization, IAR retains one of the strongest general profiles.  \nIntroduction  \nRetrieval-augmented generation is the standard engineering answer to document-grounded question answering: retrieve passages from a corpus, append them to the prompt, and aska model to answer with that evidence (Lewis et al. 2020) . This paper studies a different setting. In many deployments, retrieval may be unavailable, undesirable for latency or privacy reasons, or deliberately removed to test whether posttraining has changed the model’s parametric knowledge. We call this setting document knowledge internalization. Given a bounded document collection and document-derived questions, the model must answer held-out questions without re-  \n∗These authors contributed equally.  \n†Corresponding authors: Qian Kou ([kouqian@baai.ac.cn](kouqian@baai.ac.cn)) and Xiaofeng Shi ([xfshi@baai.ac.cn](xfshi@baai.ac.cn)).  \n‡Project leader.  \nceiving the source documents at inference time. We evaluate this problem on Common Corpus (CC) (PleIAs 2024) and CCI (BAAI 2024), and use the abbreviations CC and CCI throughout the paper.  \nThe simplest solution is supervised fine-tuning (SFT) on document-derived question-answer pairs. This approach gives the model the same input-output format that it will see at test time, but the learning signal is sparse: only the facts selected by the QA generator contribute to the loss. Continuedpretraining (CPT) provides denser exposure to document text (Gururangan et al. 2020), but it does not directly teach the model how to answer questions. Both routes also face a general-capability trade-off: domain adaptation can improve target QA behavior while degrading instruction following or broad benchmark performance, a form of catastrophic forgetting (Kirkpatrick et al. 2017) . Document internalization therefore requires both domain acquisition and controlled recovery.  \nWe propose IAR, a three-stage framework that separates these functions. Unlike raw CPT, IAR does not merely continue language modeling on domain text. Inject converts documents into instruction-conditioned supervised reconstruction tasks; Align maps the injected knowledge to a QA interface; and Recover performs post-hoc model merging between the domain-adapted checkpoint and t","cbCaitspScomgEbT","https://ap.wps.com/l/cbCaitspScomgEbT","pdf",918642,21,"English","# Abstract\n# Introduction\n## Document knowledge internalization setting\n## Limitations of SFT and continued pretraining\n## The proposed IAR framework\n# Figure 1 Overview\n# Contributions","[{\"question\":\"What problem does the document address?\",\"answer\":\"It studies retrieval-free question answering over a fixed document corpus, where the model cannot access the source documents at inference time.\"},{\"question\":\"How does IAR differ from conventional continued pretraining and SFT?\",\"answer\":\"IAR uses three stages: Inject converts documents into instruction-conditioned supervised reconstruction tasks, Align adapts the model with answer-only QA supervision, and Recover merges the domain-adapted model with the base instruction model to restore general capability.\"},{\"question\":\"What components does the paper suggest should be measured separately?\",\"answer\":\"It argues that document exposure, answer alignment, data recipe, token budget, and recovery should be evaluated independently rather than collapsed into a single fine-tuning comparison.\"}]","Inject, Align, Recover - Staged Post-Training for Retrieval-Free Document Knowledge Internalization | PDF",53]