[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-82340-en":3,"doc-seo-82340-105":29,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":13,"seo_description":14,"update_tm":27,"read_time":28},82340,1374391974585,"Genevieve","https://ap-avatar.wpscdn.com/davatar_276721f389ce27ea32af1340a28f341c",8,"Research & Report","Self-Guided Test-Time Training for Long-Context LLMs","Long-context processing is essential for large language models, yet extending the context window alone often fails to improve accuracy because relevant evidence is difficult to locate and use. Test-time training (TTT) adapts model parameters using the test context, but applying it to the full context is costly and adapting on randomly sampled spans injects substantial noise. This work shows TTT is highly sensitive to training-span quality: random spans can hurt, while oracle, evidence-aware spans improve. It proposes Self-Guided TTT (S-TTT), which selects likely evidence spans before adaptation and improves results on LongBench-v2 and LongBench-Pro.","arXiv :2607 .094 15v 1 [ cs .CL] 10 Jul 2026  \nSelf-Guided Test-Time Training for Long-Context LLMs  \nXinyu Zhu 1 ,2 , Zhe Xu 1 ,†, Xiaohan Wei 1 , Yunchen Pu 1 , Fei Tian 1 , Chonglin Sun 1 , Kaushik Rangadurai 1 , Hua Zhi 1 , Frank Shyu 1 , Sandeep Pandey 1 , Luke Simon 1 , Yu Meng2 ,‡, Xi Liu 1 ,‡  \n1 Meta AI, 2 University of Virginia  \n†Execution Lead, ‡Joint corresponding author  \nLong-context processing has become increasingly important for large language models (LLMs), but simply extending the context window does not guarantee effective utilization of long inputs. As input length grows, accuracy often degrades, indicating that models still struggle to identify and use the evidence most relevant to a question. A promising way to improve long-context utilization is test-time training (TTT), which treats the test context as a training example for instance-specific parameter adaptation. However, applying TTT to the entire long context is prohibitively expensive, while adapting on randomly sampled spans introduces severe noise. Because most spans in a long context are irrelevant to the specific question, training on them may even degrade the base model’s performance. Our preliminary study shows that TTT is highly sensitive to training-span quality: on LongBench-v2, TTT on randomly sampled spans hurts performance, whereas TTT on oracle spans substantially improves it. Motivated by this, we propose a simple method, Self-Guided TTT (S-TTT): before adaptation, the model identifies the evidence spans it should learn from, and the standard language-modeling training objective is applied only to those selected spans. On two challenging longcontext reasoning benchmarks, LongBench-v2 and LongBench-Pro, S-TTT improves accuracy for both Qwen3-4B-Thinking-2507 and Llama-3 . 1-8B-Instruct, achieving up to a 15% relative improvement.  \nCorrespondence: Yu Meng ([yumeng5@virginia.edu](yumeng5@virginia.edu)) and Xi Liu ([xliu1@meta.com](xliu1@meta.com))  \nDate: July 10, 2026  \n1 Introduction  \nLong-context capability has become a central requirement for modern language models. Recent models support context windows of hundreds of thousands of tokens, enabling them to process long inputs in a single prompt (Peng et al. , 2024 ; Chen et al. , 2024) . Despite this progress, a larger window does not by itself ensure that the model can use long inputs effectively. As context length grows, accuracy often degrades, and models struggle to keep the most relevant evidence accessible throughout reasoning and decoding (Liu et al. , 2024 ; Hsieh et al. , 2024) . This suggests that the bottleneck in long-context reasoning is not merely fitting more tokens into the prompt, but ensuring that the model can identify and use the evidence relevant to the question. Test-time training (TTT) (Sun et al. , 2020 ; Liu et al. , 2021 ; Hardt and Sun, 2024 ; Akyürek et al. , 2024 ; Tandon et al. , 2025 ; Zhang et al. , 2025a ; Feng et al. , 2026) has emerged as a promising solution. Instead of answering with a fixed model, TTT treats the test input itself as a training example, adapts the model weights for that specific instance, and uses the adapted weights to generate the answer. For long-context tasks, this is especially appealing because adaptation can turn instance-specific evidence in the context into parameter updates, making it easier to use during subsequent generation (Bansal et al. , 2026 ; Chen et al. , 2026a) .  \nHowever, a key challenge in applying TTT to long contexts is determining what data to train on—an important dimension that remains largely underexplored. Existing approaches commonly rely on either full-context adaptation (Tandon et al. , 2025 ; Zhang et al. , 2025a) or randomly sampled training spans(Bansal et al. , 2026), both of which suffer from noisy signals. Not only is performing TTT on the full context computationally expensive, but it also overwhelms the adaptation process with distractors, as the vast majority of a long context is usually","cbCailYkEztJTOFJ","https://ap.wps.com/l/cbCailYkEztJTOFJ","pdf",1259095,1,15,"English","en",105,"# Introduction\n## Long-context capability and the evidence bottleneck\n## Test-time training and the data-quality challenge\n## Self-Guided TTT (S-TTT) approach","[{\"question\":\"Why does longer context not automatically improve long-context LLM performance?\",\"answer\":\"As input length grows, accuracy often degrades because the model struggles to identify and keep the most relevant evidence accessible for the question and reasoning process.\"},{\"question\":\"What problem arises when applying test-time training (TTT) to long contexts?\",\"answer\":\"Full-context adaptation is prohibitively expensive, while adapting on randomly sampled spans introduces severe noise because most spans are irrelevant and relevant evidence may be missed.\"},{\"question\":\"How does Self-Guided TTT (S-TTT) improve over standard TTT?\",\"answer\":\"S-TTT uses the model to select evidence spans likely to support the question, then applies the standard next-token-prediction objective only on those selected spans for adaptation while generating the final answer from the full context.\"}]",1784179755,38,{"code":4,"msg":30,"data":31},"ok",{"site_id":24,"language":23,"slug":32,"title":13,"keywords":33,"description":14,"schema_data":34,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":27},"self-guided-test-time-training-for-long-context-llms","",{"@graph":35,"@context":85},[36,53,68],{"@type":37,"itemListElement":38},"BreadcrumbList",[39,43,47,50],{"item":40,"name":41,"@type":42,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":44,"name":45,"@type":42,"position":46},"https://docshare.wps.com/document/","Document",2,{"item":48,"name":12,"@type":42,"position":49},"https://docshare.wps.com/document/research-report/",3,{"item":51,"name":13,"@type":42,"position":52},"https://docshare.wps.com/document/self-guided-test-time-training-for-long-context-llms/82340/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":23,"description":14,"dateModified":61,"datePublished":62,"encodingFormat":60,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":40,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-17","2026-07-16",true,{"@type":65,"interactionType":66,"userInteractionCount":20},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"Why does longer context not automatically improve long-context LLM performance?","Question",{"text":75,"@type":76},"As input length grows, accuracy often degrades because the model struggles to identify and keep the most relevant evidence accessible for the question and reasoning process.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"What problem arises when applying test-time training (TTT) to long contexts?",{"text":80,"@type":76},"Full-context adaptation is prohibitively expensive, while adapting on randomly sampled spans introduces severe noise because most spans are irrelevant and relevant evidence may be missed.",{"name":82,"@type":73,"acceptedAnswer":83},"How does Self-Guided TTT (S-TTT) improve over standard TTT?",{"text":84,"@type":76},"S-TTT uses the model to select evidence spans likely to support the question, then applies the standard next-token-prediction objective only on those selected spans for adaptation while generating the final answer from the full context.","https://schema.org",{"og:url":51,"og:type":87,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":89,"canonical":51},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":20,"doc_module":4,"doc_module_name":45,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":46,"doc_module":4,"doc_module_name":45,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":45,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":45,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":45,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":45,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":45,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":45,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":45,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":45,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":45,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]