[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-seo-273360-105":3,"detail-sidebar-cat-0-en-105":81,"doc-detail-273360-en":130},{"code":4,"msg":5,"data":6},0,"ok",{"site_id":7,"language":8,"slug":9,"title":10,"keywords":11,"description":12,"schema_data":13,"social_meta":74,"head_meta":76,"extra_data":78,"updated_unix":80},105,"en","when-to-ponder-adaptive-compute-allocation-for-code-generation-via-test-time-training","When to Ponder: Adaptive Compute Allocation for Code Generation via Test-Time Training","","Large language models typically use uniform computation for all inputs, which wastes effort on easy cases and can under-serve complex ones. This work introduces PonderTTT, a gating strategy that uses the TTT layer’s self-supervised reconstruction loss to decide when to trigger Test-Time Training updates. The gating is training-free: it relies on a single calibrated scalar threshold on unlabeled data and is continuously adapted with EMA to maintain target update rates. Experiments on GPT-2 code modeling show 82–89% oracle recovery without ground-truth labels and improved OOD behavior versus random skip baselines.",{"@graph":14,"@context":73},[15,34,56],{"@type":16,"itemListElement":17},"BreadcrumbList",[18,23,27,31],{"item":19,"name":20,"@type":21,"position":22},"https://docshare.wps.com","Home","ListItem",1,{"item":24,"name":25,"@type":21,"position":26},"https://docshare.wps.com/document/","Document",2,{"item":28,"name":29,"@type":21,"position":30},"https://docshare.wps.com/document/technology/","Technology",3,{"item":32,"name":10,"@type":21,"position":33},"https://docshare.wps.com/document/when-to-ponder-adaptive-compute-allocation-for-code-generation-via-test-time-training/273360/",4,{"url":32,"name":10,"@type":35,"image":36,"author":41,"headline":10,"publisher":44,"fileFormat":47,"inLanguage":8,"description":12,"dateModified":48,"datePublished":49,"encodingFormat":47,"isAccessibleForFree":50,"interactionStatistic":51},"DigitalDocument",{"url":37,"@type":38,"width":39,"height":40},"https://docshare.wps.com/thumbnails/when-to-ponder-adaptive-compute-allocation-for-code-generation-via-test-time-training/273360.png","ImageObject",300,407,{"name":42,"@type":43},"Logic","Person",{"url":19,"name":45,"@type":46},"DocShare","Organization","application/pdf","2026-10-17","2026-09-15",true,{"@type":52,"interactionType":53,"userInteractionCount":55},"InteractionCounter",{"@type":54},"ViewAction",7,{"@type":57,"mainEntity":58},"FAQPage",[59,65,69],{"name":60,"@type":61,"acceptedAnswer":62},"What does PonderTTT decide during inference?","Question",{"text":63,"@type":64},"PonderTTT decides when to trigger Test-Time Training updates by using the TTT layer’s self-supervised reconstruction loss as a signal.","Answer",{"name":66,"@type":61,"acceptedAnswer":67},"How is the gating strategy trained, and does it require labels?",{"text":68,"@type":64},"The Reconstruction Gating is training-free and does not require ground-truth labels; it uses a single scalar threshold calibrated on unlabeled data and then adapted with EMA.",{"name":70,"@type":61,"acceptedAnswer":71},"Why is uniform compute inefficient for code generation?",{"text":72,"@type":64},"Generating simple code (e.g., an import statement) needs less computation than executing more complex logic, so fixed-compute models either over-provision or under-provision depending on input difficulty.","https://schema.org",{"og:url":32,"og:type":75,"og:title":10,"og:site_name":45,"og:description":12},"article",{"robots":77,"canonical":32},"index,follow",{"doc_id":79,"site_id":7},273360,1790484783,{"code":4,"msg":82,"data":83},"success",[84,88,92,96,101,105,109,114,119,122,126],{"id":22,"doc_module":4,"doc_module_name":25,"category_name":85,"show_sort_weight":86,"slug":87},"Story & Novel",90,"story-novel",{"id":26,"doc_module":4,"doc_module_name":25,"category_name":89,"show_sort_weight":90,"slug":91},"Literature",80,"literature",{"id":33,"doc_module":4,"doc_module_name":25,"category_name":93,"show_sort_weight":94,"slug":95},"Exam",70,"exam",{"id":97,"doc_module":4,"doc_module_name":25,"category_name":98,"show_sort_weight":99,"slug":100},5,"Comic",60,"comic",{"id":102,"doc_module":4,"doc_module_name":25,"category_name":29,"show_sort_weight":103,"slug":104},6,50,"technology",{"id":55,"doc_module":4,"doc_module_name":25,"category_name":106,"show_sort_weight":107,"slug":108},"Healthcare",40,"healthcare",{"id":110,"doc_module":4,"doc_module_name":25,"category_name":111,"show_sort_weight":112,"slug":113},8,"Research & Report",30,"research-report",{"id":115,"doc_module":4,"doc_module_name":25,"category_name":116,"show_sort_weight":117,"slug":118},9,"Religion & Spirituality",20,"religion-spirituality",{"id":117,"doc_module":4,"doc_module_name":25,"category_name":120,"show_sort_weight":117,"slug":121},"World Cup","world-cup",{"id":123,"doc_module":4,"doc_module_name":25,"category_name":124,"show_sort_weight":123,"slug":125},10,"Lifestyle","lifestyle",{"id":127,"doc_module":4,"doc_module_name":25,"category_name":128,"show_sort_weight":97,"slug":129},19,"General","general",{"code":4,"msg":82,"data":131},{"doc_id":79,"user_id":132,"nickname":42,"user_avatar":133,"doc_module":4,"category_id":102,"category_name":29,"doc_title":10,"doc_description":12,"doc_content":134,"file_id":135,"file_url":136,"file_type":137,"file_size":138,"view_count":55,"is_deleted":4,"is_public":22,"is_downloadable":22,"audit_status":22,"page_count":139,"language":140,"language_code":8,"site_id":7,"html_lang":8,"table_of_contents":141,"faqs":142,"seo_title":143,"seo_description":12,"update_tm":144,"read_time":145},1099513958762,"https://ap-avatar.wpscdn.com/avatar/1000023916a998db790?x-image-process=image/resize,m_fixed,w_180,h_180&k=1784791008015729253","arXiv : 260 1 .00894v2 [cs .LG] 20 Aug 2026  \nWHEN TO PONDER: ADAPTIVE COMPUTE ALLOCATION FOR CODE GENERATION VIA TEST-TIME  \nTRAINING  \nA PREPRINT  \n Gihyeon Sim  \nDongpae High School, Paju, Gyeonggi-do, Republic of Korea  \n[world@worldsw.dev](world@worldsw.dev)  \nDecember 31, 2025  \nAbstract  \nLarge language models apply uniform computation to all inputs, regardless of difficulty. We propose PonderTTT, a gating strategy using the TTT layer’s selfsupervised reconstruction loss to selectively trigger Test-Time Training (TTT) updates. The gating decision itself is training-free—requiring no learned classifieror auxiliary networks; only a single scalar threshold is initially calibrated on unlabeled data and continuously adapted via EMA to maintain target update rates. Our experiments with GPT-2 models (124M to 1.5B) on code language modeling (The Stack v2, teacher-forced perplexity) demonstrate that this signal is inference-compatible, requiring no ground-truth labels. Our Reconstruction Gating achieves 82–89% Oracle Recovery while being fully training-free, significantly outperforming Random Skip baselines (up to 16% lower loss on OOD languages) .  \nKeywords Test-Time Training · Adaptive Computation · Language Models · Code Generation · Sample Efficiency · Dynamic Inference  \n1 Introduction  \nStandard Transformer models operate on a fixed computational graph: every token processes the same number of layers and attention heads. While effective, this rigidity creates inefficiency. Consider code generation: producing a standard import statement requires far less computation than implementing a dynamic programming algorithm. A fixed-compute model must either be over-provisioned for simple cases or under-provisioned for complex ones.  \nPrior approaches to adaptive computation, such as Mixture-of-Experts (MoE) or Early Exit strategies, focus on routing tokens or skipping layers but do not modify the model’s representations based on input context. Test-Time Training (TTT) offers an alternative: the model’s parameters are updated during inference to adapt to the current input. However, standard TTT applies updates uniformly (e.g., gradient descent on every token), reintroducing computational inefficiency.  \nWe propose PonderTTT, which focuses on finding the optimal “When to Update” signal. We define“Pondering” in this context as the deliberate, adaptive allocation of the update budget—deciding whether to learn from the current context rather than how long to think. Unlike PonderNet’s geometric halting mechanism for layer-wise computation, our approach controls parameter updates at inference time. We empirically observe that the TTT layer’s self-supervised reconstruction loss provides a training-free Reconstruction Gating strategy that works efficiently and robustly.  \n• We demonstrate that TTT Reconstruction Loss is an inference-compatible proxy for learning potential. This self-supervised signal is available during inference without groundtruth labels.  \n• We introduce “Reconstruction Gating,” a threshold-based gating strategy with EMA adjustment, and analyze its effectiveness across model scales.  \n• We provide empirical analysis showing that Reconstruction Gating achieves 82–89% Oracle Recovery while being fully training-free, significantly outperforming Random Skip baselines.  \n2 Related Work  \nAdaptive Computation. Efforts to move beyond the fixed-compute paradigm include Universal Transformers [2], which loop over layers dynamically, and Early Exit models [8], which produce predictions at intermediate layers. PonderNet [1] introduced a probabilistic halting mechanism trained via variational inference. Unlike these architectural modifications, our work focuses on adapting the parameters of the model (fast weights) dynamically.  \nTest-Time Training (TTT). TTT [9] was originally proposed for generalization in vision tasks. Recently, TTT-LM [10] adapted this to language modeling by augmenting the transformer architecture with a self-","cbCaiiwSSwmfDPAW","https://ap.wps.com/l/cbCaiiwSSwmfDPAW","pdf",354780,14,"English","# Introduction\n# Related Work\n# Method\n## Preliminaries: TTT-Linear Update","[{\"question\":\"What does PonderTTT decide during inference?\",\"answer\":\"PonderTTT decides when to trigger Test-Time Training updates by using the TTT layer’s self-supervised reconstruction loss as a signal.\"},{\"question\":\"How is the gating strategy trained, and does it require labels?\",\"answer\":\"The Reconstruction Gating is training-free and does not require ground-truth labels; it uses a single scalar threshold calibrated on unlabeled data and then adapted with EMA.\"},{\"question\":\"Why is uniform compute inefficient for code generation?\",\"answer\":\"Generating simple code (e.g., an import statement) needs less computation than executing more complex logic, so fixed-compute models either over-provision or under-provision depending on input difficulty.\"}]","When to Ponder: Adaptive Compute Allocation for Code Generation via Test-Time Training | PDF",1789471059,35]