[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-82903-en":3,"doc-seo-82903-105":30,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":21,"is_downloadable":21,"audit_status":21,"page_count":22,"language":23,"language_code":24,"site_id":25,"html_lang":24,"table_of_contents":26,"faqs":27,"seo_title":13,"seo_description":14,"update_tm":28,"read_time":29},82903,8796095461564,"Liam","https://ap-avatar.wpscdn.com/davatar_155a257f0dc6eb9ab79c44ca47cae57d",8,"Research & Report","Grokking Is Conditional and Fragile A Fully Tractable Multi Seed Study at 12K Parameters","Grokking, the delayed emergence of generalization after a network fits its training set, is studied in a fully tractable ≈11,856-parameter Llama-style transformer (Glimmer-1-Base) trained on modular arithmetic. Grokking is measured as a multi-seed rate rather than a single-run outcome. Findings show a conditional transition gated by training-set coverage, reproducible effects of weight decay, and fragility to floating-point execution details, with several single-seed narratives overturned by seed confounds.","arXiv :2607 .05 104v 1 [ cs .LG] 6 Jul 2026  \nGrokking Is Conditional and Fragile: A Fully-Tractable, Multi-Seed Study at 12K Parameters  \nYoshiyuki Ootani [info@ootanl. com](info@ootanl. com)  \nIndependent Researcher  \nAbstract  \nGrokking, the delayed onset of generalization long after a network has fit its training set, is usually studied in models large enough that the learned algorithm is only partially legible and where a single training run is taken as evidence. We instead study an ≈11,856-parameter Llama-style transformer (Glimmer-1-Base) on modular arithmetic, a regime small enough to enumerate and read its weights, attention, and full input-output map directly, and we measure grokking as a multi-seed rate rather than a single outcome. Six findings emerge.  \n(i) The grokking transition is gated by training-set coverage, and the coverage threshold tracks the output cardinality (the modulus) more than composition structure, an ordering that holds above the transition and across a ten-fold change in domain size. (ii) Weight decay reproduces the Omnigrok inverted-U at 12K parameters, grok-rate rising from 20% to 90% then collapsing to 0%, a positive control on the rate measurement. (iii) Grokkingsits on a numerical knife-edge: two distinct perturbations of the floating-point environment, the CPU thread count (a pure change of reduction order) and CPU-versus-GPU execution (a device change that subsumes it), each flip a minority of seeds (49/300 and 19/100 atthe 0.70 threshold) without a detectable shift in the aggregate rate (well powered: exact McNemar p ≥ 0.39; paired-difference 95% confidence intervals within ±10 points) . (iv) The mechanistic read-out is mixed: generalizing solutions have a more periodic output map (a partly definitional consistency check), while the genuinely independent result is a negative one, namely that the dim-16 model does not form the textbook Fourier embedding circle. (v) Task decomposition helps not by enabling an otherwise impossible computation but by converting a coverage-starved sparse task into densely coverable sub-tasks; at a matched data budget a two-specialist pipeline groks where the monolith cannot (10/10 versus 0/10), while a same-budget scratchpad monolith carrying the identical decomposed supervision still fails (0/10), isolating coverage rather than supervision density as the driver.  \n(vi) Methodologically, multi-seed control with a fixed numerical environment overturns three dramatic single-run narratives in our own data, a hard task “wall”, a “thread count flips grokking” effect, and a “GPU suppresses grokking” effect, each of which proves to be a seed confound. In this fully-tractable regime, grokking is best understood as a conditional and fragile phase transition, and multi-seed numerical control is a precondition for any claim about it.  \n1 Introduction  \nA neural network can fit a small algorithmic dataset to perfect training accuracy and continue, for thousands of further steps, to generalize no better than chance, and then, abruptly, generalize almost perfectly. This delayed generalization, grokking (Power et al., 2022), has become a central object of study for the science of deep learning because it isolates the question of when a network converts memorization into an algorithm. Most of what we know about grokking comes either from mechanistic case studies of a single trained network (Nanda et al., 2023) or from phenomenological accounts in models with millions of parameters (Liu et al. , 2023; Prieto et al., 2025) . Both settings share two limitations. The learned computation is only partially legible, so claims about mechanism rest on probing rather than on a complete reading of the weights; and grokking is reported from individual runs, even though it is known to be seed-sensitive.  \nWe take the opposite tack on both counts. We study Glimmer-1-Base (CompactAI, 2026), a published ≈11,856-parameter Llama-style transformer (hidden size 16, two layers, four attention ","cbCaiiGBDfXiqRYz","https://ap.wps.com/l/cbCaiiGBDfXiqRYz","pdf",514471,2,1,14,"English","en",105,"# Abstract\n# Introduction\n# Contributions","[{\"question\":\"How does this study measure grokking compared with prior work?\",\"answer\":\"Instead of relying on a single training run, it measures grokking as a multi-seed rate, using many seeds under a fixed numerical environment.\"},{\"question\":\"What primarily gates the grokking transition?\",\"answer\":\"The grokking transition is gated by training-set coverage, and the coverage threshold tracks the output cardinality (the modulus) more than task composition structure.\"},{\"question\":\"What does the paper conclude about fragility and seed confounds?\",\"answer\":\"Small numerical perturbations (e.g., CPU thread count or CPU vs GPU execution) can flip outcomes for subsets of seeds without shifting aggregate rates, and multi-seed control overturns prior single-run narratives, showing they were seed confounds.\"}]",1784183832,35,{"code":4,"msg":31,"data":32},"ok",{"site_id":25,"language":24,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":28},"grokking-is-conditional-and-fragile-a-fully-tractable-multi-seed-study-at-12k-parameters","",{"@graph":36,"@context":85},[37,53,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,47,50],{"item":41,"name":42,"@type":43,"position":21},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":20},"https://docshare.wps.com/document/","Document",{"item":48,"name":12,"@type":43,"position":49},"https://docshare.wps.com/document/research-report/",3,{"item":51,"name":13,"@type":43,"position":52},"https://docshare.wps.com/document/grokking-is-conditional-and-fragile-a-fully-tractable-multi-seed-study-at-12k-parameters/82903/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":24,"description":14,"dateModified":61,"datePublished":62,"encodingFormat":60,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":41,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-22","2026-07-16",true,{"@type":65,"interactionType":66,"userInteractionCount":20},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"How does this study measure grokking compared with prior work?","Question",{"text":75,"@type":76},"Instead of relying on a single training run, it measures grokking as a multi-seed rate, using many seeds under a fixed numerical environment.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"What primarily gates the grokking transition?",{"text":80,"@type":76},"The grokking transition is gated by training-set coverage, and the coverage threshold tracks the output cardinality (the modulus) more than task composition structure.",{"name":82,"@type":73,"acceptedAnswer":83},"What does the paper conclude about fragility and seed confounds?",{"text":84,"@type":76},"Small numerical perturbations (e.g., CPU thread count or CPU vs GPU execution) can flip outcomes for subsets of seeds without shifting aggregate rates, and multi-seed control overturns prior single-run narratives, showing they were seed confounds.","https://schema.org",{"og:url":51,"og:type":87,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":89,"canonical":51},"index,follow",{"doc_id":7,"site_id":25},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":21,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":20,"doc_module":4,"doc_module_name":46,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":46,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":46,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":46,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":46,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":46,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":46,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]