[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-85554-en":3,"doc-seo-85554-105":29,"detail-sidebar-cat-0-en-105":82},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":4,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":13,"seo_description":14,"update_tm":27,"read_time":28},85554,16904993612988,"Olivia Brown","https://ap-avatar.wpscdn.com/davatar_a8503ba1806abce46bf441b54a3ca4cd",8,"Research & Report","Neural Collapse Dynamics: Depth, Activation, Regularisation, and Feature Norm Thresholds","Neural collapse (NC) describes how penultimate-layer features converge to a simplex equiangular tight frame, with equilibrium behaviour well understood but onset timing largely unexplored. This study identifies a single measurable control: the mean penultimate feature norm fn reaching a pair-specific critical threshold fn*. Across settings, fn* concentrates tightly and is mostly invariant, while training hyperparameters mainly affect the approach rate and activation shifts the threshold itself.","arXiv :2604 .00230v 3 [ cs .LG] 10 Jul 2026  \nNeural Collapse Dynamics: Depth, Activation, Regularisation,  \nand Feature Norm Thresholds  \nAnamika Paul Rupa∗  \nDepartment of Electrical Engineering and Computer Science, Howard University, Washington,  \nDC, 20059, USA  \nAbstract  \nNeural collapse (NC), the convergence of penultimate-layer features to a simplex equiangular tight frame, is well characterised at equilibrium, but what governs when and how fast it emerges has received little systematic study. We present a controlled study of NC onset, showing it is marked by a single measurable quantity: the mean penultimate feature norm (fn) reaching a critical threshold fn ∗ specific to each (model, dataset) pair. Three findings support this. First, fn∗ concentrates tightly within each pair (coefficient of variation below 8% at a fixed collapse criterion) and is largely invariant to depth, width, and weight decay, which set only the rate at which fn approaches it, whereas activation shifts the threshold itself. Second, a direct intervention establishes fn ∗ as an attractor of the gradient flow, not a passive correlate: after the feature norm is rescaled over a tenfold range, it relaxes to the same value. Third, a six-cell (architecture) × (dataset) grid reveals a non-additive interaction: the architecture effect on fn ∗ ranges from +22 .7% to +458% depending on the dataset (confirmed by a matched-optimiser control), so fn ∗ must be measured per pair, not extrapolated. A simplified unconstrained-features calculation rationalises the magnitude offn∗ and its insensitivity to weight decay. The temporal ordering is modest: fn crosses fn ∗ just before collapse but co-evolves with it, so fn ∗ is a stable landmark of the collapsed state, not an early-warning signal. Together, these results establish the feature-norm threshold asa reproducible, largely training-invariant characterisation of neural collapse onset.  \nKeywords: Neural collapse, Training dynamics, Feature norm, Deep learning, Weight decay, Implicit bias  \n∗ Corresponding author.  \nEmail address: [anamikal.rupa@howard.edu](anamikal.rupa@howard.edu) (Anamika Paul Rupa)  \nGraphical abstract  \nNeural collapse onset coincides with a feature-norm threshold f  \nfeature norm fn ( log )  \n101  \n100  \n(A) Feature norm falls to a pair-specific f  \nP1 P2  \ntraining conditions Neural  \nset the timing collapse  \nthreshold f⋆ ≈ 1.06  \nn  \n0 50 100 150 200 250 300  \ntraining epoch  \nfeature norm fn ( log )  \n101  \n100  \n(B) Rescaled norms self-correct to f  \n\n|  |  |\n| --- | --- |\n|  |  |\n\n200 300 400 500 training epoch  \n✔ Training factors (depth, width, weight decay) set when collapse occurs, not f⋆n  \n✔ Each (model, dataset) pair has a stable f⋆n (within-pair CV \u003C 8%)  \n✔ Rescaled feature norms return to the same f⋆n: a gradient-flow attractor  \nScope: MLPs & ResNets ⋅ MNIST / Fashion-MNIST / CIFAR-10 ⋅ robust across seeds and optimizers (SGD, Adam)  \n1. Introduction  \nNeural collapse (NC), first documented by Papyan et al. [1], is a striking geometric regularity that emerges at the end of deep network training: penultimate-layer features converge to a simplex equiangular tight frame (ETF), within-class variability vanishes (NC1), and classifier weights align with class means (NC3) . The endpoint of NC is now well characterised theoretically [2, 3, 4, 5], yet a fundamental question has received almost no systematic attention: what controls when and how fast NC emerges as a function of architecture and training hyperparameters? We answer this empirically: across architectures, datasets, and hyperparameters, NC onset is consistently marked by the mean feature norm reaching a model-dataset-specific threshold fn∗ that is largely invariant to training dynamics, is reached just before collapse, and behaves as an attractor of the gradient flow, reducing a qualitative dynamics question to a single trackable quantity.  \nUnderstanding collapse timing matters scientifically, by clarifying how represen","cbCaivLYmw0Wcq1P","https://ap.wps.com/l/cbCaivLYmw0Wcq1P","pdf",916236,1,31,"English","en",105,"# Introduction\n## Neural collapse and the onset-time question\n## Experimental design and hypotheses\n# Core findings and decomposition\n## Feature-norm threshold behaviour\n## Attractor interpretation via rescaling\n## Architecture–dataset interaction","[{\"question\":\"Does the feature-norm threshold act as a passive indicator or an active dynamical target?\",\"answer\":\"It acts as an attractor of the gradient flow: after rescaling the feature norm over a tenfold range, it relaxes back to the same fn* value.\"}]",1784204503,78,{"code":4,"msg":30,"data":31},"ok",{"site_id":24,"language":23,"slug":32,"title":13,"keywords":33,"description":14,"schema_data":34,"social_meta":77,"head_meta":79,"extra_data":81,"updated_unix":27},"neural-collapse-dynamics-depth-activation-regularisation-and-feature-norm-thresholds","",{"@graph":35,"@context":76},[36,53,67],{"@type":37,"itemListElement":38},"BreadcrumbList",[39,43,47,50],{"item":40,"name":41,"@type":42,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":44,"name":45,"@type":42,"position":46},"https://docshare.wps.com/document/","Document",2,{"item":48,"name":12,"@type":42,"position":49},"https://docshare.wps.com/document/research-report/",3,{"item":51,"name":13,"@type":42,"position":52},"https://docshare.wps.com/document/neural-collapse-dynamics-depth-activation-regularisation-and-feature-norm-thresholds/85554/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":23,"description":14,"dateModified":61,"datePublished":61,"encodingFormat":60,"isAccessibleForFree":62,"interactionStatistic":63},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":40,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-16",true,{"@type":64,"interactionType":65,"userInteractionCount":4},"InteractionCounter",{"@type":66},"ViewAction",{"@type":68,"mainEntity":69},"FAQPage",[70],{"name":71,"@type":72,"acceptedAnswer":73},"Does the feature-norm threshold act as a passive indicator or an active dynamical target?","Question",{"text":74,"@type":75},"It acts as an attractor of the gradient flow: after rescaling the feature norm over a tenfold range, it relaxes back to the same fn* value.","Answer","https://schema.org",{"og:url":51,"og:type":78,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":80,"canonical":51},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":83},[84,88,92,96,101,106,111,114,119,122,126],{"id":20,"doc_module":4,"doc_module_name":45,"category_name":85,"show_sort_weight":86,"slug":87},"Story & Novel",90,"story-novel",{"id":46,"doc_module":4,"doc_module_name":45,"category_name":89,"show_sort_weight":90,"slug":91},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":45,"category_name":93,"show_sort_weight":94,"slug":95},"Exam",70,"exam",{"id":97,"doc_module":4,"doc_module_name":45,"category_name":98,"show_sort_weight":99,"slug":100},5,"Comic",60,"comic",{"id":102,"doc_module":4,"doc_module_name":45,"category_name":103,"show_sort_weight":104,"slug":105},6,"Technology",50,"technology",{"id":107,"doc_module":4,"doc_module_name":45,"category_name":108,"show_sort_weight":109,"slug":110},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":45,"category_name":12,"show_sort_weight":112,"slug":113},30,"research-report",{"id":115,"doc_module":4,"doc_module_name":45,"category_name":116,"show_sort_weight":117,"slug":118},9,"Religion & Spirituality",20,"religion-spirituality",{"id":117,"doc_module":4,"doc_module_name":45,"category_name":120,"show_sort_weight":117,"slug":121},"World Cup","world-cup",{"id":123,"doc_module":4,"doc_module_name":45,"category_name":124,"show_sort_weight":123,"slug":125},10,"Lifestyle","lifestyle",{"id":127,"doc_module":4,"doc_module_name":45,"category_name":128,"show_sort_weight":97,"slug":129},19,"General","general"]