[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-85679-en":3,"doc-seo-85679-105":29,"detail-sidebar-cat-0-en-105":90},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":13,"seo_description":14,"update_tm":27,"read_time":28},85679,549758252649,"Ivy","https://ap-avatar.wpscdn.com/avatar/8000253669c5317157?_k=1778319167496531819",8,"Research & Report","Mitigating Early Training Collapse in CTR Models","Deep neural networks for click-through rate (CTR) prediction can show a sudden drop in validation performance right after the first training epoch, even as training loss continues to decrease. This study examines the phenomenon on large-scale industrial datasets and tests practical mitigation methods. Learning-rate reduction offers only incremental benefits, while controlling feature sparsity brings substantial gains. Removing highly sparse features and aggregating rare feature values stabilizes training, extends effective learning beyond one epoch, and improves offline AUC/log-loss/PRAUC plus online system metrics.","arXiv :2607 .09696v1 [ cs .LG] 20 Jun 2026  \nMitigating Early Training Collapse in CTR Models  \nErgun Bic¸ici 1 , Erkan C¸ etinyamac¸ 1  \n1 Intelligent Application Development, Huawei T¨urkiye R&D Center, Istanbul, Turkey  \n*Corresponding author: Ergun Bic¸ici, Huawei T¨urkiye R&D Center, Istanbul, Turkey. Email: [ergun.bicici@huawei.com](ergun.bicici@huawei.com)  \nAbstract  \nDeep neural models for click-through rate prediction often exhibit a sharp decline in validation performance immediately after the first training epoch despite continued improvement in training loss. This instability restricts effective learning and limits model performance. In this study, we analyze this behavior using large-scale industrial datasets and evaluate practical mitigation strategies. While reducing the learning rate provides only incremental gains, controlling feature sparsity yields substantial improvements. Removing highly sparse features and aggregating infrequent feature values stabilizes training, extends useful learning beyond a single epoch, and improves both offline evaluation metrics and online system performance.  \nKeywords  \nCTR prediction, overfitting, sparsity, embeddings, recommender systems  \nAbbreviations  \nCTR: Click-Through Rate; CVR: Conversion Rate; AUC: Area Under Curve; PRAUC: PrecisionRecall AUC  \n1 Introduction  \nDeep learning models are widely used in CTR prediction due to their ability to capture complex feature interactions [4, 1, 3] . However, in industrial environments with high-cardinality categorical inputs, models frequently reach peak validation performance after a single epoch and degrade thereafter. This effect has been observed in prior work and is associated with rapid overfitting in sparse feature spaces [5] .  \nCTR datasets exhibit long-tailed distributions where a small subset of feature values dominates frequency while most occur rarely [6] . This imbalance increases variance and encourages rapid memorization. Unlike vision tasks where overfitting progresses gradually [7], CTR models often degrade abruptly. Figure 1 depicts one-epoch phenomenon within a U-shaped training regime [9] .  \nPerformance (AUC)  \n1  \n0.8  \n0.6  \n\n|  | One-Ep | och Overfitting |  |  |  |  |  |  |  | training\u003Cbr>validation |  |  |\n| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |\n|  |  |  |  |  |  |  |  |  |  |  |  |  |\n|  |  |  |  |  |  |  |  |  |  |  |  |  |\n|  |  |  |  |  |  |  |  |  |  |  |  |  |\n|  |  |  |  |  |  |  |  |  |  |  |  |  |\n|  |  |  |  |  |  |  |  |  |  |  |  |  |\n|  |  |  |  |  |  |  |  |  |  |  |  |  |\n\n1 1.5 2 2.5 3 3.5 4 4.5 5  \nEpoch  \nFigure 1: Illustration of the one-epoch phenomenon within a U-shaped training regime. Training performance continues to improve while validation performance peaks after the first epoch due to overfitting.  \n2 Materials and Methods  \n2.1 Problem Mechanism  \nThis behavior arises from the interaction of model capacity, optimization dynamics, and data sparsity. Embedding layers assign parameters to each categorical value, but rare values receive very few updates, leading to unstable representations. Adaptive optimizers accelerate convergence [8], allowing the model to quickly fit noise. As a result, the model memorizes infrequent patterns early, causing validation performance to deteriorate.  \n2.2 Proposed Strategies  \nWe evaluate three approaches:  \n• Learning rate reduction: Slows convergence but does not eliminate early overfitting.  \n• Sparse feature removal: Eliminates high-cardinality features with low signal.  \n• Value filtering: Retains only frequent values, mapping others to a shared token.  \n3 Results and Discussion  \nExperiments use industrial-scale CTR datasets with hundreds of millions of samples and strong class imbalance. Performance is evaluated using AUC, log-loss, and PRAUC.  \n3.1 Offline Results  \nThe results in Table 1 show that reducing the learning rate yields small improvements but does not prevent early degradation. Removing s","cbCaigaYxHEGiuhn","https://ap.wps.com/l/cbCaigaYxHEGiuhn","pdf",141526,1,4,"English","en",105,"# Introduction\n# Materials and Methods\n## Problem Mechanism\n## Proposed Strategies\n# Results and Discussion\n## Offline Results\n## Online Results\n# Conclusion","[{\"question\":\"What is the early training collapse problem in CTR models?\",\"answer\":\"CTR models may experience validation performance dropping immediately after the first epoch while training loss keeps improving. This instability limits effective learning and overall model quality.\"},{\"question\":\"Why does feature sparsity lead to rapid degradation?\",\"answer\":\"Sparse, high-cardinality categorical features create embedding parameters that receive few updates. Adaptive optimization helps the model fit quickly, encouraging memorization of infrequent patterns and causing validation to deteriorate.\"},{\"question\":\"Which mitigation strategies show the best results?\",\"answer\":\"Reducing learning rate gives only small improvements. The strongest gains come from sparse feature removal and value filtering, which stabilize training, improve offline metrics, and translate to online advertising gains.\"}]",1784205555,10,{"code":4,"msg":30,"data":31},"ok",{"site_id":24,"language":23,"slug":32,"title":13,"keywords":33,"description":14,"schema_data":34,"social_meta":85,"head_meta":87,"extra_data":89,"updated_unix":27},"mitigating-early-training-collapse-in-ctr-models","",{"@graph":35,"@context":84},[36,52,67],{"@type":37,"itemListElement":38},"BreadcrumbList",[39,43,47,50],{"item":40,"name":41,"@type":42,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":44,"name":45,"@type":42,"position":46},"https://docshare.wps.com/document/","Document",2,{"item":48,"name":12,"@type":42,"position":49},"https://docshare.wps.com/document/research-report/",3,{"item":51,"name":13,"@type":42,"position":21},"https://docshare.wps.com/document/mitigating-early-training-collapse-in-ctr-models/85679/",{"url":51,"name":13,"@type":53,"author":54,"headline":13,"publisher":56,"fileFormat":59,"inLanguage":23,"description":14,"dateModified":60,"datePublished":61,"encodingFormat":59,"isAccessibleForFree":62,"interactionStatistic":63},"DigitalDocument",{"name":9,"@type":55},"Person",{"url":40,"name":57,"@type":58},"DocShare","Organization","application/pdf","2026-07-17","2026-07-16",true,{"@type":64,"interactionType":65,"userInteractionCount":20},"InteractionCounter",{"@type":66},"ViewAction",{"@type":68,"mainEntity":69},"FAQPage",[70,76,80],{"name":71,"@type":72,"acceptedAnswer":73},"What is the early training collapse problem in CTR models?","Question",{"text":74,"@type":75},"CTR models may experience validation performance dropping immediately after the first epoch while training loss keeps improving. This instability limits effective learning and overall model quality.","Answer",{"name":77,"@type":72,"acceptedAnswer":78},"Why does feature sparsity lead to rapid degradation?",{"text":79,"@type":75},"Sparse, high-cardinality categorical features create embedding parameters that receive few updates. Adaptive optimization helps the model fit quickly, encouraging memorization of infrequent patterns and causing validation to deteriorate.",{"name":81,"@type":72,"acceptedAnswer":82},"Which mitigation strategies show the best results?",{"text":83,"@type":75},"Reducing learning rate gives only small improvements. The strongest gains come from sparse feature removal and value filtering, which stabilize training, improve offline metrics, and translate to online advertising gains.","https://schema.org",{"og:url":51,"og:type":86,"og:title":13,"og:site_name":57,"og:description":14},"article",{"robots":88,"canonical":51},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":91},[92,96,100,104,109,114,119,122,127,130,133],{"id":20,"doc_module":4,"doc_module_name":45,"category_name":93,"show_sort_weight":94,"slug":95},"Story & Novel",90,"story-novel",{"id":46,"doc_module":4,"doc_module_name":45,"category_name":97,"show_sort_weight":98,"slug":99},"Literature",80,"literature",{"id":21,"doc_module":4,"doc_module_name":45,"category_name":101,"show_sort_weight":102,"slug":103},"Exam",70,"exam",{"id":105,"doc_module":4,"doc_module_name":45,"category_name":106,"show_sort_weight":107,"slug":108},5,"Comic",60,"comic",{"id":110,"doc_module":4,"doc_module_name":45,"category_name":111,"show_sort_weight":112,"slug":113},6,"Technology",50,"technology",{"id":115,"doc_module":4,"doc_module_name":45,"category_name":116,"show_sort_weight":117,"slug":118},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":45,"category_name":12,"show_sort_weight":120,"slug":121},30,"research-report",{"id":123,"doc_module":4,"doc_module_name":45,"category_name":124,"show_sort_weight":125,"slug":126},9,"Religion & Spirituality",20,"religion-spirituality",{"id":125,"doc_module":4,"doc_module_name":45,"category_name":128,"show_sort_weight":125,"slug":129},"World Cup","world-cup",{"id":28,"doc_module":4,"doc_module_name":45,"category_name":131,"show_sort_weight":28,"slug":132},"Lifestyle","lifestyle",{"id":134,"doc_module":4,"doc_module_name":45,"category_name":135,"show_sort_weight":105,"slug":136},19,"General","general"]