[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-81552-en":3,"doc-seo-81552-105":30,"detail-sidebar-cat-0-en-105":83},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":21,"is_downloadable":21,"audit_status":21,"page_count":22,"language":23,"language_code":24,"site_id":25,"html_lang":24,"table_of_contents":26,"faqs":27,"seo_title":13,"seo_description":14,"update_tm":28,"read_time":29},81552,549758146520,"Patrick","https://ap-avatar.wpscdn.com/avatar/80002397d8c0411e94?_k=1775819394049821470",8,"Research & Report","Contrastive Weak-to-strong Generalization","Weak-to-strong generalization aims to scale large language models by training a stronger model on samples produced by an aligned weaker model, avoiding human feedback and explicit reward modeling. Its effectiveness is limited by noise and bias in weak-model outputs. The work introduces Contrastive Weak-to-Strong Generalization (ConG), using implicit rewards via log-likelihood ratios and showing a structural equivalence with Contrastive Decoding. ConG applies contrastive decoding between pre-/post-alignment weak models to generate higher-quality, denoised samples, yielding robust capability transfer with consistent empirical gains.","Contrastive Weak-to-strong Generalization  \nHoucheng Jiang 1 2 Junfeng Fang 3 Jiaxin Wu 1 Tianyu Zhang 1 Chen Gao 4 2 Xiang Wang 1 * Xiangnan He 1 * Yang Deng 5  \narXiv :2510 .07884v2 [ cs .CL] 10 Jul 2026  \nAbstract  \nWeak-to-strong generalization provides a promising paradigm for scaling large language models (LLMs) by training stronger models on samples from aligned weaker ones, without requiring human feedback or explicit reward modeling. However, its robustness and generalization are hindered by the noise and biases in weakmodel outputs, which limit its applicability in practice. To address this challenge, we leverage implicit rewards, which approximate explicit rewards through log-likelihood ratios, and reveal their structural equivalence with Contrastive Decoding (CD), a decoding strategy shown to reduce noise in LLM generation. Building on this connection, we propose Contrastive Weak-to-Strong Generalization (ConG), a framework that employs contrastive decoding between pre-and postalignment weak models to generate higher-quality samples. This approach enables more reliable capability transfer, denoising, and improved robustness, substantially mitigating the limitations of traditional weak-to-strong methods. Empirical results across different model families confirm consistent improvements, demonstrating the generality and effectiveness of ConG. Taken together, our findings highlight the potential of ConG to advance weak-to-strong generalization and provide a promising pathway toward AGI. Our code is available at: [https://github](https://github) . com/jianghoucheng/ConG  \n1University of Science and Technology of China 2Zhongguancun Academy 3National University of Singapore 4Tsinghua University 5 Singapore Management University. Correspondence to: Xiang Wang \u003C [xiangwang1223@gmail.com](xiangwang1223@gmail.com) >, Xiangnan He \u003C [xiangnanhe@gmail.com](xiangnanhe@gmail.com) >.  \nProceedings of the 43 rd International Conference on Machine Learning, Seoul, South Korea. PMLR 306, 2026 . Copyright 2026 by the author(s) .  \n1. Introduction  \nWeak-to-strong generalization has emerged as a promising paradigm for scaling the capabilities of large language models (LLMs) (Burns et al., 2024 ; Yao et al., 2025 ; Liet al., 2025 ; Somerstep et al., 2025 ; Zhou et al., 2024b) . By leveraging supervised samples generated from an aligned weaker model, a stronger model can be directly trained without requiring additional reward modeling or human feedback (Burns et al., 2024 ; Ouyang et al., 2022 ; Lee et al., 2024) . This enables LLMs to transfer and extend capabilities to even stronger models, providing opportunities for self-enhancement and thus offering a potential pathway toward Artificial General Intelligence (AGI) (Goertzel, 2014) .  \nDespite encouraging progress, the paradigm suffers from poor robustness and limited generalization (Yao et al., 2025 ; Yang et al., 2025) . We attribute this to the inherent biases and preferences embedded in the weaker model: the samples it generates often contain noise and are of relatively low quality. As a result, the stronger model fails to generalize reliably, thereby restricting the applicability of weak-to-strong methods (Lyu et al., 2025) . This raises a central research question: How can we extract higher-quality samples from weak models, without relying on explicit rewards (e.g., human feedback or reward models), to achieve more effective weak-to-strong generalization?  \nRecent success of implicit rewards in preference alignment and reasoning enhancement motivates our approach (Yuan et al., 2024 ; Cui et al., 2025) . Specifically, implicit reward parameterizes the reward as the log-likelihood ratio between outputs from the post-alignment and pre-alignment models, and prior work has shown it to be an unbiased approximation of explicit reward (Rafailov et al., 2023) . This suggests that implicit reward can serve as a reliable signal for assessing sample quality. Moreover, we observe that its lo","cbCaiqT00hwe9WNJ","https://ap.wps.com/l/cbCaiqT00hwe9WNJ","pdf",2655360,3,1,19,"English","en",105,"# Introduction\n## Weak-to-strong generalization paradigm\n## Robustness limits and research question\n## Implicit rewards and relation to contrastive decoding\n## ConG framework and training-sample generation","[{\"question\":\"What is ConG, and how does it generate better training samples?\",\"answer\":\"ConG stands for Contrastive Weak-to-Strong Generalization. It uses contrastive decoding between pre- and post-alignment weak models to produce higher-quality, denoised samples for training the strong model, supported by an implicit-reward/contrastive-decoding connection.\"}]",1784174259,48,{"code":4,"msg":31,"data":32},"ok",{"site_id":25,"language":24,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":78,"head_meta":80,"extra_data":82,"updated_unix":28},"contrastive-weak-to-strong-generalization","",{"@graph":36,"@context":77},[37,53,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,50],{"item":41,"name":42,"@type":43,"position":21},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":20},"https://docshare.wps.com/document/research-report/",{"item":51,"name":13,"@type":43,"position":52},"https://docshare.wps.com/document/contrastive-weak-to-strong-generalization/81552/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":24,"description":14,"dateModified":61,"datePublished":62,"encodingFormat":60,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":41,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-25","2026-07-16",true,{"@type":65,"interactionType":66,"userInteractionCount":20},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71],{"name":72,"@type":73,"acceptedAnswer":74},"What is ConG, and how does it generate better training samples?","Question",{"text":75,"@type":76},"ConG stands for Contrastive Weak-to-Strong Generalization. It uses contrastive decoding between pre- and post-alignment weak models to produce higher-quality, denoised samples for training the strong model, supported by an implicit-reward/contrastive-decoding connection.","Answer","https://schema.org",{"og:url":51,"og:type":79,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":81,"canonical":51},"index,follow",{"doc_id":7,"site_id":25},{"code":4,"msg":5,"data":84},[85,89,93,97,102,107,112,115,120,123,127],{"id":21,"doc_module":4,"doc_module_name":46,"category_name":86,"show_sort_weight":87,"slug":88},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":90,"show_sort_weight":91,"slug":92},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Exam",70,"exam",{"id":98,"doc_module":4,"doc_module_name":46,"category_name":99,"show_sort_weight":100,"slug":101},5,"Comic",60,"comic",{"id":103,"doc_module":4,"doc_module_name":46,"category_name":104,"show_sort_weight":105,"slug":106},6,"Technology",50,"technology",{"id":108,"doc_module":4,"doc_module_name":46,"category_name":109,"show_sort_weight":110,"slug":111},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":113,"slug":114},30,"research-report",{"id":116,"doc_module":4,"doc_module_name":46,"category_name":117,"show_sort_weight":118,"slug":119},9,"Religion & Spirituality",20,"religion-spirituality",{"id":118,"doc_module":4,"doc_module_name":46,"category_name":121,"show_sort_weight":118,"slug":122},"World Cup","world-cup",{"id":124,"doc_module":4,"doc_module_name":46,"category_name":125,"show_sort_weight":124,"slug":126},10,"Lifestyle","lifestyle",{"id":22,"doc_module":4,"doc_module_name":46,"category_name":128,"show_sort_weight":98,"slug":129},"General","general"]