[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-seo-638902-105":3,"detail-sidebar-cat-0-en-105":81,"doc-detail-638902-en":130},{"code":4,"msg":5,"data":6},0,"ok",{"site_id":7,"language":8,"slug":9,"title":10,"keywords":11,"description":12,"schema_data":13,"social_meta":74,"head_meta":76,"extra_data":78,"updated_unix":80},105,"en","deepseek-r1-reinforcement-learning-reasoning-models-incentivizing-reasoning-capability-in-llms-via","DeepSeek-R1 - Reinforcement Learning Reasoning Models - Incentivizing Reasoning Capability in LLMs via","","This research introduces DeepSeek-R1-Zero and DeepSeek-R1, reasoning-focused large language models trained with reinforcement learning. DeepSeek-R1-Zero is trained via large-scale RL without supervised fine-tuning, producing strong and diverse reasoning behaviors but suffering from issues including poor readability and language mixing. DeepSeek-R1 addresses these by adding multi-stage training and cold-start data before RL, achieving reasoning performance comparable to OpenAI-o1-1217. The work open-sources DeepSeek-R1-Zero, DeepSeek-R1, and six distilled dense models.",{"@graph":14,"@context":73},[15,34,56],{"@type":16,"itemListElement":17},"BreadcrumbList",[18,23,27,31],{"item":19,"name":20,"@type":21,"position":22},"https://docshare.wps.com","Home","ListItem",1,{"item":24,"name":25,"@type":21,"position":26},"https://docshare.wps.com/document/","Document",2,{"item":28,"name":29,"@type":21,"position":30},"https://docshare.wps.com/document/research-report/","Research & Report",3,{"item":32,"name":10,"@type":21,"position":33},"https://docshare.wps.com/document/deepseek-r1-reinforcement-learning-reasoning-models-incentivizing-reasoning-capability-in-llms-via/638902/",4,{"url":32,"name":10,"@type":35,"image":36,"author":41,"headline":10,"publisher":44,"fileFormat":47,"inLanguage":8,"description":12,"dateModified":48,"datePublished":49,"encodingFormat":47,"isAccessibleForFree":50,"interactionStatistic":51},"DigitalDocument",{"url":37,"@type":38,"width":39,"height":40},"https://docshare.wps.com/thumbnails/deepseek-r1-reinforcement-learning-reasoning-models-incentivizing-reasoning-capability-in-llms-via/638902.png","ImageObject",300,407,{"name":42,"@type":43},"Alex Sinclair","Person",{"url":19,"name":45,"@type":46},"DocShare","Organization","application/pdf","2026-11-02","2026-10-22",true,{"@type":52,"interactionType":53,"userInteractionCount":55},"InteractionCounter",{"@type":54},"ViewAction",9,{"@type":57,"mainEntity":58},"FAQPage",[59,65,69],{"name":60,"@type":61,"acceptedAnswer":62},"What are DeepSeek-R1-Zero and DeepSeek-R1, and how do they differ?","Question",{"text":63,"@type":64},"DeepSeek-R1-Zero is trained via large-scale reinforcement learning without supervised fine-tuning as a preliminary step. DeepSeek-R1 improves readability and reasoning quality by adding multi-stage training and cold-start data before reinforcement learning.","Answer",{"name":66,"@type":61,"acceptedAnswer":67},"Why does DeepSeek-R1-Zero face challenges during training?",{"text":68,"@type":64},"It demonstrates strong reasoning behaviors but encounters poor readability and language mixing, which motivate further training enhancements in DeepSeek-R1.",{"name":70,"@type":61,"acceptedAnswer":71},"How does DeepSeek-R1 improve reasoning performance compared with DeepSeek-R1-Zero?",{"text":72,"@type":64},"DeepSeek-R1 incorporates cold-start data and a multi-stage training pipeline: it first fine-tunes using cold-start data, then performs reasoning-oriented RL, and finally adds training steps to better align and refine outputs.","https://schema.org",{"og:url":32,"og:type":75,"og:title":10,"og:site_name":45,"og:description":12},"article",{"robots":77,"canonical":32},"index,follow",{"doc_id":79,"site_id":7},638902,1793052815,{"code":4,"msg":82,"data":83},"success",[84,88,92,96,101,106,111,115,119,122,126],{"id":22,"doc_module":4,"doc_module_name":25,"category_name":85,"show_sort_weight":86,"slug":87},"Story & Novel",90,"story-novel",{"id":26,"doc_module":4,"doc_module_name":25,"category_name":89,"show_sort_weight":90,"slug":91},"Literature",80,"literature",{"id":33,"doc_module":4,"doc_module_name":25,"category_name":93,"show_sort_weight":94,"slug":95},"Exam",70,"exam",{"id":97,"doc_module":4,"doc_module_name":25,"category_name":98,"show_sort_weight":99,"slug":100},5,"Comic",60,"comic",{"id":102,"doc_module":4,"doc_module_name":25,"category_name":103,"show_sort_weight":104,"slug":105},6,"Technology",50,"technology",{"id":107,"doc_module":4,"doc_module_name":25,"category_name":108,"show_sort_weight":109,"slug":110},7,"Healthcare",40,"healthcare",{"id":112,"doc_module":4,"doc_module_name":25,"category_name":29,"show_sort_weight":113,"slug":114},8,30,"research-report",{"id":55,"doc_module":4,"doc_module_name":25,"category_name":116,"show_sort_weight":117,"slug":118},"Religion & Spirituality",20,"religion-spirituality",{"id":117,"doc_module":4,"doc_module_name":25,"category_name":120,"show_sort_weight":117,"slug":121},"World Cup","world-cup",{"id":123,"doc_module":4,"doc_module_name":25,"category_name":124,"show_sort_weight":123,"slug":125},10,"Lifestyle","lifestyle",{"id":127,"doc_module":4,"doc_module_name":25,"category_name":128,"show_sort_weight":97,"slug":129},19,"General","general",{"code":4,"msg":82,"data":131},{"doc_id":79,"user_id":132,"nickname":42,"user_avatar":133,"doc_module":4,"category_id":112,"category_name":29,"doc_title":10,"doc_description":12,"doc_content":134,"file_id":135,"file_url":136,"file_type":137,"file_size":138,"view_count":55,"is_deleted":4,"is_public":22,"is_downloadable":22,"audit_status":22,"page_count":139,"language":140,"language_code":8,"site_id":7,"html_lang":8,"table_of_contents":141,"faqs":142,"seo_title":143,"seo_description":12,"update_tm":144,"read_time":145},1099523882182,"https://ap-avatar.wpscdn.com/davatar_6f874abed73319feea01a86fa6f0fab8","DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via  \nReinforcement Learning  \nDeepSeek-AI  \n[research@deepseek.com](research@deepseek.com)  \nAbstract  \nWe introduce our first-generation reasoning models, DeepSeek-R1-Zero and DeepSeek-R1 . DeepSeek-R1-Zero, a model trained via large-scale reinforcement learning (RL) without supervised fine-tuning (SFT) as a preliminary step, demonstrates remarkable reasoning capabilities. Through RL, DeepSeek-R1-Zero naturally emerges with numerous powerful and intriguing reasoning behaviors. However, it encounters challenges such as poor readability, and language mixing. To address these issues and further enhance reasoning performance, we introduce DeepSeek-R1, which incorporates multi-stage training and cold-start data before RL. DeepSeekR1 achieves performance comparable to OpenAI-o1-1217 on reasoning tasks. To support the research community, we open-source DeepSeek-R1-Zero, DeepSeek-R1, and six dense models (1.5B, 7B, 8B, 14B, 32B, 70B) distilled from DeepSeek-R1 based on Qwen and Llama.  \n\n| | DeepSeek-R1 |  OpenAI-o1-1217 |  DeepSeek-R1-32B  | OpenAI-o1-mini |  DeepSeek-V3 |\n| --- | --- | --- | --- | --- | --- |\n|  |  |  |  |  |  |\n\nAccuracy / Percentile (%)  \n100  \n80  \n60  \n40  \n20  \n0  \n96.3 96.6 97.3 93.4  \n90.6  \n79.8 79.2  \n72.6  \n75.7  \n71.5  \n96.4  \n94.3  \n90.0 90.2  \n90.8 91.8  \n87.4 88.5 85.2  \n63.6  \n62.1  \n58.7 60.0 59.1  \n49.2 48.9  \n41.6  \n39.2  \n36.8  \n42.0  \nAIME 2024  \n(Pass@1)  \nCodeforces  \n(Percentile)  \nGPQA Diamond  \n(Pass@1)  \nMATH-500  \n(Pass@1)  \nMMLU  \n(Pass@1)  \nSWE-bench Verified  \n(Resolved)  \nFigure 1 | Benchmark performance of DeepSeek-R1 .  \nContents  \n1 Introduction 3  \n1.1 Contributions ....................................... 4  \n1.2 Summary of Evaluation Results ............................. 4  \n2 Approach 5  \n2.1 Overview .......................................... 5  \n2.2 DeepSeek-R1-Zero: Reinforcement Learning on the Base Model .......... 5  \n2.2.1 Reinforcement Learning Algorithm ...................... 5  \n2.2.2 Reward Modeling ................................ 6  \n2.2.3 Training Template ................................ 6  \n2.2.4 Performance, Self-evolution Process and Aha Moment of DeepSeek-R1-Zero 6  \n2.3 DeepSeek-R1: Reinforcement Learning with Cold Start ............... 9  \n2.3.1 Cold Start ..................................... 9  \n2.3.2 Reasoning-oriented Reinforcement Learning ................. 10  \n2.3.3 Rejection Sampling and Supervised Fine-Tuning ............... 10  \n2.3.4 Reinforcement Learning for all Scenarios ................... 11  \n2.4 Distillation: Empower Small Models with Reasoning Capability .......... 11  \n3 Experiment 11  \n3.1 DeepSeek-R1 Evaluation ................................. 13  \n3.2 Distilled Model Evaluation ............................... 14  \n4 Discussion 14  \n4.1 Distillation v.s. Reinforcement Learning ........................ 14  \n4.2 Unsuccessful Attempts .................................. 15  \n5 Conclusion, Limitations, and Future Work 16  \nA Contributions and Acknowledgments 20  \n1. Introduction  \nIn recent years, Large Language Models (LLMs) have been undergoing rapid iteration and evolution (Anthropic, 2024; Google, 2024; OpenAI, 2024a), progressively diminishing the gap towards Artificial General Intelligence (AGI) .  \nRecently, post-training has emerged as an important component of the full training pipeline. It has been shown to enhance accuracy on reasoning tasks, align with social values, and adapt to user preferences, all while requiring relatively minimal computational resources against pre-training. In the context of reasoning capabilities, OpenAI’s o1 (OpenAI, 2024b) series models were the first to introduce inference-time scaling by increasing the length of the Chain-ofThought reasoning process. This approach has achieved significant improvements in various reasoning tasks, such as mathematics, coding, and scientific reasoning. However, the challenge of effective test-time scaling remains ","cbCait45qCeKSXV1","https://ap.wps.com/l/cbCait45qCeKSXV1","pdf",1328991,22,"English","# 1 Introduction\n## 1.1 Contributions\n## 1.2 Summary of Evaluation Results\n# 2 Approach\n## 2.1 Overview\n## 2.2 DeepSeek-R1-Zero: Reinforcement Learning on the Base Model\n## 2.3 DeepSeek-R1: Reinforcement Learning with Cold Start\n## 2.4 Distillation: Empower Small Models with Reasoning Capability\n# 3 Experiment\n## 3.1 DeepSeek-R1 Evaluation\n## 3.2 Distilled Model Evaluation\n# 4 Discussion\n## 4.1 Distillation v.s. Reinforcement Learning\n## 4.2 Unsuccessful Attempts\n# 5 Conclusion, Limitations, and Future Work","[{\"question\":\"What are DeepSeek-R1-Zero and DeepSeek-R1, and how do they differ?\",\"answer\":\"DeepSeek-R1-Zero is trained via large-scale reinforcement learning without supervised fine-tuning as a preliminary step. DeepSeek-R1 improves readability and reasoning quality by adding multi-stage training and cold-start data before reinforcement learning.\"},{\"question\":\"Why does DeepSeek-R1-Zero face challenges during training?\",\"answer\":\"It demonstrates strong reasoning behaviors but encounters poor readability and language mixing, which motivate further training enhancements in DeepSeek-R1.\"},{\"question\":\"How does DeepSeek-R1 improve reasoning performance compared with DeepSeek-R1-Zero?\",\"answer\":\"DeepSeek-R1 incorporates cold-start data and a multi-stage training pipeline: it first fine-tunes using cold-start data, then performs reasoning-oriented RL, and finally adds training steps to better align and refine outputs.\"}]","DeepSeek-R1 - Reinforcement Learning Reasoning Models - Incentivizing Reasoning Capability in LLMs via | PDF",1792700309,55]