[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-seo-298440-105":3,"detail-sidebar-cat-0-en-105":81,"doc-detail-298440-en":131},{"code":4,"msg":5,"data":6},0,"ok",{"site_id":7,"language":8,"slug":9,"title":10,"keywords":11,"description":12,"schema_data":13,"social_meta":74,"head_meta":76,"extra_data":78,"updated_unix":80},105,"en","chain-of-thought-prompting-elicits-reasoning-in-large-language-models","Chain-of-Thought Prompting Elicits Reasoning in Large Language Models","","Explores how generating a chain of thought—intermediate reasoning steps—can significantly improve large language models’ ability to perform complex reasoning. Introduces chain-of-thought prompting, where a few exemplar demonstrations are provided in the prompt, and shows reasoning abilities emerge naturally in sufficiently large models. Experiments on three large language models demonstrate improved arithmetic, commonsense, and symbolic reasoning, including striking gains on GSM8K where PaLM 540B with eight exemplars reaches state-of-the-art accuracy.",{"@graph":14,"@context":73},[15,34,56],{"@type":16,"itemListElement":17},"BreadcrumbList",[18,23,27,31],{"item":19,"name":20,"@type":21,"position":22},"https://docshare.wps.com","Home","ListItem",1,{"item":24,"name":25,"@type":21,"position":26},"https://docshare.wps.com/document/","Document",2,{"item":28,"name":29,"@type":21,"position":30},"https://docshare.wps.com/document/research-report/","Research & Report",3,{"item":32,"name":10,"@type":21,"position":33},"https://docshare.wps.com/document/chain-of-thought-prompting-elicits-reasoning-in-large-language-models/298440/",4,{"url":32,"name":10,"@type":35,"image":36,"author":41,"headline":10,"publisher":44,"fileFormat":47,"inLanguage":8,"description":12,"dateModified":48,"datePublished":49,"encodingFormat":47,"isAccessibleForFree":50,"interactionStatistic":51},"DigitalDocument",{"url":37,"@type":38,"width":39,"height":40},"https://docshare.wps.com/thumbnails/chain-of-thought-prompting-elicits-reasoning-in-large-language-models/298440.png","ImageObject",300,407,{"name":42,"@type":43},"\tJames","Person",{"url":19,"name":45,"@type":46},"DocShare","Organization","application/pdf","2026-11-11","2026-09-18",true,{"@type":52,"interactionType":53,"userInteractionCount":55},"InteractionCounter",{"@type":54},"ViewAction",23,{"@type":57,"mainEntity":58},"FAQPage",[59,65,69],{"name":60,"@type":61,"acceptedAnswer":62},"What is chain-of-thought prompting and how is it used?","Question",{"text":63,"@type":64},"It is a prompting method where the prompt includes a few exemplars, each containing the input, an intermediate chain of thought, and the final output. The model learns to produce a similar intermediate reasoning sequence to reach the answer.","Answer",{"name":66,"@type":61,"acceptedAnswer":67},"Why does chain-of-thought prompting improve complex reasoning?",{"text":68,"@type":64},"The paper argues that providing natural-language intermediate reasoning steps helps large language models perform multi-step reasoning tasks more effectively than standard prompting, especially at sufficient model scale.",{"name":70,"@type":61,"acceptedAnswer":71},"What tasks and benchmarks show improvements?",{"text":72,"@type":64},"Experiments report gains on arithmetic, commonsense, and symbolic reasoning tasks. A highlighted example is GSM8K math word problems, where PaLM 540B with eight chain-of-thought exemplars achieves state-of-the-art accuracy.","https://schema.org",{"og:url":32,"og:type":75,"og:title":10,"og:site_name":45,"og:description":12},"article",{"robots":77,"canonical":32},"index,follow",{"doc_id":79,"site_id":7},298440,1790219024,{"code":4,"msg":82,"data":83},"success",[84,88,92,96,101,106,111,115,120,123,127],{"id":22,"doc_module":4,"doc_module_name":25,"category_name":85,"show_sort_weight":86,"slug":87},"Story & Novel",90,"story-novel",{"id":26,"doc_module":4,"doc_module_name":25,"category_name":89,"show_sort_weight":90,"slug":91},"Literature",80,"literature",{"id":33,"doc_module":4,"doc_module_name":25,"category_name":93,"show_sort_weight":94,"slug":95},"Exam",70,"exam",{"id":97,"doc_module":4,"doc_module_name":25,"category_name":98,"show_sort_weight":99,"slug":100},5,"Comic",60,"comic",{"id":102,"doc_module":4,"doc_module_name":25,"category_name":103,"show_sort_weight":104,"slug":105},6,"Technology",50,"technology",{"id":107,"doc_module":4,"doc_module_name":25,"category_name":108,"show_sort_weight":109,"slug":110},7,"Healthcare",40,"healthcare",{"id":112,"doc_module":4,"doc_module_name":25,"category_name":29,"show_sort_weight":113,"slug":114},8,30,"research-report",{"id":116,"doc_module":4,"doc_module_name":25,"category_name":117,"show_sort_weight":118,"slug":119},9,"Religion & Spirituality",20,"religion-spirituality",{"id":118,"doc_module":4,"doc_module_name":25,"category_name":121,"show_sort_weight":118,"slug":122},"World Cup","world-cup",{"id":124,"doc_module":4,"doc_module_name":25,"category_name":125,"show_sort_weight":124,"slug":126},10,"Lifestyle","lifestyle",{"id":128,"doc_module":4,"doc_module_name":25,"category_name":129,"show_sort_weight":97,"slug":130},19,"General","general",{"code":4,"msg":82,"data":132},{"doc_id":79,"user_id":133,"nickname":42,"user_avatar":134,"doc_module":4,"category_id":112,"category_name":29,"doc_title":10,"doc_description":12,"doc_content":135,"file_id":136,"file_url":137,"file_type":138,"file_size":139,"view_count":55,"is_deleted":4,"is_public":22,"is_downloadable":22,"audit_status":22,"page_count":140,"language":141,"language_code":8,"site_id":7,"html_lang":8,"table_of_contents":142,"faqs":143,"seo_title":144,"seo_description":12,"update_tm":145,"read_time":146,"preview_page_count":4},2336474466412,"https://ap-avatar.wpscdn.com/davatar_155a257f0dc6eb9ab79c44ca47cae57d","arXiv :2201 . 11903v6 [ cs .CL] 10 Jan 2023  \nChain-of-Thought Prompting Elicits Reasoning in Large Language Models  \nJason Wei Xuezhi Wang Dale Schuurmans Maarten Bosma Brian Ichter Fei Xia Ed H. Chi Quoc V. Le Denny Zhou  \nGoogle Research, Brain Team {jasonwei,[dennyzhou}@google.com](dennyzhou}@google.com)  \nAbstract  \nWe explore how generating a chain of thought—a series of intermediate reasoning steps—significantly improves the ability of large language models to perform complex reasoning. In particular, we show how such reasoning abilities emerge naturally in sufficiently large language models via a simple method called chain-ofthought prompting, where a few chain of thought demonstrations are provided as exemplars in prompting.  \nExperiments on three large language models show that chain-of-thought prompting improves performance on a range of arithmetic, commonsense, and symbolic reasoning tasks. The empirical gains can be striking. For instance, prompting a PaLM 540B with just eight chain-of-thought exemplars achieves state-of-the-art accuracy on the GSM8K benchmark of math word problems, surpassing even finetuned GPT-3 with a verifier.  \nFigure 1: Chain-of-thought prompting enables large language models to tackle complex arithmetic, commonsense, and symbolic reasoning tasks. Chain-of-thought reasoning processes are highlighted.  \n36th Conference on Neural Information Processing Systems (NeurIPS 2022) .  \n1 Introduction  \nThe NLP landscape has recently been revolutionized by language models (Peters et al., 2018 ; Devlin et al., 2019 ; Brown et al., 2020, inter alia) . Scaling up the size of language models has been shown to confer a range of benefits, such as improved performance and sample efficiency (Kaplan et al., 2020 ; Brown et al., 2020, inter alia) . However, scaling up model size alone has not proved sufficient for achieving high performance on challenging tasks such as arithmetic, commonsense, and symbolic reasoning (Rae et al., 2021) .  \nThis work explores how the reasoning ability of large language models can be unlocked by a simple method motivated by two ideas. First, techniques for arithmetic reasoning can benefit from generating natural language rationales that lead to the final answer. Prior work has given models the ability to generate natural language intermediate steps by training from scratch (Ling et al., 2017) or finetuning a pretrained model (Cobbe et al., 2021), in addition to neuro-symbolic methods that use formal languages instead of natural language (Roy and Roth, 2015 ; Chiang and Chen, 2019 ; Amini et al., 2019 ; Chen et al., 2019) . Second, large language models offer the exciting prospect of in-context few-shot learning via prompting.  \n Finetuned GPT-3 175B  Prior best  \n PaLM 540B: standard prompting  \n PaLM 540B: chain-of-thought prompting  \nSolve rate (%)  \n100  \n80  \n60  \n40  \n20  \n0  \nMath Word Problems (GSM8K)  \nFigure 2: PaLM 540B uses chain-ofthought prompting to achieve new stateof-the-art performance on the GSM8K benchmark of math word problems. Finetuned GPT-3 and prior best are from Cobbe et al. (2021) .  \nThat is, instead of finetuning a separate  \nlanguage model checkpoint for each new task, one can simply “prompt” the model with a few input–output exemplars demonstrating the task. Remarkably, this has been successful for a range of simple question-answering tasks (Brown et al., 2020) .  \nBoth of the above ideas, however, have key limitations. For rationale-augmented training and finetuning methods, it is costly to create a large set of high quality rationales, which is much more complicated than simple input–output pairs used in normal machine learning. For the traditional fewshot prompting method used in Brown et al. (2020), it works poorly on tasks that require reasoning abilities, and often does not improve substantially with increasing language model scale (Rae et al., 2021) . In this paper, we combine the strengths of these two ideas in a way that avoids their limitations. Specific","cbCaisCsP0P3G483","https://ap.wps.com/l/cbCaisCsP0P3G483","pdf",891773,43,"English","# Abstract\n# Introduction\n# Chain-of-Thought Prompting","[{\"question\":\"What is chain-of-thought prompting and how is it used?\",\"answer\":\"It is a prompting method where the prompt includes a few exemplars, each containing the input, an intermediate chain of thought, and the final output. The model learns to produce a similar intermediate reasoning sequence to reach the answer.\"},{\"question\":\"Why does chain-of-thought prompting improve complex reasoning?\",\"answer\":\"The paper argues that providing natural-language intermediate reasoning steps helps large language models perform multi-step reasoning tasks more effectively than standard prompting, especially at sufficient model scale.\"},{\"question\":\"What tasks and benchmarks show improvements?\",\"answer\":\"Experiments report gains on arithmetic, commonsense, and symbolic reasoning tasks. A highlighted example is GSM8K math word problems, where PaLM 540B with eight chain-of-thought exemplars achieves state-of-the-art accuracy.\"}]","Chain-of-Thought Prompting Elicits Reasoning in Large Language Models | PDF",1789757632,108]