[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-seo-273480-105":3,"detail-sidebar-cat-0-en-105":81,"doc-detail-273480-en":131},{"code":4,"msg":5,"data":6},0,"ok",{"site_id":7,"language":8,"slug":9,"title":10,"keywords":11,"description":12,"schema_data":13,"social_meta":74,"head_meta":76,"extra_data":78,"updated_unix":80},105,"en","when-do-explanations-help-in-context-learning-a-comparative-study-of-natural-language-explanation-types-and-faithfulness","When Do Explanations Help In-Context Learning? - A Comparative Study of Natural Language Explanation Types and Faithfulness","","Natural language explanations (NLEs) are increasingly used to influence model behavior in in-context learning (ICL), yet their relative effectiveness across explanation types remains unclear. This study compares four instruction-tuned models across six benchmarks, analyzing effects of NLE source (human, self-generated, external LLM) and selection strategy (random vs faithfulness-based). Results show frequent gains on classification-style tasks from adding NLEs, with external LLM explanations performing strongly and self-generated explanations depending more on selection. On math reasoning, outcomes vary by model and source, and faithfulness metrics may conflict, yielding different selected examples and downstream utility.",{"@graph":14,"@context":73},[15,34,56],{"@type":16,"itemListElement":17},"BreadcrumbList",[18,23,27,31],{"item":19,"name":20,"@type":21,"position":22},"https://docshare.wps.com","Home","ListItem",1,{"item":24,"name":25,"@type":21,"position":26},"https://docshare.wps.com/document/","Document",2,{"item":28,"name":29,"@type":21,"position":30},"https://docshare.wps.com/document/research-report/","Research & Report",3,{"item":32,"name":10,"@type":21,"position":33},"https://docshare.wps.com/document/when-do-explanations-help-in-context-learning-a-comparative-study-of-natural-language-explanation-types-and-faithfulness/273480/",4,{"url":32,"name":10,"@type":35,"image":36,"author":41,"headline":10,"publisher":44,"fileFormat":47,"inLanguage":8,"description":12,"dateModified":48,"datePublished":49,"encodingFormat":47,"isAccessibleForFree":50,"interactionStatistic":51},"DigitalDocument",{"url":37,"@type":38,"width":39,"height":40},"https://docshare.wps.com/thumbnails/when-do-explanations-help-in-context-learning-a-comparative-study-of-natural-language-explanation-types-and-faithfulness/273480.png","ImageObject",300,407,{"name":42,"@type":43},"Aurora","Person",{"url":19,"name":45,"@type":46},"DocShare","Organization","application/pdf","2026-11-01","2026-09-15",true,{"@type":52,"interactionType":53,"userInteractionCount":55},"InteractionCounter",{"@type":54},"ViewAction",12,{"@type":57,"mainEntity":58},"FAQPage",[59,65,69],{"name":60,"@type":61,"acceptedAnswer":62},"How does the study compare different natural language explanation types in in-context learning?","Question",{"text":63,"@type":64},"It evaluates four instruction-tuned models across six benchmarks while varying explanation source (human, self-generated, external LLM) and explanation selection (random vs faithfulness-based filtering).","Answer",{"name":66,"@type":61,"acceptedAnswer":67},"What overall trends are observed when explanations are added to few-shot prompts?",{"text":68,"@type":64},"On classification-style benchmarks, adding NLEs to few-shot prompts often improves accuracy compared with few-shot prompting without explanations.",{"name":70,"@type":61,"acceptedAnswer":71},"How do faithfulness-based selection results differ across tasks and metrics?",{"text":72,"@type":64},"Average gains from faithfulness-based selection of self-generated explanations are small, but performance can increase or decrease depending on the evaluation metric, task, and model. Different faithfulness metrics can select different examples and lead to substantially different downstream utility.","https://schema.org",{"og:url":32,"og:type":75,"og:title":10,"og:site_name":45,"og:description":12},"article",{"robots":77,"canonical":32},"index,follow",{"doc_id":79,"site_id":7},273480,1789471636,{"code":4,"msg":82,"data":83},"success",[84,88,92,96,101,106,111,115,120,123,127],{"id":22,"doc_module":4,"doc_module_name":25,"category_name":85,"show_sort_weight":86,"slug":87},"Story & Novel",90,"story-novel",{"id":26,"doc_module":4,"doc_module_name":25,"category_name":89,"show_sort_weight":90,"slug":91},"Literature",80,"literature",{"id":33,"doc_module":4,"doc_module_name":25,"category_name":93,"show_sort_weight":94,"slug":95},"Exam",70,"exam",{"id":97,"doc_module":4,"doc_module_name":25,"category_name":98,"show_sort_weight":99,"slug":100},5,"Comic",60,"comic",{"id":102,"doc_module":4,"doc_module_name":25,"category_name":103,"show_sort_weight":104,"slug":105},6,"Technology",50,"technology",{"id":107,"doc_module":4,"doc_module_name":25,"category_name":108,"show_sort_weight":109,"slug":110},7,"Healthcare",40,"healthcare",{"id":112,"doc_module":4,"doc_module_name":25,"category_name":29,"show_sort_weight":113,"slug":114},8,30,"research-report",{"id":116,"doc_module":4,"doc_module_name":25,"category_name":117,"show_sort_weight":118,"slug":119},9,"Religion & Spirituality",20,"religion-spirituality",{"id":118,"doc_module":4,"doc_module_name":25,"category_name":121,"show_sort_weight":118,"slug":122},"World Cup","world-cup",{"id":124,"doc_module":4,"doc_module_name":25,"category_name":125,"show_sort_weight":124,"slug":126},10,"Lifestyle","lifestyle",{"id":128,"doc_module":4,"doc_module_name":25,"category_name":129,"show_sort_weight":97,"slug":130},19,"General","general",{"code":4,"msg":82,"data":132},{"doc_id":79,"user_id":133,"nickname":42,"user_avatar":134,"doc_module":4,"category_id":112,"category_name":29,"doc_title":10,"doc_description":12,"doc_content":135,"file_id":136,"file_url":137,"file_type":138,"file_size":139,"view_count":55,"is_deleted":4,"is_public":22,"is_downloadable":22,"audit_status":22,"page_count":140,"language":141,"language_code":8,"site_id":7,"html_lang":8,"table_of_contents":142,"faqs":143,"seo_title":144,"seo_description":12,"update_tm":80,"read_time":145,"preview_page_count":4},4810365810221,"https://ap-avatar.wpscdn.com/davatar_155a257f0dc6eb9ab79c44ca47cae57d","arXiv :2608 . 16627v1 [ cs .CL] 17 Aug 2026  \nWhen Do Explanations Help In-Context Learning? A Comparative Study of Natural Language Explanation Types and Faithfulness  \nMahdi Dhaini∗  \nMaiNLP lab, CIS, LMU Munich  \nMunich Center for Machine Learning (MCML)  \nAdam Dejl  \nDepartment of Computing, Imperial College London  \nJuraj Vladika  \nTechnical University of Munich  \nVolkan Özer  \nTechnical University of Munich, Munich Center for Machine Learning (MCML)  \nBarbara Plank  \nMaiNLP lab, CIS, LMU Munich  \nMunich Center for Machine Learning (MCML)  \nGjergji Kasneci  \nTechnical University of Munich, Munich Center for Machine Learning (MCML)  \n[mahdi. dhaini@lmu. de](mahdi. dhaini@lmu. de)  \n[adam.dejl18@imperial. ac.uk](adam.dejl18@imperial. ac.uk)  \n[juraj.vladika@tum. de](juraj.vladika@tum. de)  \n[volkan. oezer@tum. de](volkan. oezer@tum. de)  \n[b.plank@lmu. de](b.plank@lmu. de)  \n[gjergji.kasneci@tum. de](gjergji.kasneci@tum. de)  \nAbstract  \nNatural language explanations (NLEs) are increasingly used as inputs, for example, as fewshot rationales that influence model behavior in in-context learning (ICL) . However, it remains unclear how different types of NLEs compare in their effects on downstream model performance in explanation-augmented prompting. Therefore, we provide a comparative evaluation across six benchmarks and four instruction-tuned models, studying how NLE source (human-written when available, self-generated explanations, generated by an external LLM) and NLE selection (random vs faithfulness-based filtering) affect downstream utility of NLEs when used in ICL settings. Our extensive evaluation shows that, on classificationstyle benchmarks, adding NLEs to few-shot prompts often improves accuracy over few-shot prompting without explanations; among NLE sources, externally generated LLM-NLEs often provide strong downstream utility and remain competitive with human rationales where both are available, whereas self-NLEs are more sensitive to the selection strategy. On math reasoning, the effects are more model-and source-dependent. We further show that faithfulness-based selection of self-NLEs yields small average gains overall, but can improve or reduce performance depending on the metric, task, and model. Different faithfulness metrics can disagree substantially, affecting which self-NLE examples are selected and their downstream predictive utility. Robustness tests with randomly swapped and outof-distribution rationales indicate partial robustness, suggesting that semantic alignment contributes to performance gains. Overall, our results provide insights for selecting and reporting explanations that influence model behavior in practical prompting pipelines.  \n∗ Corresponding author  \n1 Introduction  \nNatural language explanations (NLEs), also referred to as free-text rationales, have emerged as a prominent approach to improving the interpretability of language model (LM) predictions (Cambria et al., 2023; Wiegreffe & Marasovic, 2021; Bhan et al., 2024; Dhaini et al., 2025) . As natural-language justifications for model predictions, NLEs are widely used to communicate model reasoning 1 to users. Beyond interpretability, however, NLEs may also serve a functional role: recent work suggests that providing explanations during inference can improve model performance (Dhaini et al., 2025) . Human-annotated rationales are often considered a gold standard, but they are expensive, slow to obtain, and subject to annotation bias and inconsistency (Yao et al., 2023; Hartmann & Sonntag, 2022) . An alternative is to generate explanations automatically, either via self-explanations, in which the model justifies its own predictions, or by prompting an auxiliary LLM to generate rationales (Mishra et al., 2024; Wang et al., 2025; Wei Jie et al., 2024) .  \nIn parallel, in-context learning (ICL) has emerged as one of the key LLM capabilities, enabling task adaptation via example-driven prompts without parameter updates (Liu et al., 2021) .","cbCaigYLHe0L63ZL","https://ap.wps.com/l/cbCaigYLHe0L63ZL","pdf",1693555,53,"English","# Abstract\n# Introduction\n## Motivation and background\n## NLE roles in real workflows\n## Research gap and objectives","[{\"question\":\"How does the study compare different natural language explanation types in in-context learning?\",\"answer\":\"It evaluates four instruction-tuned models across six benchmarks while varying explanation source (human, self-generated, external LLM) and explanation selection (random vs faithfulness-based filtering).\"},{\"question\":\"What overall trends are observed when explanations are added to few-shot prompts?\",\"answer\":\"On classification-style benchmarks, adding NLEs to few-shot prompts often improves accuracy compared with few-shot prompting without explanations.\"},{\"question\":\"How do faithfulness-based selection results differ across tasks and metrics?\",\"answer\":\"Average gains from faithfulness-based selection of self-generated explanations are small, but performance can increase or decrease depending on the evaluation metric, task, and model. Different faithfulness metrics can select different examples and lead to substantially different downstream utility.\"}]","When Do Explanations Help In-Context Learning? - A Comparative Study of Natural Language Explanation Types and Faithfulness | PDF",134]