[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-85214-en":3,"doc-seo-85214-105":30,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":21,"is_downloadable":21,"audit_status":21,"page_count":22,"language":23,"language_code":24,"site_id":25,"html_lang":24,"table_of_contents":26,"faqs":27,"seo_title":13,"seo_description":14,"update_tm":28,"read_time":29},85214,13056703019662,"Evangeline","https://ap-avatar.wpscdn.com/avatar/be000253a8e92610077?_k=1778726343310543188",8,"Research & Report","Mitigating LLM Sycophancy in Code Smell Detection Using Evidence Guided Reasoning Prompts","Large Language Models (LLMs) are increasingly applied to code smell detection by leveraging their ability to interpret program semantics, yet their reliability under changing prompt conditions remains insufficiently studied. The work focuses on sycophancy bias, where outputs follow user-provided assumptions rather than objective code evidence. Using the MLCQ dataset, the study evaluates prompt framings such as confirmation bias, contradictory hints, and false premises, measuring Decision Flip Rates and False Alignment Rates. Evidence-Guided Debiasing Prompting (EGDP) reduces instability and improves robustness by enforcing evidence-first reasoning, demonstrating a practical mitigation for dependable software quality analysis.","Mitigating LLM Sycophancy in Code Smell Detection Using Evidence-Guided Reasoning Prompts  \nIstiaq Ahmed Fahad  \n[bsse1204@iit.du.ac.bd](bsse1204@iit.du.ac.bd)[ ](bsse1204@iit.du.ac.bd)Institute of Information Technology University of Dhaka Dhaka, Bangladesh  \nKamruzzaman Asif  \n[bsse1217@iit.du.ac.bd](bsse1217@iit.du.ac.bd)[ ](bsse1217@iit.du.ac.bd)Institute of Information Technology University of Dhaka Dhaka, Bangladesh  \nMd. Nurul Ahad Tawhid  \n[tawhid@iit.du.ac.bd](tawhid@iit.du.ac.bd)[ ](tawhid@iit.du.ac.bd)Institute of Information Technology University of Dhaka Dhaka, Bangladesh  \narXiv :2607 . 104 1 1v 1 [ cs . SE] 11 Jul 2026  \nAbstract  \nLarge Language Models (LLMs) are increasingly used for code smell detection tasks due to their ability to interpret program semantics. However, their reliability in this context remains poorly explored, particularly under varying prompt conditions where model predictions may be influenced by external cues rather than code characteristics. One such limitation is sycophancy bias, where models tend to align their outputs with user-provided assumptions instead of performing objective analysis. In this paper, we present the first systematic empirical study of sycophancy bias in LLM-based code smell detection. Using the MLCQ dataset, we evaluate how different prompt framings like confirmation bias, contradictory hints, and false premises affect model predictions. Our results show that LLMs are highly sensitive to prompt variations, with Decision Flip Rates reaching up to 72% and False Alignment Rates exceeding 90%, indicating substantial instability and agreement with misleading prompts. To address this issue, we propose Evidence-Guided Debiasing Prompting (EGDP), a structured prompting strategy that enforces evidence-first reasoning. EGDP reduces decision instability and improves robustness, lowering Decision Flip Rates to as low as 12% and False Alignment Rates to as low as 21%, while increasing reliance on structurally grounded evidence. Our findings demonstrate that sycophancy bias poses a critical threat to the reliability of LLM-based code smell detection, and that evidenceguided reasoning provides an effective and generalizable mitigation approach.  \nCCS Concepts  \n• Software and its engineering → Software maintenance tools; Automated static analysis; Software testing and debugging; • Computing methodologies → Natural language processing.  \nKeywords  \ncode smell detection, large language models, sycophancy bias, prompt engineering, software quality, static analysis  \nPermission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission [and/or a fee. Request permissions from permissions@acm.org](and/or a fee. Request permissions from permissions@acm.org).  \nConference’17, Washington, DC, USA  \n© 2026 Copyright held by the owner/author(s) . Publication rights licensed to ACM. ACM ISBN 978-1-4503-XXXX-X/2026/06  \n[https://doi.org/10.1145/nnnnnnn.nnnnnnn](https://doi.org/10.1145/nnnnnnn.nnnnnnn)  \nACM Reference Format:  \nIstiaq Ahmed Fahad, Kamruzzaman Asif, and Md. Nurul Ahad Tawhid.  \n2026. Mitigating LLM Sycophancy in Code Smell Detection Using EvidenceGuided Reasoning Prompts. In . ACM, New York, NY, USA, 11 pages. [https:](https:)//[doi.org/10.1145/nnnnnnn.nnnnnnn](doi.org/10.1145/nnnnnnn.nnnnnnn)  \n1 Introduction  \nSoftware maintenance accounts for a significant portion of the total cost of a software system, often reaching up to 70%[19] . A major contributor to this cost is the presence of code smells, which reflect poor design deci","cbCaiiZg9EmjXAeR","https://ap.wps.com/l/cbCaiiZg9EmjXAeR","pdf",693746,3,1,11,"English","en",105,"# Introduction\n# Abstract\n# CCS Concepts\n# Keywords\n# Method and Evaluation\n# Evidence-Guided Debiasing Prompting (EGDP)","[{\"question\":\"What reliability problem do the authors target in LLM-based code smell detection?\",\"answer\":\"They target the unreliability caused by sycophancy bias, where the model aligns outputs with user assumptions instead of evidence from the code, especially under different prompt framings.\"},{\"question\":\"How do the authors evaluate the impact of prompt variations?\",\"answer\":\"They use the MLCQ dataset to test how confirmation bias, contradictory hints, and false premises affect predictions, reporting metrics such as Decision Flip Rates and False Alignment Rates.\"},{\"question\":\"What is EGDP and how does it mitigate sycophancy?\",\"answer\":\"Evidence-Guided Debiasing Prompting (EGDP) enforces evidence-first reasoning in a structured prompting strategy, reducing decision instability and lowering both Decision Flip Rates and False Alignment Rates.\"}]",1784201789,28,{"code":4,"msg":31,"data":32},"ok",{"site_id":25,"language":24,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":28},"mitigating-llm-sycophancy-in-code-smell-detection-using-evidence-guided-reasoning-prompts","",{"@graph":36,"@context":85},[37,53,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,50],{"item":41,"name":42,"@type":43,"position":21},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":20},"https://docshare.wps.com/document/research-report/",{"item":51,"name":13,"@type":43,"position":52},"https://docshare.wps.com/document/mitigating-llm-sycophancy-in-code-smell-detection-using-evidence-guided-reasoning-prompts/85214/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":24,"description":14,"dateModified":61,"datePublished":62,"encodingFormat":60,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":41,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-24","2026-07-16",true,{"@type":65,"interactionType":66,"userInteractionCount":20},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"What reliability problem do the authors target in LLM-based code smell detection?","Question",{"text":75,"@type":76},"They target the unreliability caused by sycophancy bias, where the model aligns outputs with user assumptions instead of evidence from the code, especially under different prompt framings.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"How do the authors evaluate the impact of prompt variations?",{"text":80,"@type":76},"They use the MLCQ dataset to test how confirmation bias, contradictory hints, and false premises affect predictions, reporting metrics such as Decision Flip Rates and False Alignment Rates.",{"name":82,"@type":73,"acceptedAnswer":83},"What is EGDP and how does it mitigate sycophancy?",{"text":84,"@type":76},"Evidence-Guided Debiasing Prompting (EGDP) enforces evidence-first reasoning in a structured prompting strategy, reducing decision instability and lowering both Decision Flip Rates and False Alignment Rates.","https://schema.org",{"og:url":51,"og:type":87,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":89,"canonical":51},"index,follow",{"doc_id":7,"site_id":25},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":21,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":46,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":46,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":46,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":46,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":46,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":46,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]