[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-86476-en":3,"doc-seo-86476-105":30,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":21,"is_downloadable":21,"audit_status":21,"page_count":22,"language":23,"language_code":24,"site_id":25,"html_lang":24,"table_of_contents":26,"faqs":27,"seo_title":13,"seo_description":14,"update_tm":28,"read_time":29},86476,1099514068035,"Ezra","https://ap-avatar.wpscdn.com/davatar_276721f389ce27ea32af1340a28f341c",8,"Research & Report","Which Neurons Detect Malicious Code? A Probing Study of LLM Security Knowledge","Large language models increasingly generate and understand source code, yet their internal mechanisms for recognizing malicious or vulnerable patterns remain poorly explained. This study probes feed-forward network (FFN) neurons encoding LLM malware detection and validates neuron attributions using causal interventions. Experiments on Llama3 1-8B, Mistralv0.3-7B, and Qwen2.5-7B use 1,500 malicious and 1,500 benign PyPI packages to identify facilitating and inhibiting neurons and compare guardrail mechanisms across models.","arXiv :2607 . 1022 1v 1 [ cs . SE] 11 Jul 2026  \nWhich Neurons Detect Malicious Code? A Probing Study of LLM Security Knowledge  \nLam D. Dao \\#  RMIT University Vietnam Vang T. Nguyen \\#  Hanoi University, Vietnam Anh M. T. Bui \\#   \nSOICT, Hanoi University of Science and Technology  \nPhuong T. Nguyen \\#   \nUniversità degli studi dell’Aquila, Italy  \n~~ Abstract  ~~Background. Large language models (LLMs) have become increasingly capable of understanding and generating source code, leading to their widespread adoption in software engineering tasks such as code completion, repair, and vulnerability detection. However, despite their strong empirical performance, the internal mechanisms through which LLMs recognize malicious or vulnerable code patterns remain poorly understood.  \nAim. We investigated where the malware detection behavior is encoded inside LLMs Feed Forward Network (FFN) neurons and verified the attribution with causal interventions on the neurons identified. This aims to identify the most important neurons in detecting malicious code. Methods. We applied mechanistic interpretability methods to locate the neurons being responsible for malware-detection behavior in three instruction-tuned LLMs: Llama3 . 1-8B-Instruct, Mistralv0.3-7B-Instruct, and Qwen2 .5-7B-Instruct. Using 1,500 malicious and 1,500 benign PyPI packages from the PyPI Malregistry, we attribute the behavior to a set of neurons.  \nResults. The experimental results reveal that amplifying facilitating neurons for malware detection while suppressing inhibiting ones can boost accuracy, while the reverse collapses predictions toward a single class, although the magnitude and consistency is heavily model-dependent. We demonstrated that the guardrail detection mechanism varies across models, each represents its malware detection behavior differently within its FFN layers.  \nConclusions. Probing the neurons associated with security-relevant knowledge helps us gain insights into how LLMs encode malicious programming concepts, identify potentially harmful memorized behaviors, paving the way toward more reliable defense mechanisms, such as neuron-level editing, selective unlearning, and security-aware alignment for code-focused LLMs.  \n2012 ACM Subject Classification Security and privacy → Software security engineering Keywords and phrases malicious code, probing methods, LLMs  \nDigital Object Identifier 10.4230/LIPIcs.ESEM.2016.23  \n 1  Introduction  \nLarge Language Models (LLMs) have transformed software engineering (SE) by powering advanced code-generation tools such as GitHub Copilot, or DeepSeek-Coder [4, 13] . While these models significantly boost developer productivity, they also introduce serious security risks [22, 23] . Despite their strong empirical performance, the internal mechanisms through which LLMs recognize malicious or vulnerable code patterns remain poorly understood. This lack of interpretability raises important concerns regarding trustworthiness, robustness, and security, particularly in safety-critical software development scenarios where incorrect or manipulated predictions may introduce severe risks [14, 25] . Such vulnerabilities stem  \n© Jane Open Access and Joan R. Public;  \nlicensed under Creative Commons License CC-BY 4.0  \nThe International Symposium on Empirical Software Engineering and Measurement (ESEM 2026) .  \nEditors: John Q. Open and Joan R. Access; Article No. 23; pp. 23:1–23:13  \nLeibniz International Proceedings in Informatics  \nSchloss Dagstuhl – Leibniz-Zentrum für Informatik, Dagstuhl Publishing, Germany  \n23:2 Which Neurons Detect Malicious Code? A Probing Study of LLM Security Knowledge  \nfrom the models’ tendency to memorize and reproduce harmful patterns present in their pre-training or fine-tuning data, especially from public repositories and security-related datasets [3, 11] . To mitigate this, current defense strategies primarily rely on prompt engineering, safety fine-tuning, or post-hoc filtering. However, th","cbCaidUgLX6KFAse","https://ap.wps.com/l/cbCaidUgLX6KFAse","pdf",2504242,2,1,13,"English","en",105,"# Introduction\n# Motivation\n# Objectives\n# Methods and Findings\n# Conclusions","[{\"question\":\"What does the study investigate about LLMs and malicious code?\",\"answer\":\"It investigates which FFN neurons inside code-focused LLMs encode malware-detection behavior, and how those neurons contribute to recognizing malicious or vulnerable code patterns.\"},{\"question\":\"How are neuron attributions for malware detection verified?\",\"answer\":\"The work uses mechanistic interpretability to localize responsible neurons and then verifies the attribution with causal interventions on the identified neurons.\"},{\"question\":\"What dataset is used for the probing experiments?\",\"answer\":\"The experiments use 1,500 malicious and 1,500 benign PyPI packages sourced from the PyPI Malregistry.\"}]",1784211980,33,{"code":4,"msg":31,"data":32},"ok",{"site_id":25,"language":24,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":28},"which-neurons-detect-malicious-code-a-probing-study-of-llm-security-knowledge","",{"@graph":36,"@context":85},[37,53,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,47,50],{"item":41,"name":42,"@type":43,"position":21},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":20},"https://docshare.wps.com/document/","Document",{"item":48,"name":12,"@type":43,"position":49},"https://docshare.wps.com/document/research-report/",3,{"item":51,"name":13,"@type":43,"position":52},"https://docshare.wps.com/document/which-neurons-detect-malicious-code-a-probing-study-of-llm-security-knowledge/86476/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":24,"description":14,"dateModified":61,"datePublished":62,"encodingFormat":60,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":41,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-24","2026-07-16",true,{"@type":65,"interactionType":66,"userInteractionCount":20},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"What does the study investigate about LLMs and malicious code?","Question",{"text":75,"@type":76},"It investigates which FFN neurons inside code-focused LLMs encode malware-detection behavior, and how those neurons contribute to recognizing malicious or vulnerable code patterns.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"How are neuron attributions for malware detection verified?",{"text":80,"@type":76},"The work uses mechanistic interpretability to localize responsible neurons and then verifies the attribution with causal interventions on the identified neurons.",{"name":82,"@type":73,"acceptedAnswer":83},"What dataset is used for the probing experiments?",{"text":84,"@type":76},"The experiments use 1,500 malicious and 1,500 benign PyPI packages sourced from the PyPI Malregistry.","https://schema.org",{"og:url":51,"og:type":87,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":89,"canonical":51},"index,follow",{"doc_id":7,"site_id":25},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":21,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":20,"doc_module":4,"doc_module_name":46,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":46,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":46,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":46,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":46,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":46,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":46,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]