[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-118609-en":3,"doc-seo-118609-105":30,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":4,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":27,"seo_description":14,"update_tm":28,"read_time":29},118609,1374391974585,"Genevieve","https://ap-avatar.wpscdn.com/davatar_276721f389ce27ea32af1340a28f341c",8,"Research & Report","Pitfalls in Machine Learning for Computer Security - Actionable Recommendations and Research Directions","Machine learning for computer security can deliver major breakthroughs, but it is also vulnerable to subtle design, implementation, and evaluation pitfalls that erode performance and limit practical deployment. The paper critically identifies 10 common pitfalls, studies their prevalence across 30 top-tier security papers from the past decade, and shows how individual pitfalls can produce unrealistic results and misleading interpretations. It further proposes actionable recommendations, analyzes impact across multiple security domains, and highlights open research problems.","research highlights  \nDOI:10 . 1145/3643456  \nPitfalls in  \nTo view the accompanying Technical Perspective, visit [doi.acm.org/10.1145/3655635](doi.acm.org/10.1145/3655635)  \ntp  \nMachine Learning for Computer Security  \nBy Daniel Arp, Erwin Quiring, Feargus Pendlebury, Alexander Warnecke, Fabio Pierazzi, Christian Wressnegger, Lorenzo Cavallaro, and Konrad Rieck  \nAbstract  \nWith the growing processing power of computing systems and the increasing availability of massive datasets, machinelearning (ML) algorithms have led to major breakthroughsin many different areas. This development has influenced computer security, spawning a series of work on learningbased security systems, such as for malware detection, vulnerability discovery, and binary code analysis. Despite great potential, ML in security is prone to subtle pitfalls that undermine its performance and render learning-based systems potentially unsuitable for security tasks and practical deployment.  \nIn this paper, we look at this problem with critical eyes. First, we identify common pitfalls in the design, implementation, and evaluation of learning-based security systems. We conduct a study of 30 papers from top-tier security conferences within the past 10 years, confirming that these pitfalls are widespread in the current security literature. In an empirical analysis, we further demonstrate how individual pitfalls can lead to unrealistic performance and interpretations, obstructing the understanding of the security problem at hand. As a remedy, we propose actionable recommendations to support researchers in avoiding or mitigating the pitfalls where possible. Furthermore, we identify open problems when applying ML in security and provide directions for further research.  \n1. INTRODUCTION  \nNo day goes by without reading machine-learning success stories. The widespread access to specialized computational resources and large datasets, along with novel concepts and architectures for deep learning, have paved the way for ML breakthroughs in several areas, such asthe translation of natural languages22 and the recognition of image content.14 This development has naturally influenced security research: Although mostly confined to specific applications in the past, ML has become one of the key enablers to studying and addressing security-relevant problems at large in several application domains, including intrusion detection,17 malware analysis,11 vulnerability discovery,25 and binary code analysis.20  \nMachine learning, however, has no clairvoyant abilities and requires reasoning about statistical properties of data across a fairly delicate workflow: Incorrect assumptions and experimental biases may cast doubts on this process to the extent that it becomes unclear whether we can trust scientific discoveries made using learning algorithms at all. Attempts to identify such challenges and limitations in specific security domains, such as network intrusion detection, started two decades ago5 and were extended more recently to other domains.12,18 Orthogonal to this line of work, however, we argue that there exist generic pitfalls related to machine learning that affect all security domains and have received little attention so far.  \nThese pitfalls can lead to over-optimistic results and, even worse, affect the entire ML workflow, weakening assumptions, conclusions, and lessons learned. As a consequence, a false sense of achievement is felt that hinders the adoption of research advances in academia and industry. A sound scientific methodology is fundamental to support intuitions and draw conclusions. We argue that this need is especially relevant in security, where processes are often undermined by adversaries that actively aim to bypass analysis and break systems.  \nIn this paper, we identify 10 common—yet subtle—pitfalls that pose a threat to validity and hinder interpretation of research results. To support this claim, we analyze the prevalence of these pitfalls in 30 top-tier ","cbCaiu4sl1SCj9VZ","https://ap.wps.com/l/cbCaiu4sl1SCj9VZ","pdf",1222246,1,9,"English","en",105,"# Introduction\n## Pitfall Identification and Scope\n## Prevalence Analysis\n## Impact Analysis\n## Summary of Contributions","[{\"question\":\"What are the main pitfalls addressed in the paper?\",\"answer\":\"The paper identifies 10 subtle pitfalls across the design, implementation, and evaluation of learning-based security systems, describing them as do-nots and providing corresponding do’s as recommendations.\"},{\"question\":\"How do the authors validate that these pitfalls are widespread?\",\"answer\":\"They analyze 30 papers from top-tier security conferences over the past decade and find that each paper suffers from at least three pitfalls, with several affecting most of the papers.\"},{\"question\":\"What impact do the pitfalls have on reported security results?\",\"answer\":\"The paper demonstrates that specific pitfalls can lead to unrealistic performance and interpretations, introducing experimental bias that obstructs understanding of the underlying security problem.\"}]","Pitfalls in Machine Learning for Computer Security - Actionable Recommendations and Research Directions | PDF",1785684495,23,{"code":4,"msg":31,"data":32},"ok",{"site_id":24,"language":23,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":28},"pitfalls-in-machine-learning-for-computer-security-actionable-recommendations-and-research-directions","",{"@graph":36,"@context":85},[37,54,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":53},"https://docshare.wps.com/document/pitfalls-in-machine-learning-for-computer-security-actionable-recommendations-and-research-directions/118609/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":23,"description":14,"dateModified":62,"datePublished":62,"encodingFormat":61,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":41,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-02",true,{"@type":65,"interactionType":66,"userInteractionCount":4},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"What are the main pitfalls addressed in the paper?","Question",{"text":75,"@type":76},"The paper identifies 10 subtle pitfalls across the design, implementation, and evaluation of learning-based security systems, describing them as do-nots and providing corresponding do’s as recommendations.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"How do the authors validate that these pitfalls are widespread?",{"text":80,"@type":76},"They analyze 30 papers from top-tier security conferences over the past decade and find that each paper suffers from at least three pitfalls, with several affecting most of the papers.",{"name":82,"@type":73,"acceptedAnswer":83},"What impact do the pitfalls have on reported security results?",{"text":84,"@type":76},"The paper demonstrates that specific pitfalls can lead to unrealistic performance and interpretations, introducing experimental bias that obstructs understanding of the underlying security problem.","https://schema.org",{"og:url":52,"og:type":87,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":89,"canonical":52},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,127,130,134],{"id":20,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":46,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":46,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":21,"doc_module":4,"doc_module_name":46,"category_name":124,"show_sort_weight":125,"slug":126},"Religion & Spirituality",20,"religion-spirituality",{"id":125,"doc_module":4,"doc_module_name":46,"category_name":128,"show_sort_weight":125,"slug":129},"World Cup","world-cup",{"id":131,"doc_module":4,"doc_module_name":46,"category_name":132,"show_sort_weight":131,"slug":133},10,"Lifestyle","lifestyle",{"id":135,"doc_module":4,"doc_module_name":46,"category_name":136,"show_sort_weight":106,"slug":137},19,"General","general"]