[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-118260-en":3,"doc-seo-118260-105":30,"detail-sidebar-cat-0-en-105":95},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":4,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":27,"seo_description":14,"update_tm":28,"read_time":29},118260,687197207639,"Asher","https://ap-avatar.wpscdn.com/davatar_a8503ba1806abce46bf441b54a3ca4cd",8,"Research & Report","Explaining the Contributing Factors for Vulnerability Detection in Machine Learning","Increasingly, vulnerabilities are mined from software repositories and detected automatically using machine learning, yet the core research gap remains: how mining and learning factors affect accuracy across software projects with different characteristics. This work examines how combining vulnerability features with three representative ML models influences detection in 17 real-world projects. Two vulnerability representations are compared: NLP-derived code features using multiple tokenization and embeddings, and eight architectural metrics capturing system design. Results show a recommended bag-of-words baseline with random forests and a limitation of signature transferability across domains.","Explaining the Contributing Factors for Vulnerability Detection in Machine Learning  \nEsma Mouinea , Yan Liua , Lu Xiaob , Rick Kazmanc and Xiao Wangb  \na Concordia University, 1455 De Maisonneuve Blvd. W. Montreal, Qc, Canada, H3G 1M8 b Stevens Institute of Technology, Castle Point Terrace, Hoboken, NJ 07030, United States c University of Hawaii, 2500 Campus Rd, Honolulu, HI 96822, United States  \narXiv :2406 .03577v 1 [ cs . SE] 5 Jun 2024  \nARTICLE INFO  \nKeywords:  \nsoftware vulnerability, machine learning, natural language processing, architectural metrics  \nAB STRACT  \nThere is an increasing trend to mine vulnerabilities from software repositories and use machine learning techniques to automatically detect software vulnerabilities. A fundamental but unresolved research question is: how do different factors in the mining and learning process impact the accuracy of identifying vulnerabilities in software projects of varying characteristics? Substantial research has been dedicated in this area, including source code static analysis, software repository mining, and NLP-based machine learning. However, practitioners lack experience regarding the key factors for building a baseline model of the state-of-the-art. In addition, there lacks of experience regarding the transferability of the vulnerability signatures from project to project. This study investigates how the combination of different vulnerability features and three representative machine learning models impact the accuracy of vulnerability detection in 17 real-world projects. We examine two types of vulnerability representations: 1) code features extracted through NLP with varying tokenization strategies and three different embedding techniques (bag-of-words, word2vec, and fastText) and 2) a set of eight architectural metrics that capture the abstract design of the software systems. The three machine learning algorithms include a random forest model, a support vector machines model, and a residual neural network model. Overall, 95% of the learning metrics (precision, recall, and f1 score, etc.) are above 0.77 in the experiments out of 10 hypothesis tests and 408 experiments. Further analysis shows a recommended baseline model with signatures extracted through bag-of-words embedding, combined with the random forest, consistently increases the detection accuracy by about 4% compared to other combinations in all 17 projects. Furthermore, we observe the limitation of transferring vulnerability signatures across domains based on our experiments.  \n1. INTRODUCTION  \nThe National Institute of Standards and Technology (NIST) defines security vulnerability as a weakness in an information system, system security procedures, internal controls, or implementation that could be exploited or triggered by a threat source [1] . Software vulnerability management is the practice of identifying, classifying, remediating, and mitigating vulnerabilities. Early detection of vulnerable code reduces the risks of run-time errors, faults, threats, and the collapse of a system. As software scales expand, vulnerability detection with sufficient accuracy and efficiency remains a challenge from both research [2, 3, 4, 5] and industrial perspectives [6, 7] . The goal is to learn from representations of vulnerable features and to automate the discovery of vulnerabilities in source code.  \nIn industrial practice, security flaws are regularly reported to the Common Vulnerabilities and Exposures (CVE) database [8] . This database is used to collect and share publicly disclosed information about security vulnerabilities. Likewise, Common Weakness Enumeration (CWE) is a communitydeveloped list of common software and hardware security weaknesses [9] . The Open Web Application Security Project (OWASP) Benchmark is a Java test suite that contains thou-  \n[e_mouine@encs.concordia.ca](e_mouine@encs.concordia.ca) (E. Mouine); [yan.liu@concordia.ca](yan.liu@concordia.ca)[ ](yan.liu@concordia.ca)(Y. Liu); [lxiao6","cbCairYWtlB0eAIV","https://ap.wps.com/l/cbCairYWtlB0eAIV","pdf",757184,1,16,"English","en",105,"# Introduction\n## Security vulnerability background and datasets\n## Static code analysis and its limitations\n## Feature engineering for vulnerability detection\n# Method overview\n## Vulnerability representations and ML models\n## Evaluation setup and metrics\n# Results and analysis\n## Recommended baseline model\n## Transferability limitations","[{\"question\":\"What is the main research question of the study?\",\"answer\":\"How different factors in the mining and learning process impact the accuracy of vulnerability identification across software projects with varying characteristics.\"},{\"question\":\"Which vulnerability representations are investigated?\",\"answer\":\"The study evaluates NLP-extracted code features with varying tokenization and embedding techniques, and eight architectural metrics representing software system design.\"},{\"question\":\"Which machine learning models are compared?\",\"answer\":\"A random forest model, a support vector machines model, and a residual neural network model are used to assess detection accuracy.\"},{\"question\":\"What does the study conclude about the transferability of vulnerability signatures?\",\"answer\":\"Experiments indicate a limitation: vulnerability signatures do not transfer reliably across domains.\"}]","Explaining the Contributing Factors for Vulnerability Detection in Machine Learning | PDF",1785682698,40,{"code":4,"msg":31,"data":32},"ok",{"site_id":24,"language":23,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":90,"head_meta":92,"extra_data":94,"updated_unix":28},"explaining-the-contributing-factors-for-vulnerability-detection-in-machine-learning","",{"@graph":36,"@context":89},[37,54,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":53},"https://docshare.wps.com/document/explaining-the-contributing-factors-for-vulnerability-detection-in-machine-learning/118260/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":23,"description":14,"dateModified":62,"datePublished":62,"encodingFormat":61,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":41,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-02",true,{"@type":65,"interactionType":66,"userInteractionCount":4},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81,85],{"name":72,"@type":73,"acceptedAnswer":74},"What is the main research question of the study?","Question",{"text":75,"@type":76},"How different factors in the mining and learning process impact the accuracy of vulnerability identification across software projects with varying characteristics.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"Which vulnerability representations are investigated?",{"text":80,"@type":76},"The study evaluates NLP-extracted code features with varying tokenization and embedding techniques, and eight architectural metrics representing software system design.",{"name":82,"@type":73,"acceptedAnswer":83},"Which machine learning models are compared?",{"text":84,"@type":76},"A random forest model, a support vector machines model, and a residual neural network model are used to assess detection accuracy.",{"name":86,"@type":73,"acceptedAnswer":87},"What does the study conclude about the transferability of vulnerability signatures?",{"text":88,"@type":76},"Experiments indicate a limitation: vulnerability signatures do not transfer reliably across domains.","https://schema.org",{"og:url":52,"og:type":91,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":93,"canonical":52},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":96},[97,101,105,109,114,119,123,126,131,134,138],{"id":20,"doc_module":4,"doc_module_name":46,"category_name":98,"show_sort_weight":99,"slug":100},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":102,"show_sort_weight":103,"slug":104},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":46,"category_name":106,"show_sort_weight":107,"slug":108},"Exam",70,"exam",{"id":110,"doc_module":4,"doc_module_name":46,"category_name":111,"show_sort_weight":112,"slug":113},5,"Comic",60,"comic",{"id":115,"doc_module":4,"doc_module_name":46,"category_name":116,"show_sort_weight":117,"slug":118},6,"Technology",50,"technology",{"id":120,"doc_module":4,"doc_module_name":46,"category_name":121,"show_sort_weight":29,"slug":122},7,"Healthcare","healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":124,"slug":125},30,"research-report",{"id":127,"doc_module":4,"doc_module_name":46,"category_name":128,"show_sort_weight":129,"slug":130},9,"Religion & Spirituality",20,"religion-spirituality",{"id":129,"doc_module":4,"doc_module_name":46,"category_name":132,"show_sort_weight":129,"slug":133},"World Cup","world-cup",{"id":135,"doc_module":4,"doc_module_name":46,"category_name":136,"show_sort_weight":135,"slug":137},10,"Lifestyle","lifestyle",{"id":139,"doc_module":4,"doc_module_name":46,"category_name":140,"show_sort_weight":110,"slug":141},19,"General","general"]