[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-128652-en":3,"doc-seo-128652-105":30,"detail-sidebar-cat-0-en-105":92},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":27,"seo_description":14,"update_tm":28,"read_time":29},128652,962084925782,"Ava Thompson","https://ap-avatar.wpscdn.com/davatar_9964176cb1d06d4a9deccf72a44ae3dc",8,"Research & Report","IMPACTS OF DATA PREPROCESSING AND HYPERPARAMETER OPTIMIZATION ON THE PERFORMANCE OF MACHINE LEARNING MODELS APPLIED TO INTRUSION DETECTION SYSTEMS","Modern communication networks require strong cybersecurity, and Intrusion Detection Systems (IDS) increasingly adopt machine learning to identify threats. While prior research uses preprocessing and hyperparameter optimization in IDS pipelines, it often reports choices without measuring each action’s impact on resulting threat-identification models. This study evaluates preprocessing variations and hyperparameter optimization effects using two datasets, demonstrating improved robustness and substantially lower training and testing execution times while preserving or enhancing predictive performance.","arXiv :2407 . 1 1 105v 1 [ cs .CR] 15 Jul 2024  \nIMPACTS OF DATA PREPROCESSING AND HYPERPARAMETER OPTIMIZATION ON THE PERFORMANCE OF MACHINE LEARNING MODELS APPLIED TO INTRUSION DETECTION SYSTEMS  \nMateus Guimarães Lima, Antony Carvalho, João Gabriel Álvares  \nCommand and Control Company  \nBrazillian Army  \nBrasília-DF-Brasil  \n{lima.mateus,antony.carvalho,[alvares.joao}@eb.mil.br](alvares.joao}@eb.mil.br)  \nClayton Escouper das Chagas, Ronaldo Ribeiro Goldschmidt  \nInstituto Militar de Engenharia (IME/RJ)  \nRio de Janeiro-RJ-Brasil  \n[escouper@ime.eb.br](escouper@ime.eb.br)  \nABSTRACT  \nIn the context of cybersecurity of modern communications networks, Intrusion Detection Systems (IDS) have been continuously improved, many of them incorporating machine learning (ML) techniques to identify threats. Although there are researches focused on the study of these techniques applied to IDS, the state-of-the-art lacks works concentrated exclusively on the evaluation of the impacts of data pre-processing actions and the optimization of the values of the hyperparameters of the ML algorithms in the construction of the models of threat identification. This article aims to present a study that fills this research gap. For that, experiments were carried out with two datasets, comparing attack scenarios with variations of pre-processing techniques and optimization of hyperparameters. The results confirm that the proper application of these techniques, in general, makes the generated classification models more robust and greatly reduces the execution times of these models’ training and testing processes.  \n1 Introduction  \nCommunication networks play a vital role in modern society, being present in numerous locations, from corporate environments to households. As a consequence of this scenario, protection against cyber threats is a crucial concern, as communication networks are susceptible to sophisticated attacks that seek to compromise the integrity, confidentiality, and availability of transmitted information [2] . Therefore, effective intrusion detection in these networks is fundamental for safeguarding transmitted data.  \nIntrusion Detection Systems (IDS), are traffic monitoring solutions that identify suspicious activities in the network or hosts, such as copying, modifying, or deleting applications, files, or directories. IDSs are based on signatures, anomalies, or, more recently, on anomaly detection with machine learning (ML) [13] .  \nThe ability to detect advanced threats in real-time using traditional intrusion detection techniques is limited, which is why ML has become an important tool for enhancing these techniques. ML offers a sophisticated and adaptive approach to intrusion detection, given its effectiveness against Zero-Day threats [18] . The use of ML algorithms can enable IDSsto be trained to recognize suspicious network traffic patterns, identify anomalous behaviors, and thus, detect threats that are difficult to detect by other methods.  \nMany studies have explored the incorporation of ML techniques into IDSs, in which, during their flow, pre-processing actions and optimization of hyperparameters are executed to prepare the data for ML algorithm processing. Although  \nthere is consensus that data pre-processing and hyperparameter optimization are important steps in the ML flow, existing works in the literature generally limit themselves to reporting the pre-processing actions performed and the use of optimized hyperparameters, without further investigating the impacts of each action on the generated models.  \nWithin the presented context, this work aims to answer the following research question: what impacts can pre-processing actions and hyperparameter optimization of ML algorithms have on the performance of classification models for threat identification in communication networks and also on the execution time of training and testing processes of these models?  \nFor this purpose, binary classification experiments (betwe","cbCaiuCCViDn5Lxb","https://ap.wps.com/l/cbCaiuCCViDn5Lxb","pdf",408443,1,11,"English","en",105,"# Abstract\n# Introduction\n## Intrusion Detection Systems and Machine Learning\n## Research Question and Experiment Design\n# Related Works\n## Preprocessing Stages\n## Feature Selection, Cleaning, Normalization\n## Performance and Computational-Time Challenges","[{\"question\":\"What research gap does the study address regarding IDS and ML?\",\"answer\":\"It focuses on evaluating how data preprocessing actions and hyperparameter optimization specifically impact the performance of ML classification models, rather than only reporting the selected steps and tuned values.\"},{\"question\":\"How were experiments structured in the study?\",\"answer\":\"The study conducted binary classification experiments distinguishing normal traffic from cyber-attacks, comparing scenarios with different preprocessing techniques and optimized hyperparameter settings across two datasets.\"},{\"question\":\"What were the key outcomes regarding model performance and time?\",\"answer\":\"Proper preprocessing combined with hyperparameter optimization improved robustness and, across most evaluated attack scenarios and algorithms, reduced training and testing execution times while maintaining or improving predictive metrics.\"}]","IMPACTS OF DATA PREPROCESSING AND HYPERPARAMETER OPTIMIZATION ON THE PERFORMANCE OF MACHINE LEARNING MODELS APPLIED TO INTRUSION DETECTION SYSTEMS | PDF",1786002322,28,{"code":4,"msg":31,"data":32},"ok",{"site_id":24,"language":23,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":87,"head_meta":89,"extra_data":91,"updated_unix":28},"impacts-of-data-preprocessing-and-hyperparameter-optimization-on-the-performance-of-machine-learning-models-applied-to-intrusion-detection-systems","",{"@graph":36,"@context":86},[37,54,69],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":53},"https://docshare.wps.com/document/impacts-of-data-preprocessing-and-hyperparameter-optimization-on-the-performance-of-machine-learning-models-applied-to-intrusion-detection-systems/128652/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":23,"description":14,"dateModified":62,"datePublished":63,"encodingFormat":61,"isAccessibleForFree":64,"interactionStatistic":65},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":41,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-23","2026-08-06",true,{"@type":66,"interactionType":67,"userInteractionCount":20},"InteractionCounter",{"@type":68},"ViewAction",{"@type":70,"mainEntity":71},"FAQPage",[72,78,82],{"name":73,"@type":74,"acceptedAnswer":75},"What research gap does the study address regarding IDS and ML?","Question",{"text":76,"@type":77},"It focuses on evaluating how data preprocessing actions and hyperparameter optimization specifically impact the performance of ML classification models, rather than only reporting the selected steps and tuned values.","Answer",{"name":79,"@type":74,"acceptedAnswer":80},"How were experiments structured in the study?",{"text":81,"@type":77},"The study conducted binary classification experiments distinguishing normal traffic from cyber-attacks, comparing scenarios with different preprocessing techniques and optimized hyperparameter settings across two datasets.",{"name":83,"@type":74,"acceptedAnswer":84},"What were the key outcomes regarding model performance and time?",{"text":85,"@type":77},"Proper preprocessing combined with hyperparameter optimization improved robustness and, across most evaluated attack scenarios and algorithms, reduced training and testing execution times while maintaining or improving predictive metrics.","https://schema.org",{"og:url":52,"og:type":88,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":90,"canonical":52},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":93},[94,98,102,106,111,116,121,124,129,132,136],{"id":20,"doc_module":4,"doc_module_name":46,"category_name":95,"show_sort_weight":96,"slug":97},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":99,"show_sort_weight":100,"slug":101},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":46,"category_name":103,"show_sort_weight":104,"slug":105},"Exam",70,"exam",{"id":107,"doc_module":4,"doc_module_name":46,"category_name":108,"show_sort_weight":109,"slug":110},5,"Comic",60,"comic",{"id":112,"doc_module":4,"doc_module_name":46,"category_name":113,"show_sort_weight":114,"slug":115},6,"Technology",50,"technology",{"id":117,"doc_module":4,"doc_module_name":46,"category_name":118,"show_sort_weight":119,"slug":120},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":122,"slug":123},30,"research-report",{"id":125,"doc_module":4,"doc_module_name":46,"category_name":126,"show_sort_weight":127,"slug":128},9,"Religion & Spirituality",20,"religion-spirituality",{"id":127,"doc_module":4,"doc_module_name":46,"category_name":130,"show_sort_weight":127,"slug":131},"World Cup","world-cup",{"id":133,"doc_module":4,"doc_module_name":46,"category_name":134,"show_sort_weight":133,"slug":135},10,"Lifestyle","lifestyle",{"id":137,"doc_module":4,"doc_module_name":46,"category_name":138,"show_sort_weight":107,"slug":139},19,"General","general"]