[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-126467-en":3,"doc-seo-126467-105":30,"detail-sidebar-cat-0-en-105":92},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":11,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":27,"seo_description":14,"update_tm":28,"read_time":29},126467,962084925290,"Ophelia","https://ap-avatar.wpscdn.com/davatar_085a072bc5b1113ac321206ff7593b45",8,"Research & Report","Log-Based Anomaly Detection - Comparative Study of Real-World System Logs using Machine Learning And Deep Learning Approaches","The reliability and security of modern smart and autonomous systems depend on effective anomaly detection, yet industrial logs are massive, noisy, unstructured, and rarely labeled, making manual inspection impractical. This thesis explores machine learning and deep learning approaches for anomaly detection in real-world system logs from an intelligent autonomous display device. It first compares ML and DL methods using a small manually labeled dataset to assess accuracy and computational efficiency. It then evaluates advanced deep learning under weak supervision, semi-supervision, and unsupervised learning, benchmarking against fully supervised baselines. Finally, it provides guidelines for selecting learning strategies based on label availability, data quality, and deployment constraints for scalable, robust industrial IoT monitoring.","Log-Based Anomaly Detection: Comparative Study of Real-World System Logs using Machine Learning And Deep Learning Approaches  \nNadira Anjum Nipa  \nA Thesis  \nin  \nThe Department  \nof  \nConcordia Institute for Information Systems Engineering  \nPresented in Partial Fulfillment of the Requirements  \nfor the Degree of  \nMaster of Applied Science (Quality Systems Engineering) at Concordia University  \nMontral, Qubec, Canada  \nJuly 2025  \n© Nadira Anjum Nipa, 2025  \nCONCORDIA UNIVERSITY  \nSchool of Graduate Studies  \nThis is to certify that the thesis prepared  \nBy: Nadira Anjum Nipa  \nEntitled: Log-Based Anomaly Detection: Comparative Study of Real-World Sys  \ntem Logs using Machine Learning And Deep Learning Approaches  \nand submitted in partial fulfillment of the requirements for the degree of  \nMaster of Applied Science (Quality Systems Engineering)  \ncomplies with the regulations of this University and meets the accepted standards with respect to originality and quality.  \nSigned by the Final Examining Committee:  \n  Chair & Examiner Dr. Jeremy Clark  \n  Examiner  \nDr. Honghao Fu  \nDr. Zachary Patterson  Supervisor Dr. Nizar Bouguila  Supervisor  \nApproved by  Dr. Chun Wang, Chair  Department of Concordia Institute for Information Systems Engineering  \n~~ ~~ 2025  Dr. Mourad Debbabi, Dean   \nFaculty of Engineering and Computer Science  \nAbstract  \nLog-Based Anomaly Detection: Comparative Study of Real-World System Logs using Machine Learning And Deep Learning Approaches  \nNadira Anjum Nipa  \nThe reliability and security of today’s smart and autonomous systems increasingly rely on effective anomaly detection capabilities. Logs generated by intelligent devices during runtime offer valuable insights for system monitoring and troubleshooting. Nonetheless, the enormous quantity and complexity of these logs render manual anomaly inspection impractical and error-prone. To address this, various automated log-based anomaly detection methods have been developed. However, many of these approaches are evaluated in controlled environments with publicly available datasets, which differ significantly from the noisy, unstructured, and unlabeled logs encountered in industrial settings. This thesis explores and adapts existing machine learning and deep learning techniques for anomaly detection in real-world system logs produced by an intelligent autonomous display device. Initially, we conduct a comparative analysis of machine learning and deep learning methods using a small manually labeled dataset to evaluate the detection accuracy and computational efficiency. Our results highlight the most suitable approaches for enabling proactive maintenance and enhancing system reliability. Expanding on this, we evaluate advanced deep learning methods across weakly supervised, semi-supervised, and unsupervised learning paradigms, using heuristically labeled logsand benchmark them against fully supervised baselines to examine the trade-offs between label dependency, detection performance, and industrial applicability. Finally, we propose a systematic approach for managing unlabeled and noisy log data, providing practical guidelines for selecting suitable learning strategies based on label availability, data quality, and real-world constraints. The findings of this work provide valuable insights for the implementation of scalable, accurate, and robust log-based anomaly detection in industrial IoT environments.  \nAcknowledgments  \nFirst and foremost, I would like to express my heartfelt gratitude to my supervisors, Prof. Nizar Bouguila and Prof. Zachary Patterson, for their guidance, support, and encouragement throughout my studies. Their patience, insight, and thoughtful feedback have been a constant source of motivation and have played a crucial role in shaping this work.  \nI am also truly thankful to the Mitacs Accelerate Program for providing me with the opportunity to collaborate with Buspas Inc. It was a valuable experience that allowed me to apply my ","cbCain8bcmjiurB8","https://ap.wps.com/l/cbCain8bcmjiurB8","pdf",1226931,1,60,"English","en",105,"# Introduction\n## Contributions\n## Thesis Organization\n# Literature Review\n## Traditional Machine Learning-Based Approaches\n## Deep Learning-Based Approaches\n## Dataset Usage and Real-World Challenges\n## Summary and Research Gap\n# A Comparative Study of Log-Based Anomaly Detection Methods in Real-World System Logs\n## Common Framework\n## Log Parsing\n## Log Grouping\n## Log Representation\n## Anomaly Detection","[{\"question\":\"Why is anomaly detection on real-world system logs challenging?\",\"answer\":\"Industrial logs are extremely large and complex, often noisy, unstructured, and unlabeled, which makes manual inspection slow, error-prone, and inefficient.\"},{\"question\":\"How does the thesis compare machine learning and deep learning methods?\",\"answer\":\"It conducts a comparative analysis using a small manually labeled dataset to evaluate detection accuracy and computational efficiency, identifying the most suitable approaches.\"},{\"question\":\"Which learning paradigms are evaluated for deep learning-based anomaly detection?\",\"answer\":\"The thesis evaluates advanced deep learning methods under weakly supervised, semi-supervised, and unsupervised learning paradigms and compares them with fully supervised baselines to study trade-offs.\"}]","Log-Based Anomaly Detection - Comparative Study of Real-World System Logs using Machine Learning And Deep Learning Approaches | PDF",1785905206,151,{"code":4,"msg":31,"data":32},"ok",{"site_id":24,"language":23,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":87,"head_meta":89,"extra_data":91,"updated_unix":28},"log-based-anomaly-detection-comparative-study-of-real-world-system-logs-using-machine-learning-and-deep-learning-approaches","",{"@graph":36,"@context":86},[37,54,69],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":53},"https://docshare.wps.com/document/log-based-anomaly-detection-comparative-study-of-real-world-system-logs-using-machine-learning-and-deep-learning-approaches/126467/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":23,"description":14,"dateModified":62,"datePublished":63,"encodingFormat":61,"isAccessibleForFree":64,"interactionStatistic":65},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":41,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-23","2026-08-05",true,{"@type":66,"interactionType":67,"userInteractionCount":11},"InteractionCounter",{"@type":68},"ViewAction",{"@type":70,"mainEntity":71},"FAQPage",[72,78,82],{"name":73,"@type":74,"acceptedAnswer":75},"Why is anomaly detection on real-world system logs challenging?","Question",{"text":76,"@type":77},"Industrial logs are extremely large and complex, often noisy, unstructured, and unlabeled, which makes manual inspection slow, error-prone, and inefficient.","Answer",{"name":79,"@type":74,"acceptedAnswer":80},"How does the thesis compare machine learning and deep learning methods?",{"text":81,"@type":77},"It conducts a comparative analysis using a small manually labeled dataset to evaluate detection accuracy and computational efficiency, identifying the most suitable approaches.",{"name":83,"@type":74,"acceptedAnswer":84},"Which learning paradigms are evaluated for deep learning-based anomaly detection?",{"text":85,"@type":77},"The thesis evaluates advanced deep learning methods under weakly supervised, semi-supervised, and unsupervised learning paradigms and compares them with fully supervised baselines to study trade-offs.","https://schema.org",{"og:url":52,"og:type":88,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":90,"canonical":52},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":93},[94,98,102,106,110,115,120,123,128,131,135],{"id":20,"doc_module":4,"doc_module_name":46,"category_name":95,"show_sort_weight":96,"slug":97},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":99,"show_sort_weight":100,"slug":101},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":46,"category_name":103,"show_sort_weight":104,"slug":105},"Exam",70,"exam",{"id":107,"doc_module":4,"doc_module_name":46,"category_name":108,"show_sort_weight":21,"slug":109},5,"Comic","comic",{"id":111,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":46,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":46,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":46,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":46,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":46,"category_name":137,"show_sort_weight":107,"slug":138},19,"General","general"]