[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-128193-en":3,"doc-seo-128193-105":31,"detail-sidebar-cat-0-en-105":96},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":21,"is_downloadable":21,"audit_status":21,"page_count":22,"language":23,"language_code":24,"site_id":25,"html_lang":24,"table_of_contents":26,"faqs":27,"seo_title":28,"seo_description":14,"update_tm":29,"read_time":30},128193,2336475104042,"Skyler","https://ap-avatar.wpscdn.com/avatar/22000c4c32af1715be0?x-image-process=image/resize,m_fixed,w_180,h_180&k=1786537525561427321",8,"Research & Report","Extending Machine Learning of Cyberattack Strategies with Continuous Transition Rates","Cyberattacks are an increasing threat to organizations across industries. The dissertation focuses on Petri Nets with Players, Strategies, and Costs (PNPSC), a formalism built to model cyberattacks through attacker–defender strategy representation. It introduces a reinforcement learning approach tailored to PNPSC nets, combining game tree and deep reinforcement learning to improve agent learning performance. The method supports continuous transition rates, trains agents for composite models and multi-learner environments, and compares results with Monte Carlo reinforcement learning. It also surveys and validates more accurate transition rates for PNPSC.","University of Alabama in Huntsville  \nLOUIS  \n\n| Dissertations | UAH Electronic Theses and Dissertations |\n| --- | --- |\n| 2023\u003Cbr>Extending machine learning of cyberattack strategies with continuous transition rates\u003Cbr>Edwin Michael Bearss\u003Cbr>Follow this and additional works at: [https://louis.uah.edu/uah-dissertations](https://louis.uah.edu/uah-dissertations) |  |\n\nRecommended Citation  \nBearss, Edwin Michael, \"Extending machine learning of cyberattack strategies with continuous transition rates\" (2023) . Dissertations. 348.  \n[https://louis.uah.edu/uah-dissertations/348](https://louis.uah.edu/uah-dissertations/348)  \nThis Dissertation is brought to you for free and open access by the UAH Electronic Theses and Dissertations at LOUIS. It has been accepted for inclusion in Dissertations by an authorized administrator of LOUIS.  \nExtending Machine Learning of Cyberattack Strategies with Continuous Transition Rates  \nEdwin Michael Bearss  \nA DISSERTATION  \nSubmitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy  \nin  \nThe Department of Computer Science  \nto  \nThe Graduate School  \nof  \nThe University of Alabama in Huntsville  \nAugust 2023  \nApproved by:  \nDr. Mikel Petty, Research Advisor/Committee Chair Dr. John Bland, Committee Member  \nDr. Letha Etzkorn, Committee Member  \nDr. Vineetha Menon, Committee Member Dr. Tathagata Mukherjee, Committee Member Dr. Letha Etzkorn, Department Chair  \nDr. Rainer Steinwandt, College Dean Dr. Jon Hakkila, Graduate Dean  \nAbstract  \nExtending Machine Learning of Cyberattack Strategies with Continuous Transition Rates  \nEdwin Michael Bearss  \nA dissertation submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy  \nComputer Science  \nThe University of Alabama in Huntsville  \nAugust 2023  \nCyberattacks are a growing threat to organizations of all sizes and industries. By better understanding these attacks, better ways to defend against them can be developed. Petri Nets with Players, Strategies, and Costs (PNPSC) is an extension of Petri nets specifically designed to model cyberattacks. The PNPSC formalism includes a representation of the strategies used by the competing players, i. e. , attacker and defender, to achieve their goals. This formalism has been the basis for along-running research program consisting of several interconnected research projects. Projects within that program include automatically generating PNPSC nets from the MITRE Common Attack Pattern Enumeration and Classification (CAPEC) database of cyberattack patterns, verification and validation of the models using several complementary methods, composing multiple PNPSC nets into models of realistic computer systems, and using machine learning to improve the strategies of players present in the formalism. This work describes a novel method of reinforcement learning tailored to PNPSC nets. A combination of game tree and deep reinforcement learning algorithms is used to significantly boost the learning performance of the agents used to improve player strategies in PNPSC nets. Two different deep reinforcement learning  \nalgorithms were used to improve the strategies of players present in the PNPSC formalism. These algorithms make use of function approximation that allows them to work effectively even when continuous transition rates are used. In addition to the existing component PNPSC nets, these algorithms were trained to improve strategies of players for composite models consisting of multiple component models, environments with more than one learner present, and models integrating a representation of the system user. The performance of these algorithms is also compared to existing work using Monte Carlo reinforcement learning methods. This work also includes a survey effort used to collect and validate more accurate transition rates used in the PNPSC nets.  \niv  \nAcknowledgements  \nI want to take this opportunity to express my sincere gratitude to my resear","cbCaieMC9XUAYN2E","https://ap.wps.com/l/cbCaieMC9XUAYN2E","pdf",7971011,3,1,264,"English","en",105,"# Abstract\n# Acknowledgements\n# Table of Contents\n# List of Figures\n# List of Tables\n# Epigraph\n# Chapter 1. Introduction\n## Research Program\n## Research Questions\n## Structure of This Dissertation\n# Chapter 2. Background\n## Petri Nets\n## PNPSC Net Formalism\n## PNPSC Research Program\n## Generating Cyberattack Component Models\n## Adding Defenders and Users to Cyberattack Component Models\n## Selecting and Composing Cyberattack Models\n## Verification and Validation of Ex","[{\"question\":\"What is the main focus of the dissertation?\",\"answer\":\"The dissertation develops a reinforcement learning method tailored to PNPSC nets to improve cyberattack strategy learning, including when continuous transition rates are used.\"},{\"question\":\"What is PNPSC and how is it used?\",\"answer\":\"PNPSC is an extension of Petri nets designed to model cyberattacks by representing competing players (attacker and defender) along with their strategies and costs.\"},{\"question\":\"Which learning approach is proposed for PNPSC nets?\",\"answer\":\"The work combines game tree methods with deep reinforcement learning to significantly boost learning performance of agents that improve player strategies in PNPSC.\"},{\"question\":\"How are the results validated and compared?\",\"answer\":\"Performance is compared to existing Monte Carlo reinforcement learning methods, and the dissertation includes a survey effort to collect and validate more accurate transition rates for PNPSC.\"}]","Extending Machine Learning of Cyberattack Strategies with Continuous Transition Rates | PDF",1785945449,665,{"code":4,"msg":32,"data":33},"ok",{"site_id":25,"language":24,"slug":34,"title":13,"keywords":35,"description":14,"schema_data":36,"social_meta":91,"head_meta":93,"extra_data":95,"updated_unix":29},"extending-machine-learning-of-cyberattack-strategies-with-continuous-transition-rates","",{"@graph":37,"@context":90},[38,54,69],{"@type":39,"itemListElement":40},"BreadcrumbList",[41,45,49,51],{"item":42,"name":43,"@type":44,"position":21},"https://docshare.wps.com","Home","ListItem",{"item":46,"name":47,"@type":44,"position":48},"https://docshare.wps.com/document/","Document",2,{"item":50,"name":12,"@type":44,"position":20},"https://docshare.wps.com/document/research-report/",{"item":52,"name":13,"@type":44,"position":53},"https://docshare.wps.com/document/extending-machine-learning-of-cyberattack-strategies-with-continuous-transition-rates/128193/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":24,"description":14,"dateModified":62,"datePublished":63,"encodingFormat":61,"isAccessibleForFree":64,"interactionStatistic":65},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":42,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-28","2026-08-05",true,{"@type":66,"interactionType":67,"userInteractionCount":20},"InteractionCounter",{"@type":68},"ViewAction",{"@type":70,"mainEntity":71},"FAQPage",[72,78,82,86],{"name":73,"@type":74,"acceptedAnswer":75},"What is the main focus of the dissertation?","Question",{"text":76,"@type":77},"The dissertation develops a reinforcement learning method tailored to PNPSC nets to improve cyberattack strategy learning, including when continuous transition rates are used.","Answer",{"name":79,"@type":74,"acceptedAnswer":80},"What is PNPSC and how is it used?",{"text":81,"@type":77},"PNPSC is an extension of Petri nets designed to model cyberattacks by representing competing players (attacker and defender) along with their strategies and costs.",{"name":83,"@type":74,"acceptedAnswer":84},"Which learning approach is proposed for PNPSC nets?",{"text":85,"@type":77},"The work combines game tree methods with deep reinforcement learning to significantly boost learning performance of agents that improve player strategies in PNPSC.",{"name":87,"@type":74,"acceptedAnswer":88},"How are the results validated and compared?",{"text":89,"@type":77},"Performance is compared to existing Monte Carlo reinforcement learning methods, and the dissertation includes a survey effort to collect and validate more accurate transition rates for PNPSC.","https://schema.org",{"og:url":52,"og:type":92,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":94,"canonical":52},"index,follow",{"doc_id":7,"site_id":25},{"code":4,"msg":5,"data":97},[98,102,106,110,115,120,125,128,133,136,140],{"id":21,"doc_module":4,"doc_module_name":47,"category_name":99,"show_sort_weight":100,"slug":101},"Story & Novel",90,"story-novel",{"id":48,"doc_module":4,"doc_module_name":47,"category_name":103,"show_sort_weight":104,"slug":105},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":47,"category_name":107,"show_sort_weight":108,"slug":109},"Exam",70,"exam",{"id":111,"doc_module":4,"doc_module_name":47,"category_name":112,"show_sort_weight":113,"slug":114},5,"Comic",60,"comic",{"id":116,"doc_module":4,"doc_module_name":47,"category_name":117,"show_sort_weight":118,"slug":119},6,"Technology",50,"technology",{"id":121,"doc_module":4,"doc_module_name":47,"category_name":122,"show_sort_weight":123,"slug":124},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":47,"category_name":12,"show_sort_weight":126,"slug":127},30,"research-report",{"id":129,"doc_module":4,"doc_module_name":47,"category_name":130,"show_sort_weight":131,"slug":132},9,"Religion & Spirituality",20,"religion-spirituality",{"id":131,"doc_module":4,"doc_module_name":47,"category_name":134,"show_sort_weight":131,"slug":135},"World Cup","world-cup",{"id":137,"doc_module":4,"doc_module_name":47,"category_name":138,"show_sort_weight":137,"slug":139},10,"Lifestyle","lifestyle",{"id":141,"doc_module":4,"doc_module_name":47,"category_name":142,"show_sort_weight":111,"slug":143},19,"General","general"]