[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-seo-141174-105":3,"detail-sidebar-cat-0-en-105":81,"doc-detail-141174-en":131},{"code":4,"msg":5,"data":6},0,"ok",{"site_id":7,"language":8,"slug":9,"title":10,"keywords":11,"description":12,"schema_data":13,"social_meta":74,"head_meta":76,"extra_data":78,"updated_unix":80},105,"en","safe-deep-reinforcement-learning-for-super-mario-bros-using-heuristics-bachelors-project-thesis","Safe Deep Reinforcement Learning for Super Mario Bros using Heuristics - Bachelor’s Project Thesis","","The thesis investigates how to embed heuristic-based safety mechanisms into Deep Reinforcement Learning to improve agent survival in Super Mario Bros, a dynamic and stochastic game environment. It evaluates heuristic-augmented Actor-Critic models and compares actor network designs (separate vs shared) and different frame-stack input configurations. Experiments quantify survival-rate gains, particularly strong performance around the 10,000th episode mark, while also analyzing how safety constraints may restrict exploration and long-term reward optimization. Results show a clear trade-off between safety and reward maximization.",{"@graph":14,"@context":73},[15,34,56],{"@type":16,"itemListElement":17},"BreadcrumbList",[18,23,27,31],{"item":19,"name":20,"@type":21,"position":22},"https://docshare.wps.com","Home","ListItem",1,{"item":24,"name":25,"@type":21,"position":26},"https://docshare.wps.com/document/","Document",2,{"item":28,"name":29,"@type":21,"position":30},"https://docshare.wps.com/document/research-report/","Research & Report",3,{"item":32,"name":10,"@type":21,"position":33},"https://docshare.wps.com/document/safe-deep-reinforcement-learning-for-super-mario-bros-using-heuristics-bachelors-project-thesis/141174/",4,{"url":32,"name":10,"@type":35,"image":36,"author":41,"headline":10,"publisher":44,"fileFormat":47,"inLanguage":8,"description":12,"dateModified":48,"datePublished":49,"encodingFormat":47,"isAccessibleForFree":50,"interactionStatistic":51},"DigitalDocument",{"url":37,"@type":38,"width":39,"height":40},"https://docshare.wps.com/thumbnails/safe-deep-reinforcement-learning-for-super-mario-bros-using-heuristics-bachelors-project-thesis/141174.png","ImageObject",300,407,{"name":42,"@type":43},"Arica Lee","Person",{"url":19,"name":45,"@type":46},"DocShare","Organization","application/pdf","2026-11-01","2026-08-25",true,{"@type":52,"interactionType":53,"userInteractionCount":55},"InteractionCounter",{"@type":54},"ViewAction",17,{"@type":57,"mainEntity":58},"FAQPage",[59,65,69],{"name":60,"@type":61,"acceptedAnswer":62},"What problem does the thesis address in Deep Reinforcement Learning for Super Mario Bros?","Question",{"text":63,"@type":64},"It addresses the need to improve agent survival by integrating safety mechanisms into Deep RL while operating in a complex, dynamic game environment.","Answer",{"name":66,"@type":61,"acceptedAnswer":67},"How does the study implement safety in the Actor-Critic framework?",{"text":68,"@type":64},"It uses heuristic-augmented Actor-Critic models, testing different configurations that influence agent decisions and survival outcomes.",{"name":70,"@type":61,"acceptedAnswer":71},"What trade-off does the research observe when adding safety constraints?",{"text":72,"@type":64},"Safety improvements can limit exploration, which may constrain long-term reward optimization, revealing tension between safety and maximizing reward.","https://schema.org",{"og:url":32,"og:type":75,"og:title":10,"og:site_name":45,"og:description":12},"article",{"robots":77,"canonical":32},"index,follow",{"doc_id":79,"site_id":7},141174,1787650183,{"code":4,"msg":82,"data":83},"success",[84,88,92,96,101,106,111,115,120,123,127],{"id":22,"doc_module":4,"doc_module_name":25,"category_name":85,"show_sort_weight":86,"slug":87},"Story & Novel",90,"story-novel",{"id":26,"doc_module":4,"doc_module_name":25,"category_name":89,"show_sort_weight":90,"slug":91},"Literature",80,"literature",{"id":33,"doc_module":4,"doc_module_name":25,"category_name":93,"show_sort_weight":94,"slug":95},"Exam",70,"exam",{"id":97,"doc_module":4,"doc_module_name":25,"category_name":98,"show_sort_weight":99,"slug":100},5,"Comic",60,"comic",{"id":102,"doc_module":4,"doc_module_name":25,"category_name":103,"show_sort_weight":104,"slug":105},6,"Technology",50,"technology",{"id":107,"doc_module":4,"doc_module_name":25,"category_name":108,"show_sort_weight":109,"slug":110},7,"Healthcare",40,"healthcare",{"id":112,"doc_module":4,"doc_module_name":25,"category_name":29,"show_sort_weight":113,"slug":114},8,30,"research-report",{"id":116,"doc_module":4,"doc_module_name":25,"category_name":117,"show_sort_weight":118,"slug":119},9,"Religion & Spirituality",20,"religion-spirituality",{"id":118,"doc_module":4,"doc_module_name":25,"category_name":121,"show_sort_weight":118,"slug":122},"World Cup","world-cup",{"id":124,"doc_module":4,"doc_module_name":25,"category_name":125,"show_sort_weight":124,"slug":126},10,"Lifestyle","lifestyle",{"id":128,"doc_module":4,"doc_module_name":25,"category_name":129,"show_sort_weight":97,"slug":130},19,"General","general",{"code":4,"msg":82,"data":132},{"doc_id":79,"user_id":133,"nickname":42,"user_avatar":134,"doc_module":4,"category_id":112,"category_name":29,"doc_title":10,"doc_description":12,"doc_content":135,"file_id":136,"file_url":137,"file_type":138,"file_size":139,"view_count":55,"is_deleted":4,"is_public":22,"is_downloadable":22,"audit_status":22,"page_count":140,"language":141,"language_code":8,"site_id":7,"html_lang":8,"table_of_contents":142,"faqs":143,"seo_title":144,"seo_description":12,"update_tm":80,"read_time":145,"preview_page_count":4},8796096645457,"https://ap-avatar.wpscdn.com/avatar/800003749518d68ffe3?x-image-process=image/resize,m_fixed,w_180,h_180&k=1779345340919836971","Safe Deep Reinforcement Learning for Super Mario Bros using Heuristics  \nBachelor’s Project Thesis  \nAmir Abdul Aziz, s4019555, [a.abdulaziz@student.rug.nl](a.abdulaziz@student.rug.nl)  \nSupervisors: J.D. C´ardenas Cartagena  \nThis thesis explores the integration of heuristic safety mechanisms within Deep Reinforcement Learning (Deep RL) frameworks to enhance agent survival in the complex, dynamic environment of Super Mario Bros. We examine the influence of heuristic-augmented Actor-Critic models on agent behavior, particularly in terms of safety without compromising learning efficiency. The study compares various configurations, including separate and shared neural network architecturesand different combinations of frame stacks for the input layer. It quantitatively measures survival rates, highlighting a significant improvement in avoiding fatal encounters with the most effective configuration around the 10000th episode mark. However, these safety mechanisms may also constrain exploration and long-term reward optimization. Despite achieving a stable survival pattern and a consistent minimum mean loss, a tension between safety and reward maximization is evident. This work contributes to the domain of Safe RL, emphasizing the nuanced trade-offsin stochastic gaming environments.  \n1 Introduction  \nReinforcement Learning (RL) embodies a specific approach within the field of artificial intelligence, aiming to equip agents with the capability to learn optimal behaviors through autonomous interaction with their environment. This learning paradigm, distinguished by its use of a reward function as a feedback mechanism, allows agents to choose actions that maximize cumulative rewards over time. The significance of RL extends beyond theoretical interest, finding practical applications in domains ranging from robotics to complex game environments (Kober, Bagnell, and Peters (2013)) .  \nVideo game environments, exemplified by the OpenAI Gym interface introduced by Brockmanet al (2016), offer a perfect ground for the application and testing of RL algorithms. These simulated settings provide controlled, yet challenging, arenas where RL agents can be trained, tested, and optimized without the constraints and unpredictability of real-world environments. The inherent continuous and dynamic nature of such games, combined with the essential requirement for realtime decision-making, highlights the critical demand for sophisticated RL strategies tailored to manage complex, sequential decision-making tasks. The actor-critic method, while a significant innovation within the RL framework, was not the first to make impactful strides in video game environments, such as those represented in the OpenAI  \nGym. It was not until 2016, that adaptations of the actor-critic method, specifically the Asynchronous Advantage Actor-Critic (A2C) algorithm, began to show promise Mnih, Badia, Mirza, Graves, Lillicrap, Harley, Silver, and Kavukcuoglu (2016) . These methods were built upon the foundation laid by DQNs, offering an architecture that integrated the strengths of both policy-based and value-based approaches to address the nuanced challenges of such environments. Therefore, while the actorcritic approach now stands as a robust framework for enhancing learning dynamics, it represents a continuing evolution in the search for more efficient and nuanced reinforcement learning processes.  \nThe actor-critic architecture, introduced by Konda and Tsitsiklis (1999) branches the learning process into two interconnected components: the actor, responsible for policy execution by selecting actions, and the critic, which evaluates the actions taken by the actor based on the received reward signal. This division not only accelerates the learning process, by using the value function to adjust the policy directly but also enriches the agent’s ability to navigate complex decision spaces, using the critic’s evaluation to identify which actions lead to better long-term outcomes","cbCaijLR68jfJriC","https://ap.wps.com/l/cbCaijLR68jfJriC","pdf",759774,15,"English","# Introduction\n## Reinforcement Learning and reward-based learning\n## Actor-Critic and Deep Reinforcement Learning\n## Safe Reinforcement Learning principles\n## Safety constraints vs exploration-exploitation trade-off\n## Research focus: Super Mario Bros with heuristic Safe Deep RL","[{\"question\":\"What problem does the thesis address in Deep Reinforcement Learning for Super Mario Bros?\",\"answer\":\"It addresses the need to improve agent survival by integrating safety mechanisms into Deep RL while operating in a complex, dynamic game environment.\"},{\"question\":\"How does the study implement safety in the Actor-Critic framework?\",\"answer\":\"It uses heuristic-augmented Actor-Critic models, testing different configurations that influence agent decisions and survival outcomes.\"},{\"question\":\"What trade-off does the research observe when adding safety constraints?\",\"answer\":\"Safety improvements can limit exploration, which may constrain long-term reward optimization, revealing tension between safety and maximizing reward.\"}]","Safe Deep Reinforcement Learning for Super Mario Bros using Heuristics - Bachelor’s Project Thesis | PDF",38]