[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-82294-en":3,"doc-seo-82294-105":29,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":13,"seo_description":14,"update_tm":27,"read_time":28},82294,13056703019662,"Evangeline","https://ap-avatar.wpscdn.com/avatar/be000253a8e92610077?_k=1778726343310543188",8,"Research & Report","Risk-Aware General-Utility Markov Decision Processes","This paper studies general-utility Markov decision processes (GUMDPs) with risk-aware objectives, where an agent optimizes a risk measure over the distribution of objective values. Objective values depend on state visitation frequencies induced by the policy, enabling trade-offs between expected performance and risk aversion across a broad class of utility functions. The focus is the entropic risk measure (ERM). The work then develops online planning solutions, presenting an MCTS-based method that provably achieves any desired accuracy, validated by experiments across standard, exploration, imitation-learning, and multi-objective tasks.","arXiv :2607 .09298v 1 [ cs .LG] 10 Jul 2026  \nRisk-Aware General-Utility Markov Decision Processes  \nPedro P. Santos, Fábio Vital, Alberto Sardinha, Francisco S. Melo  \nKeywords: Risk-aware decision-making, Reinforcement learning, Planning.  \nSummary  \nWe study general-utility Markov decision processes (GUMDPs) with risk-aware objectives. In this framework, an agent aims to optimize a risk measure of the distribution of objective values, where the objective function depends on the frequency of visitation of states induced by the agent’s policy. First, we motivate, propose and formalize risk-aware GUMDPs, which enable agents and decision makers to trade off expected performance and risk aversion while benefiting from the rich set of objectives that can be cast under the framework of GUMDPs. We focus our attention to the entropic risk measure (ERM) . Second, we show how we can solve risk-aware GUMDPs with ERM objectives by resorting to online planning techniques. In particular, we propose an MCTS-based approach to provably solve risk-aware GUMDPs up to any desired accuracy. Third, we provide a set of experimental results showcasing that our approach is successful when optimizing for a spectrum of risk-aware behaviors in the context of GUMDPs under diverse tasks (standard MDPs, maximum state entropy exploration, imitation learning, and multi-objective MDPs) .  \nContribution(s)  \n1. We motivate, propose and formalize risk-aware GUMDPs, which enable agents and decision makers to trade off expected performance and risk aversion while benefiting from the rich set of objectives that can be cast under the framework of GUMDPs.  \nContext: Previous works studied risk-neutral (expected performance) policy optimization in GUMDPs (Zahavy et al., 2021 ; Mutti et al., 2023 ; Santos et al., 2025) .  \n2. We show how we can solve risk-aware GUMDPs by resorting to online planning techniques, proposing an MCTS-based approach to provably solve risk-aware GUMDPs up to any desired accuracy.  \nContext: None.  \n3. We provide a set of experimental results showcasing that our approach is successful when optimizing for a spectrum of risk-aware behaviors in the context of GUMDPs under diverse tasks (standard MDPs, maximum state entropy exploration, imitation learning, and multiobjective MDPs) .  \nContext: None.  \nRisk-Aware General-Utility Markov Decision Processes  \nPedro P. Santos 1,2 , Fábio Vital 1,2 , Alberto Sardinha 1,3 , Francisco S. Melo 1,2 {pedro.pinto.santos, [fabiovital}@tecnico.ulisboa.pt](fabiovital}@tecnico.ulisboa.pt)[sardinha@inf.puc-rio.br](sardinha@inf.puc-rio.br) , [fmelo@inesc-id.pt](fmelo@inesc-id.pt)  \n1INESC-ID  \n2Instituto Superior Técnico, University of Lisbon  \n3Pontifical Catholic University of Rio de Janeiro  \nAbstract  \nWe study general-utility Markov decision processes (GUMDPs) with risk-aware objectives. In this framework, an agent aims to optimize a risk measure of the distribution of objective values, where the objective function depends on the frequency of visitation of states induced by the agent’s policy. First, we motivate, propose, and formalize riskaware GUMDPs, which enable agents and decision makers to trade off expected performance by risk aversion while benefiting from the rich set of objectives that can be cast under the framework of GUMDPs. We focus our attention on the entropic risk measure (ERM) . Second, we show how we can solve risk-aware GUMDPs with ERM objectives by resorting to online planning techniques. In particular, we propose an approach based on Monte Carlo Tree Search (MCTS) to provably solve risk-aware GUMDPs up to any desired accuracy. Third, we provide a set of experimental results showcasing that our approach is successful when optimizing for a spectrum of risk-aware behaviors in the context of GUMDPs under diverse tasks (standard MDPs, maximum state entropy exploration, imitation learning, and multi-objective MDPs) . Code available at [https://github.com/gh0stwin/risk-aware-gumdp](https://git","cbCaioK5wz6tbnx3","https://ap.wps.com/l/cbCaioK5wz6tbnx3","pdf",572556,1,27,"English","en",105,"# Introduction\n## Motivation for GUMDPs and risk-aware objectives\n## Related work and problem setup\n# Summary of contributions\n## Formalization of risk-aware GUMDPs\n## MCTS-based online planning with ERM objectives\n## Experimental results across tasks","[{\"question\":\"What is the main goal of risk-aware general-utility Markov decision processes (GUMDPs)?\",\"answer\":\"The agent optimizes a risk measure of the distribution of objective values, where objective values are determined by visitation frequencies induced by the policy.\"},{\"question\":\"Which risk measure is the paper primarily focused on?\",\"answer\":\"The paper focuses on the entropic risk measure (ERM) as the risk-aware objective.\"},{\"question\":\"How are risk-aware GUMDPs solved in the proposed approach?\",\"answer\":\"The method uses online planning, specifically an MCTS-based approach, with guarantees to provably solve the problem up to any desired accuracy.\"}]",1784179444,68,{"code":4,"msg":30,"data":31},"ok",{"site_id":24,"language":23,"slug":32,"title":13,"keywords":33,"description":14,"schema_data":34,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":27},"risk-aware-general-utility-markov-decision-processes","",{"@graph":35,"@context":85},[36,53,68],{"@type":37,"itemListElement":38},"BreadcrumbList",[39,43,47,50],{"item":40,"name":41,"@type":42,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":44,"name":45,"@type":42,"position":46},"https://docshare.wps.com/document/","Document",2,{"item":48,"name":12,"@type":42,"position":49},"https://docshare.wps.com/document/research-report/",3,{"item":51,"name":13,"@type":42,"position":52},"https://docshare.wps.com/document/risk-aware-general-utility-markov-decision-processes/82294/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":23,"description":14,"dateModified":61,"datePublished":62,"encodingFormat":60,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":40,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-17","2026-07-16",true,{"@type":65,"interactionType":66,"userInteractionCount":20},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"What is the main goal of risk-aware general-utility Markov decision processes (GUMDPs)?","Question",{"text":75,"@type":76},"The agent optimizes a risk measure of the distribution of objective values, where objective values are determined by visitation frequencies induced by the policy.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"Which risk measure is the paper primarily focused on?",{"text":80,"@type":76},"The paper focuses on the entropic risk measure (ERM) as the risk-aware objective.",{"name":82,"@type":73,"acceptedAnswer":83},"How are risk-aware GUMDPs solved in the proposed approach?",{"text":84,"@type":76},"The method uses online planning, specifically an MCTS-based approach, with guarantees to provably solve the problem up to any desired accuracy.","https://schema.org",{"og:url":51,"og:type":87,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":89,"canonical":51},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":20,"doc_module":4,"doc_module_name":45,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":46,"doc_module":4,"doc_module_name":45,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":45,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":45,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":45,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":45,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":45,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":45,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":45,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":45,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":45,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]