[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-83242-en":3,"doc-seo-83242-105":30,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":21,"is_downloadable":21,"audit_status":21,"page_count":22,"language":23,"language_code":24,"site_id":25,"html_lang":24,"table_of_contents":26,"faqs":27,"seo_title":13,"seo_description":14,"update_tm":28,"read_time":29},83242,962075114101,"Seraphina","https://ap-avatar.wpscdn.com/avatar/e000253a75eb197efd?x-image-process=image/resize,m_fixed,w_180,h_180&k=1780044092746381165",8,"Research & Report","RLVP Penalize the Path, Reward the Outcome","Agents operating in real-world settings must learn online from costly, often irreversible interactions rather than cheap simulator steps. The work shows two deployability requirements: success depends on the agent’s path as well as the final outcome, and learning must be sample-efficient from very few examples. Reinforcement learning from verifiable rewards (RLVR) fails because it optimizes outcome only and wastes rollouts. A proposed approach penalizes the path while rewarding verifiable outcomes, achieving high task success with near-zero violations using four penalty design rules.","arXiv :2607 .07435v 1 [ cs .LG] 8 Jul 2026  \nRLVP: Penalize the Path, Reward the Outcome  \nBojie Li Noah Shi  \nPine AI University of Washington  \nAbstract  \nAgents acting on our behalf in the real world—placing phone calls, resolving support tickets—must learn online from costly, often irreversible interactions rather than cheap simulator steps. Two things follow. First, deployability depends on the path, not only the outcome. An agent must respect outcome-neutral constraints such as not repeatedly calling an unresponsive user, respecting business hours, or completing required authentication—constraints that outcome-based rewards cannot express, since violating them frequently improves apparent success. Second, because each interaction is expensive, the agent must learn efficiently from very few examples.  \nReinforcement learning from verifiable rewards (RLVR) is blind to both challenges: it optimizes solely on the outcome and wastes expensive rollouts on all-fail groups where group-relative advantage collapses to zero. Attempts to densify supervision by rewarding progress target the hard-to-verify direction. In contrast, real agentic environments can cheaply detect bad moves. Since group-relative advantage is equivalent to within-group variance, a dense signal helps only when it supplies variance the outcome lacks. A verifiable penalty on the path meets this condition reliably, while a progress potential helps only where partial progress is reachable. The resulting recipe—penalize the path, reward the outcome—achieves high task success with near-zero violations, where outcome-only training violates constraints on nearly every episode. We provide four design rules for effective penalties, including avoidance of the inaction trap that arises when a penalty is used in isolation.  \nCode: [https://github.com/19PINE-AI/rlvp](https://github.com/19PINE-AI/rlvp)[ ](https://github.com/19PINE-AI/rlvp)Website: [https://01.me/research/rlvp](https://01.me/research/rlvp)  \nTwo verifiable path signals for agents that learn from costly, real-world rollouts  \nv io lat ion-free rate  \n1.0  \n0.8  \n0.6  \n0.4  \n0.2  \n0.0  \n(a) Penalize the path  \n0 500 1000 1500 2000 2500 3000  \nepisodes generated  \ntask success  \n1.0  \n0.8  \n0.6  \n0.4  \n0.2  \n0.0  \n|  |  |  |\n| --- | --- | --- |\n|  |  |  |\n|  | competence in fewer rollouts | \u003Cbr>outcome-only\u003Cbr>+ verifiable progress potential |\n\n(b) Reward progress where reachable  \n5 10 15 20 25 30 training iterations (each = costly rollouts)  \nFigure 1. Two verifiable path signals for agents that learn from costly, real-world rollouts. (a) Penalize the path: rewarding the outcome while penalizing the path drives violation-free episodes from near zero to ≈100%—a deployability constraint outcome-only GRPO is blind to and never reaches—at no cost to task success (§3) . (b) Reward progress where reachable: a dense potential reaches competence in fewer costly rollouts than outcome-only training (§4) . Mean over seeds; bands ±1 s.d.  \n1 Introduction  \nConsider an agent that acts on behalf of a person in the real world: it calls a bank, navigates through the automatic voice menu, waits when put on hold, and eventually speaks with a representative to resolve a billing dispute. Every action is a real, often irreversible interaction—a genuine call rather than a resettable simulator step that can be replayed millions of times. A rapidly growing class of deployed agents operate in this high-stakes regime [Pine AI, 2026a] . Unlike the reversible, self-contained environments of math problems, coding benchmarks, and simulated games that shaped today’s foundation models, these agents must contend with partial observability, irreversible consequences, long feedback delays, and genuine uncertainty about the state of the external world. This regime differs from the reversible, high-throughput environments used to train today’s models in two fundamental ways.  \nFirst, deployability depends on the path the agent takes, not the outc","cbCaijiVY3AT39SK","https://ap.wps.com/l/cbCaijiVY3AT39SK","pdf",856352,2,1,31,"English","en",105,"# Abstract\n# Introduction\n## Deployability depends on the path\n## Sample efficiency in real-world learning\n# RLVR limitations\n## Outcome-only optimization\n## Inefficient credit assignment","[{\"question\":\"为什么在真实世界的智能体学习中，路径约束（path constraints）比仅看最终结果更重要？\",\"answer\":\"作者指出，达到正确最终结论仍可能不可部署，例如在用户拒绝联系、超出营业时间拨打或绕过必需认证时，结果虽成功但路径违反规则。仅用结果奖励无法表达这些约束，因为短期捷径往往会提高表面成功率。\"},{\"question\":\"样本效率（少量交互学习）在文中具体意味着什么？\",\"answer\":\"真实交互往往不可重放：电话不能像模拟环境那样生成海量回放轨迹。因此智能体必须从极少例子中立即泛化新知识，否则系统无法在现实中持续改进并最终无法部署。\"},{\"question\":\"RLVR（可验证奖励的强化学习）为何会在文中被认为同时面临两类挑战？\",\"answer\":\"文中认为RLVR只优化环境能廉价评估的最终结果，使路径不可见；同时它会把昂贵交互浪费在全失败轨迹上，群组相对优势为零时几乎没有可学习信号。为此作者提出通过对路径惩罚、对可验证结果奖励来改善。\"}]",1784186190,78,{"code":4,"msg":31,"data":32},"ok",{"site_id":25,"language":24,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":28},"rlvp-penalize-the-path-reward-the-outcome","",{"@graph":36,"@context":85},[37,53,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,47,50],{"item":41,"name":42,"@type":43,"position":21},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":20},"https://docshare.wps.com/document/","Document",{"item":48,"name":12,"@type":43,"position":49},"https://docshare.wps.com/document/research-report/",3,{"item":51,"name":13,"@type":43,"position":52},"https://docshare.wps.com/document/rlvp-penalize-the-path-reward-the-outcome/83242/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":24,"description":14,"dateModified":61,"datePublished":62,"encodingFormat":60,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":41,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-22","2026-07-16",true,{"@type":65,"interactionType":66,"userInteractionCount":20},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"为什么在真实世界的智能体学习中，路径约束（path constraints）比仅看最终结果更重要？","Question",{"text":75,"@type":76},"作者指出，达到正确最终结论仍可能不可部署，例如在用户拒绝联系、超出营业时间拨打或绕过必需认证时，结果虽成功但路径违反规则。仅用结果奖励无法表达这些约束，因为短期捷径往往会提高表面成功率。","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"样本效率（少量交互学习）在文中具体意味着什么？",{"text":80,"@type":76},"真实交互往往不可重放：电话不能像模拟环境那样生成海量回放轨迹。因此智能体必须从极少例子中立即泛化新知识，否则系统无法在现实中持续改进并最终无法部署。",{"name":82,"@type":73,"acceptedAnswer":83},"RLVR（可验证奖励的强化学习）为何会在文中被认为同时面临两类挑战？",{"text":84,"@type":76},"文中认为RLVR只优化环境能廉价评估的最终结果，使路径不可见；同时它会把昂贵交互浪费在全失败轨迹上，群组相对优势为零时几乎没有可学习信号。为此作者提出通过对路径惩罚、对可验证结果奖励来改善。","https://schema.org",{"og:url":51,"og:type":87,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":89,"canonical":51},"index,follow",{"doc_id":7,"site_id":25},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":21,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":20,"doc_module":4,"doc_module_name":46,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":46,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":46,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":46,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":46,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":46,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":46,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]