[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-84729-en":3,"doc-seo-84729-105":30,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":21,"is_downloadable":21,"audit_status":21,"page_count":22,"language":23,"language_code":24,"site_id":25,"html_lang":24,"table_of_contents":26,"faqs":27,"seo_title":13,"seo_description":14,"update_tm":28,"read_time":29},84729,137441390410,"Hazel","https://ap-avatar.wpscdn.com/avatar/2000252f4ab5702993?_k=1776741390130283984",8,"Research & Report","Agent Step Value Auditing Evaluator-Channel Reversals in Black-Box Agent Traces","Agent Step Value (ASV) audits evaluator-derived step rewards in black-box agent traces by exposing sign reversals across scoring channels. When the same frozen transition is replayed under different evaluator/audit configurations, the step update can flip despite conditional state information being preserved. ASV is defined as a channel-indexed target-margin gain and identifies the missing state-by-channel interaction via complete state-by-view “faces” rather than first-difference isolation. On 1,100 PubMed QA transitions, channel reversal yields negative interaction (−0.323) with 507/1,004 cross-zero flips and higher stored-success ROC AUC under quote.","Agent Step Value: Auditing Evaluator-Channel Reversals in Black-Box  \nAgent Traces  \nAndrew Zhang 1 Chengzhan Li2  \n1 KTH Royal Institute of Technology  \n2University of Electronic Science and Technology of China  \n[yibingz@kth.se](yibingz@kth.se) [chengzhan.li@std.uestc.edu.cn](chengzhan.li@std.uestc.edu.cn)  \narXiv :2607 .044 19v 3 [ cs .AI] 13 Jul 2026  \nAbstract  \nWhen evaluator-derived step rewards are pooled or compared across scoring channels, their sign is treated as transportable. Yet the same frozen transition can change sign with the scoring channel. Process rewards vary agent states, while evaluator audits vary scoring configurations; neither first difference isolates their interaction. We define Agent Step Value (ASV) as channel-indexed target-margin gain and identify the missing state-by-channel interaction by replaying complete faces. Of  \n1,100 PubMed open question-answering transitions, 1,004 were complete across four cyclic layouts. Their mean update is positive under direct scoring (+0.163 [0.102, 0.218]) and negative with an externally generated view (-0.160 [ - 0.244, -0.079]), giving an interaction of-0.323 [-0.418, -0.232] . Across paired transitions, 507/1,004 cross zero in one direction or the other. Matched templates isolate a generatedminus-quote interaction of -1.121 [-1.534, - 0.703] . Its direction remains negative after the readout and evaluator-stack changes, and quote yields a higher paired area under the receiver operating characteristic curve (AUC) for stored success at all three bridge vertices. Because the two views preserve the same task information conditional on the retained state, a representation-invariance null predicts equal responses and success rankings across them. ASV provides a transport audit for evaluatorderived step measurements; causal actor credit lies outside its estimand.  \n1 Introduction  \nStep-level signals now label reasoning traces, guide search, and optimize policies. Progress has beenoperationalized as a change in future success or answer likelihood, and process rewards are already used in agentic retrieval and factual question answering (Setlur et al., 2025 ; Lee et al., 2026 ; Zhang et al., 2025a ; Fan et al., 2026b) . In parallel, audits of large language model (LLM) evaluators  \nshow that prompts, feedback contracts, and readouts can change outputs and rankings (Chiang and Lee, 2023 ; Tripathi et al., 2025 ; Hua et al., 2025) . These two first differences meet in routine practice: evaluator protocols aggregate scores across prompt variants, process labels are compared or reused, and trajectory benchmarks compare multiple judges (Lior et al., 2025 ; Zhang et al., 2025b ; Lù et al., 2025) . Each operation assumes more than score stability: changing the channel should not reverse the update assigned to a transition. The missing quantity is their interaction.  \nRaw-score sensitivity cannot answer this question. A channel may add the same offset to both endpoints without changing the transition update. Let Fρ(x) be the target-margin response at state x under channel ρ . The mixed difference is  \nΩ = 􀀂Fρ1 (x+ ) − Fρ1 (x − )􀀃  \n(1)  \n− 􀀂Fρ0 (x+ ) − Fρ0 (x − )􀀃 .  \nAn additive channel offset cancels inside each bracket. A nonzero Ω therefore records a statedependent channel response. Identifying it without additivity assumptions requires all four matched corners. A diagonal evaluator swap also contains the reference gain and the channel offset at x − , soit cannot separate the interaction.  \nWe call the full operational scoring configuration an evaluation channel. Agent Step Value (ASV) is the before/after target-margin gain of a frozen transition within that channel; Equation 1 compares two such measurements. On the PubMed traces studied here, the direct mean of +0.163 becomes-0.160 with an externally generated view, giving ¯Ω = −0 .323. This aggregate reversal coexists with bidirectional paired flips in 507/1,004 transitions, so the claim concerns channel-indexed popul","cbCaiuegYzKMVBBE","https://ap.wps.com/l/cbCaiuegYzKMVBBE","pdf",466249,3,1,12,"English","en",105,"# Abstract\n# Introduction","[{\"question\":\"What problem does Agent Step Value (ASV) address in evaluator audits?\",\"answer\":\"ASV addresses the fact that step rewards pooled or compared across scoring channels may change sign for the same frozen transition. It isolates the missing state-by-channel interaction that first-difference methods cannot separate.\"},{\"question\":\"How is ASV defined in the document?\",\"answer\":\"ASV is defined as the channel-indexed target-margin gain of a frozen transition under a specific evaluation channel. The method compares gains across channels to estimate the interaction term.\"},{\"question\":\"What evidence does the document provide from PubMed agent traces?\",\"answer\":\"Across 1,100 open QA transitions (1,004 complete across four cyclic layouts), direct scoring updates are positive while externally generated views are negative, yielding an interaction of −0.323. Paired transitions show bidirectional cross-zero flips and template-matched analyses isolate a generated-minus-quote interaction.\"}]",1784197902,30,{"code":4,"msg":31,"data":32},"ok",{"site_id":25,"language":24,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":28},"agent-step-value-auditing-evaluator-channel-reversals-in-black-box-agent-traces","",{"@graph":36,"@context":85},[37,53,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,50],{"item":41,"name":42,"@type":43,"position":21},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":20},"https://docshare.wps.com/document/research-report/",{"item":51,"name":13,"@type":43,"position":52},"https://docshare.wps.com/document/agent-step-value-auditing-evaluator-channel-reversals-in-black-box-agent-traces/84729/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":24,"description":14,"dateModified":61,"datePublished":62,"encodingFormat":60,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":41,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-20","2026-07-16",true,{"@type":65,"interactionType":66,"userInteractionCount":20},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"What problem does Agent Step Value (ASV) address in evaluator audits?","Question",{"text":75,"@type":76},"ASV addresses the fact that step rewards pooled or compared across scoring channels may change sign for the same frozen transition. It isolates the missing state-by-channel interaction that first-difference methods cannot separate.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"How is ASV defined in the document?",{"text":80,"@type":76},"ASV is defined as the channel-indexed target-margin gain of a frozen transition under a specific evaluation channel. The method compares gains across channels to estimate the interaction term.",{"name":82,"@type":73,"acceptedAnswer":83},"What evidence does the document provide from PubMed agent traces?",{"text":84,"@type":76},"Across 1,100 open QA transitions (1,004 complete across four cyclic layouts), direct scoring updates are positive while externally generated views are negative, yielding an interaction of −0.323. Paired transitions show bidirectional cross-zero flips and template-matched analyses isolate a generated-minus-quote interaction.","https://schema.org",{"og:url":51,"og:type":87,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":89,"canonical":51},"index,follow",{"doc_id":7,"site_id":25},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,122,127,130,134],{"id":21,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":46,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":46,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":29,"slug":121},"research-report",{"id":123,"doc_module":4,"doc_module_name":46,"category_name":124,"show_sort_weight":125,"slug":126},9,"Religion & Spirituality",20,"religion-spirituality",{"id":125,"doc_module":4,"doc_module_name":46,"category_name":128,"show_sort_weight":125,"slug":129},"World Cup","world-cup",{"id":131,"doc_module":4,"doc_module_name":46,"category_name":132,"show_sort_weight":131,"slug":133},10,"Lifestyle","lifestyle",{"id":135,"doc_module":4,"doc_module_name":46,"category_name":136,"show_sort_weight":106,"slug":137},19,"General","general"]