[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-85375-en":3,"doc-seo-85375-105":29,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":13,"seo_description":14,"update_tm":27,"read_time":28},85375,7971461741311,"Ophelia","https://ap-avatar.wpscdn.com/avatar/74000253aff267980c6?x-image-process=image/resize,m_fixed,w_180,h_180&k=1779345379180704826",8,"Research & Report","Inside the Unfair Judge A Mechanistic Interpretability Account of LLM-as-Judge Bias","LLM judges that score, compare answers, and provide reward signals for alignment can exhibit scoring bias driven by surface cues unrelated to answer quality. This work develops a representation-level interpretation, complementing input–output perturbation studies, by analyzing the judge’s hidden states. Across seven judges, seven bias types, and nine benchmarks, the authors show bias-specific activation geometry, causal control via hidden-state steering, and an operational linear projection that predicts judge failures on unseen benchmarks.","arXiv :2607 . 1 187 1v 1 [ cs .LG] 13 Jul 2026  \nInside the Unfair Judge: A Mechanistic  \nInterpretability Account of LLM-as-Judge Bias  \nZixiang Xu1,2,3,†, Sixian Li4 , Huaxing Liu1 , Xiang Wang1 , Shuai Li1 , Zirui Song1,2 and Xiuying Chen2  \n1AMAP, Alibaba Group, 2 Mohamed bin Zayed University of Artificial Intelligence, 3University of Southern California, 4University of Michigan, Ann Arbor  \nExisting studies of LLM-as-judge scoring bias work predominantly at the input-output level: they perturb inputs, measure score deltas, and propose prompt-level mitigations. We argue that the same biases admit a representation-level account in the judge’s hidden state, complementary to the input-output view and operationally useful in ways it does not afford. We report three findings, across seven judges, seven bias types, and nine benchmarks. Geometry: baseline judging inputs occupy a tight activation manifold while biased inputs are displaced along a low-dimensional, type-specific subspace that sharpens with depth and is recovered consistently by three families of estimators. Causal control: steering hidden states along this subspace drives scoring in both directions, forward shifts reproducing biased scoring on clean inputs and reverse shifts restoring baseline scoring on biased ones, while matched-norm random directions produce shifts an order of magnitude smaller. Operational: a simple linear projection onto the same bias-direction features anticipates judge failures on three entirely unseen benchmarks, substantially outperforming textbased alternatives. Reading bias as activation geometry, rather than as input-output noise, unifies geometric structure, causal control, and operational prediction within a single framework. The project page is available at [https://xzx34.github.io/unfair-judge/](https://xzx34.github.io/unfair-judge/) .  \n1. Introduction  \nLarge language models are now routinely deployed as automatic judges: they rate answers, compare candidates, and supply reward signals for alignment and preference learning (Zheng et al., 2023; Liu et al., 2023; Gu et al., 2024; Li et al., 2024a) . This convenience comes at a cost. A growing body of work documents that LLM judges shift their scores in response to surface cues that have nothing to do with answer quality, including stated authorship, verbosity, declared peer consensus, emotional tone, and metacognitive claims (Wang et al., 2024a; Saito et al., 2023; Panickssery et al., 2024a; Li et al., 2024c; Koo et al., 2024) . Such scoring biases undermine benchmark validity and, because judges sit inside RLHF and evaluation pipelines, silently propagate into the models they are meant to audit.  \nThe dominant response to this problem has been to push harder on the input-output interface: perturb inputs more systematically, measure score deltas more carefully, and mitigate with prompt engineering, persona constraints, or ensembling (Dubois et al., 2024; Wataoka et al., 2024; Verga et al., 2024) . This view treats the judge as a black box whose biases are described by what comes out the other side. It has produced a useful empirical catalog, but it leaves the obvious next question untouched: when the judge issues an unfair score, what is happening inside the model? A parallel thread in mechanistic interpretability has by now built a sharp toolkit for that question, showing that many high-level behaviors, truthfulness, refusal, social attitudes of outputs, are concentrated along low-dimensional directions in a transformer’s residual stream and admit direct causal manipulation (Turner et al., 2023; Li et al., 2023; Zou et al., 2023; Arditi et al., 2024; Park et al., 2024; Siddique et al., 2025) . These tools have largely targeted generative behaviors; only very recently have internal representations been used on the judging side at all, to aggregate cross-layer signals into betteraligned scores (Lai et al., 2025) or, in question answering, to steer away answer-selection bias (Adila  \ne","cbCaic96EM2ua9rO","https://ap.wps.com/l/cbCaic96EM2ua9rO","pdf",9265215,1,58,"English","en",105,"# Introduction\n## Geometric\n## Causal control","[{\"question\":\"What limitation do prior studies on LLM-as-judge scoring bias mainly have?\",\"answer\":\"They focus on the input–output interface by perturbing inputs, measuring score deltas, and proposing prompt-level mitigations, without explaining what happens inside the judge model when unfair scores occur.\"},{\"question\":\"How is bias characterized in the judge model according to this work?\",\"answer\":\"Bias is presented as a representation-level phenomenon: an internal mapping from semantics-irrelevant surface cues to score predictions, reflected in hidden-state activation geometry.\"},{\"question\":\"What evidence supports causal control of bias and its operational usefulness?\",\"answer\":\"Steering hidden states along the recovered bias subspace changes scoring in both directions, while a simple linear projection onto the same bias-direction features anticipates failures on unseen benchmarks, outperforming text-based alternatives.\"}]",1784202964,146,{"code":4,"msg":30,"data":31},"ok",{"site_id":24,"language":23,"slug":32,"title":13,"keywords":33,"description":14,"schema_data":34,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":27},"inside-the-unfair-judge-a-mechanistic-interpretability-account-of-llm-as-judge-bias","",{"@graph":35,"@context":85},[36,53,68],{"@type":37,"itemListElement":38},"BreadcrumbList",[39,43,47,50],{"item":40,"name":41,"@type":42,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":44,"name":45,"@type":42,"position":46},"https://docshare.wps.com/document/","Document",2,{"item":48,"name":12,"@type":42,"position":49},"https://docshare.wps.com/document/research-report/",3,{"item":51,"name":13,"@type":42,"position":52},"https://docshare.wps.com/document/inside-the-unfair-judge-a-mechanistic-interpretability-account-of-llm-as-judge-bias/85375/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":23,"description":14,"dateModified":61,"datePublished":62,"encodingFormat":60,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":40,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-17","2026-07-16",true,{"@type":65,"interactionType":66,"userInteractionCount":20},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"What limitation do prior studies on LLM-as-judge scoring bias mainly have?","Question",{"text":75,"@type":76},"They focus on the input–output interface by perturbing inputs, measuring score deltas, and proposing prompt-level mitigations, without explaining what happens inside the judge model when unfair scores occur.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"How is bias characterized in the judge model according to this work?",{"text":80,"@type":76},"Bias is presented as a representation-level phenomenon: an internal mapping from semantics-irrelevant surface cues to score predictions, reflected in hidden-state activation geometry.",{"name":82,"@type":73,"acceptedAnswer":83},"What evidence supports causal control of bias and its operational usefulness?",{"text":84,"@type":76},"Steering hidden states along the recovered bias subspace changes scoring in both directions, while a simple linear projection onto the same bias-direction features anticipates failures on unseen benchmarks, outperforming text-based alternatives.","https://schema.org",{"og:url":51,"og:type":87,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":89,"canonical":51},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":20,"doc_module":4,"doc_module_name":45,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":46,"doc_module":4,"doc_module_name":45,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":45,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":45,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":45,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":45,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":45,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":45,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":45,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":45,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":45,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]