Most LLM Conformity Needs No Speaker: Measuring the Speaker-Free Floor in Peer-Pressure Benchmarks

Jul 16, 2026

LLM conformity describes cases where a model shifts a correct answer toward a peer or group response. This work shows that most apparent conformity remains even after removing the peer. The core confound is that standard prompts jointly include an explicit speaker cue and a repeated wrong answer, so benchmarks cannot isolate the speaker’s contribution. A no-source condition keeps the asserted answer while deleting the speaker, causing harmful revisions in 66.5% of initially correct cases versus 10.3% for a plain re-ask. Across multiple models and datasets, source framing changes the baseline modestly, and recalibration cannot recover the original answer once flipped.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages15

Reading Time38 min

Abstract

Introduction

Confound in conformity prompts

No-source control condition

Experimental findings

Most LLM Conformity Needs No Speaker Measuring the Speaker-Free Floor in Peer-Pressure Benchmarks Yibo Hu* Illinois Institute of Technology [yhu89@illinoistech.edu](yhu89@illinoistech.edu) Jiaming Qu† Amazon [qjiaming@amazon.com](qjiaming@amazon.com) arXiv :2607 .05545v 1 [ cs .CL] 6 Jul 2026 Abstract LLM conformity is often used to describe cases where a model changes a correct answer toward a peer or group response. We show that most of this apparent conformity survives even after the peer is removed. The reason is a confound: standard conformity prompts mix two cues at once, the presence of a speaker and the repeated wrong answer itself. Existing benchmarks vary these cues together, so they cannot tell how much of the revision actually depends on the speaker. We introduce a no-source condition: the same asserted answer with the explicit speaker removed. Across six open-weight LLMs and seven QA and reasoning datasets, this condition alone causes harmful revision in 66.5% of initially correct cases, compared with 10.3% under a plain re-ask. The effect also remains when the repeated answer is paraphrased and when answer options are hidden in an openended setting. Source framing mainly modulates this floor: expert-panel framing raises it, while minimal person labels do not reliably raise it. When models flip, they are usually confidently wrong, and simple recalibration does not recover the original answer. Source attribution still matters, but it should be measured as an increment above this speaker-free floor. The methodological lesson is that conformity benchmarks should first measure what remains after the speaker is removed; without this step, benchmarks may mistake repeated text for social influence.1 1 Introduction When a large language model (LLM) drops a correct answer after its peers endorse a wrong one, the behavior is commonly interpreted as social conformity: the model deferring to a majority, by analogy * Corresponding author. †This research was conducted independently in a personal capacity and does not reflect the author’s position at Amazon. 1Code and data: [https://github.com/yibo-hu-lab/](https://github.com/yibo-hu-lab/)[ ](https://github.com/yibo-hu-lab/)llm-speaker-free-floor Even with no speaker, the model flips A →B. Experts: “B” no-source: “B” ×6 plain re-ask +12 .9 pp 10.3% 66.5% Figure 1: Most apparent LLM “conformity” survives without an explicit speaker. Removing the speaker leaves a 66.5% harmful revision rate, compared with 79.4% under the strongest expert-panel framing and 10.3% under a plain re-ask with no inserted content. The strongest expert-panel framing adds only +12 .9 pp above the no-source floor. Aggregated over six models and seven datasets; the A→B icon illustrates a multiplechoice answer flipping from correct to wrong. to how people yield to group pressure (Asch, 1955 ; Zhu et al., 2025 ; Weng et al., 2025 ; Cho et al., 2025 ; Choi et al., 2026 ; Zhong et al., 2025 ; Qu et al., 2026) . Recent work studies this behavior across different majority sizes, confidence levels, and authority cues, and connects it to classical theories of social influence (Milgram, 1963 ; French and Raven, 1959 ; Latané, 1981) . There is a confound at the center of this paradigm. A conformity prompt bundles two things at once: it names an explicit speaker, and it repeats an answer. Existing benchmarks vary the two together, so when a model revises we cannot tell whether it responded to the speaker, to the repeated answer, or to both. The distinction is not academic: LLM choices already shift under repetition, option position, and authoritative wording with no social content at all (Brucks and Toubia, 2025 ; Pezeshkpour and Hruschka, 2024 ; Turpin et al., 2023 ; Laban et al., 2023) . The question is therefore not whether conformity prompts move models, which they clearly do, but how much of that movement requires the explicit speaker, and how much remains when the speak

FAQs

什么是文中提出的“no-source（无来源）”条件？

no-source条件在保持被断言答案不变的情况下移除显式说话人，只保留重复的答案文本。模型在第二次回答前只看到插入的断言内容，因此任何变化都可归因于插入文本而非说话人。

文中如何衡量“说话人移除后的底线效应（speaker-free floor）”？

作者在对比实验中把断言答案固定，并删除显式说话人，同时与最少标注的人、带更丰富同伴表述的框架以及专家小组进行比较。结果用有害修正率来刻画该底线效应，并进一步做针对性控制来检验选项位置、逐字重复与来源数量等因素的影响。

实验结果表明，“多数压力下的表面从众”主要依赖哪些因素？

大多数看似从众的行为不依赖显式说话人。即使在说话人移除后，no-source条件仍会在66.5%的最初正确样本上触发有害修正；而仅进行plain re-ask时为10.3%。作者还指出专家式框架能在该底线之上增加幅度，但幅度相对有限，并且模型翻转后通常会“自信地错误”，简单重标定无法恢复原答案。

Jul 16, 2026

Upload by Theodore