Demographic Prompting at Scale: When More Attributes Hurt LLM–Human Agreement

Jul 16, 2026

Investigates how annotator demographic attributes, used as prompt cues, affect alignment between large language model (LLM) predictions and human annotations across five subjective NLP tasks. Five open-source LLMs are evaluated while varying demographic prompt composition from single-attribute to full-attribute settings. Results show alignment peaks with one to three high-signal attributes and declines under over-specification. Influence magnitude alone does not predict gains; learnability and directional coherence matter. Neuron probing links specialized activation to alignment only with coherent annotation signals, indicating steerability is not guaranteed by activation volume.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages26

Reading Time1 h 6 min

Abstract

Introduction

Research Questions (RQ1–RQ3)

Experimental Setup and Contributions

Demographic Prompting at Scale: When More Attributes Hurt LLM–Human Agreement Mahammed Kamruzzaman, Shrabon Kumar Das, Gene Louis Kim Bellini College of AI, Cybersecurity and Computing University of South Florida {kamruzzaman1, das157, genekim,}@[usf.edu](usf.edu) arXiv :2607 . 10590v 1 [ cs .CL] 12 Jul 2026 Abstract We investigate how annotator demographic attributes, supplied as prompt cues, shape the alignment between large language model (LLM) predictions and human annotations across five tasks. Using five open-source LLMs, we systematically vary the number and composition of demographic components in the prompt, spanning every combination from single-attribute through full-attribute configurations. Our experiments reveal three principal findings. First, alignment consistently peaks with one to three high-signal attributes and degrades under the full attribute set, establishing a clear over-specification threshold. Second, the overall magnitude of demographic influence on human annotations does not predict which attributes improve LLM alignment; instead, both the learnability and the directional coherence of each attribute’s annotation signal need to be considered jointly. Third, neuron probing reveals that specialized activation correlates with alignment gains only under coherent annotation signals, and that activation volume alone does not imply steerability. Together, these results demonstrate that demographic prompting is not a monolithic intervention: its utility is highly context-dependent, shaped by attribute signal quality, task characteristics, and model architecture. 1 Introduction LLMs are increasingly used as substitutes for, or complements to, human annotators on subjective NLP tasks such as toxicity detection, sentiment analysis, and offensiveness rating (Gilardi et al., 2023 ; Törnberg, 2023) . Because these tasks reflect annotator subjectivity, a natural question arises: how well can an LLM actually model a given demographic perspective when prompted to do so? A growing body of work shows that LLM outputs are not demographically neutral, predictions tend to align more closely with certain demographic groups in the absence of demographic cues, and explicitly incorporating such cues into the prompt can shift model behaviour in ways that are neither uniform nor always beneficial (Beck et al., 2024 ; Sun et al., 2025 ; Alipour et al., 2025 ; Kamruzzamanet al., 2025 ; Schäfer et al., 2025) . Despite this progress, the existing literature leaves several important questions unresolved. Most prior studies examine only one or two specific demographic attributes at a time, or compare a no-demographic baseline against a single all-attributes prompt, without exploring the combinatorial space in between (Beck et al., 2024 ; Sun et al., 2025 ; Alipour et al., 2025) . None systematically chart the incremental trajectory from singleattribute through multi-attribute to full-attribute prompting, nor investigate the signal to noise relationship along this trajectory. To address this, we pose our first research question: RQ1: To what extent do individual versus combined annotator demographic attributes shape LLM–human alignment, which demographic features (or combinations) most significantly affect alignment, and is there a threshold beyond which additional demographic information no longer benefits alignment? While several studies report that demographic prompting sometimes helps and sometimes hurts alignment (Gupta et al., 2023 ; Brown et al., 2025 ; Kamruzzaman et al., 2024), the field lacks a principled account of why: what properties of a demographic attribute, at the dataset level, predict whether prompting with it will improve a given model’s agreement with human annotators? An attribute may strongly predict variation in human labels yet carry internally opposed subgroup signals that no single persona prompt can resolve. Understanding this requires moving beyond aggregate importance measures to cha

FAQs

How do annotator demographic attributes in prompts affect LLM–human agreement?

Demographic prompting influences alignment between LLM outputs and human annotations, and the effect changes with the number and composition of attributes. Alignment peaks with one to three high-signal attributes and degrades when the full attribute set is used.

Why doesn’t the overall strength of demographic influence predict alignment gains?

The document states that alignment improvements depend jointly on how learnable each attribute’s annotation signal is and whether its signal is directionally coherent. The magnitude of influence on human labels alone is insufficient.

What do neuron probing results suggest about internal mechanisms?

Specialized neuron activation correlates with alignment gains only when the corresponding annotation signals are coherent. High activation volume can occur without resulting in steerability or improved agreement.

Jul 16, 2026

Upload by Evangeline