Do Large Language Model Voters Strategize? An Oracle-Based Benchmark for Manipulation under Voting Rules

Jul 16, 2026

Upload by Aurora

Strategic voting is a key failure mode in collective choice, where voters can report ballots that differ from their true preferences to achieve more desirable outcomes. This paper presents an oracle-based benchmark that tests whether large language model (LLM) voters can discover and execute profitable manipulations. Each benchmark instance supplies true preference rankings, other voters’ ballots, a deterministic voting rule, and prompt framing conditions. An exact oracle enumerates all feasible reports, computes sincere and alternative outcomes, identifies profitable and optimal manipulations, and records the best achievable results. The benchmark covers multiple voting rules and prompt framings, using a fixed electorate size and balanced election instances to provide ground-truth calibration without subjective human grading.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages19

Reading Time48 min

Introduction

Benchmark design and oracle calibration

Research questions

Contributions

Do Large Language Model Voters Strategize? An Oracle-Based Benchmark for Manipulation under Voting Rules arXiv :2606 .2 100 1v2 [ cs .GT] 10 Jul 2026 Seyed Pouyan Mousavi Davoudi Independent Researcher in AI and Statistics Tehran, Iran [spouyan.mousavi@gmail.com](spouyan.mousavi@gmail.com) Amin Gholami Davodi Independent Researcher in AI and Statistics Tehran, Iran [a.g.davodi@gmail.com](a.g.davodi@gmail.com) Arshia Alireza Amiri-Margavi Model Risk Manager The Bank of New York New York, NY, USA [alireza.amirimargavi@Bny.com](alireza.amirimargavi@Bny.com) Hamidreza Hasani Balyani AI Evaluation Engineer, Amazon Lab126, Hardware Technology Organization Sunnyvale, CA, USA [rezahsni@amazon.com](rezahsni@amazon.com)[ ](rezahsni@amazon.com)Gharagozlou* Mathematics & Statistics Department University of Minnesota Duluth Duluth, MN, USA [ghara027@d.umn.edu](ghara027@d.umn.edu) * Equal contribution. Abstract Strategic voting is a canonical failure mode for collective choice: a voter may obtain amore preferred outcome by reporting a ballot that differs from its true preferences. This paper introduces an oracle-based benchmark for testing whether large language model (LLM) voters can discover and execute such manipulations. Each instance gives an LLM voter a true preference ranking, the other voters’ ballots, a deterministic voting rule, and a prompt condition. An exact oracle enumerates every feasible report by the LLM voter, computes the sincere outcome, identifies all profitable reports, and records the best achievable outcome. The benchmark therefore supplies ground truth for strategic success without human labels or subjective grading of explanations. The benchmark covers plurality, Borda, approval, instant-runoff voting, and Copeland-style pairwise majority voting; prompt conditions separate sincere, strategic, civic, and expert framings. To keep the primary study defensible while preserving the main comparisons, the registered core design fixes a single electorate size, uses 600 balanced election instances, and produces 9,600 model–prompt responses when run with four model configurations and four prompt conditions. Because existing peer-reviewed work does not report manipulation discovery, optimal manipulation, false manipulation, near-miss, or invalid-ballot rates for this exact task, we do not impute LLM performance from unrelated studies. Instead, we report exact oracle-calibration baselines that bound and contextualize subsequent model results. By reducing strategic-voting behavior to exact counterfactual evaluation, the benchmark turns the question“Do LLM voters vote sincerely or strategically?” into a reproducible social-choice experiment. 1 Introduction Voting rules are increasingly relevant to AI systems. LLM agents may be asked to participate in collective decisions, simulate voters, advise human voters, aggregate preferences from multiple models, or choose among candidate outputs generated by other systems. In these settings, a model’s ballot is not merely a text completion; it can change a collective outcome. Strategic voting is therefore a useful diagnostic for LLM agency: when the model is given preferences, a voting rule, and enough information to reason about the election, does it report sincerely or exploit a profitable misreport? Classical social choice gives a sharp theoretical reason to expect strategic opportunities. The Gibbard–Satterthwaite theorem shows that every deterministic, onto, non-dictatorial social choice rule with at least three alternatives is manipulable at some profile [11, 14] . Computational social choice then asks whether finding such a manipulation is easy or hard under particular rules and candidate regimes [4–6, 8] . These literatures establish that manipulation exists and that its computational difficulty varies. They do not answer the empirical question raised by LLM agents: when an election profile is presented in natural language, can a mod

FAQs

What does the oracle-based benchmark measure about LLM voters?

It measures whether an LLM voter can discover and carry out profitable and optimal ballot manipulations relative to a deterministic voting rule, using exact oracle enumeration and ground-truth outcomes.

Which voting rules are included in the benchmark?

The benchmark covers plurality, Borda, approval, instant-runoff voting, and Copeland-style pairwise majority voting.

How do prompt framings affect evaluation in the benchmark?

Prompt conditions separate sincere, strategic, civic, and expert framings, allowing comparisons of how framing changes model behavior under the same voting tasks.

Jul 16, 2026

Upload by Aurora