Human Capital, Not Model Benchmarks, Predicts Hybrid Intelligence in Forecasting

Jul 16, 2026

Human–AI collaboration in forecasting is often summarized as a single average effect, but this pilot tests whether the benefit depends on measurable human capital. Using a real-money prediction market (Polymarket) with externally resolved outcomes as an objective benchmark, results show hybrid performance is trimodal: many forecasters either defer to the model or rubber-stamp prior beliefs and underperform, while a smaller group performs as well as or better than the market. Perspective-taking, intellectual humility, and curiosity distinguish the complementary-reasoning “cyborgs.”

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages5

Reading Time13 min

Introduction

Methods

Participants and design

Forecasting task and ground truth

Outcome metric

Human-capital measures

Interaction styles

Analysis

Results

Human Capital, Not Model Benchmarks, Predicts Hybrid Intelligence in Forecasting Vivienne Ming The Human Trust ; Possibility Science; UCL Global Business School for Health Preprint—pilot study. Findings are preliminary and intended to motivate a pre-registered replication. Significance. Whether pairing people with AI helps or hurts is usually reported as a single average effect. Using a real-money prediction market (Polymarket) as an objective, externally resolved benchmark, this pilot shows that the value of human–AI collaboration depends on a specific, measurable form of human capital. Analyzed at the level of the individual forecaster, hybrid performance is trimodal: most people either deferred to the model (matching it) or used it to rubber-stamp a prior guess (performing worse than the model alone), while a minority engaged in genuine complementary reasoning and reached accuracy matching or even exceeding (i.e. , lower error than) the market itself. Collaborative traits—perspective-taking, intellectual humility, and curiosity—rather than raw cognitive ability or model benchmarks, distinguished who reached that mode. The results are preliminary but statistically robust, and motivate a pre-registered replication now in preparation. Introduction Studies of human–AI collaboration report conflicting effects: some find augmentation of lower-skilled workers (1, 2), others find the benefits accrue mainly to experienced users (3), still others find that access to capable models degrades human performance relative to the model working alone (4, 5), and others show substantial returns on complex, elite tasks (6) . A recurring confound is evaluation. In some cases AI systems may simply have encoded the problem and its solution, so that measured“ability” reflects contamination rather than reasoning (7, 8) . And even on less constrained tasks,“quality” is often scored by expert raters who are themselves swayed by the fluency of AI-assisted prose rather than the substance of the work (9, 10) . Forecasting against a real-money prediction market sidesteps both problems: every prediction is scored against an externally resolved ground truth on a common, style-free metric. I report a pilot designed to estimate (i) whether human capital moderates the value of AI assistance in forecasting, and (ii) whether distinct interaction styles mediate that moderation. Because each participant entered their own probabilistic forecast, all analyses are conducted at the level of the individual forecaster. I frame the work as hypothesis-generating: several cells are small, and the aim is to obtain effect-size estimates—now statistically supported—to power a confirmatory trial in preparation. Methods Participants and design. One hundred eight adults were recruited by flyer in Berkeley, CA and compensated $20 per session for three sessions. The main study comprised 78 participants (42 UC Berkeley students; 36 community adults; mean age 28.7, range 18–60) organized into 26 three-person teams and assigned either to a Human-only condition (12 teams) or a Hybrid condition (14 teams) with access to one of four large language models. A separate pool of 30 participants (10 teams) worked with a “Socratic” model that withholds answers, reported separately (Supplementary S1) . Participants collaborated in teams but each entered an individual forecast; analyses are at the participant level. One participant was excluded for an out-of-range Brier score (−0 .005), leaving 77 analyzed in the main pool. Forecasting task and ground truth. The 30 questions were live Polymarket contracts resolving between November 2025 and January 2026, spanning economics, international relations, and business. Each question’s externally certified resolution (o ∈ {0,1}) provided ground truth, and each team forecast 10 questions drawn at random from the pool. The four models also forecast all 30 questions independently (AI-only baseline): Llama 3.1 (8B), Qwen3 (8B), GPT-4o

FAQs

Why does the study use a real-money prediction market as a benchmark?

Predictions are scored against an externally resolved ground truth using a common, style-free metric, avoiding confounds from model-encoded solutions or subjective expert ratings.

What does the study find about hybrid (human+AI) forecasting performance?

Hybrid performance splits into three modes: most people either match the model or validate their own prior guess and do worse than the model alone, while a minority uses complementary reasoning and achieves accuracy matching or exceeding the market.

Which human-capital traits differentiate the best-performing interaction style?

Perspective-taking, intellectual humility, and curiosity—not raw cognitive ability or simple model benchmarks—are associated with reaching the complementary-reasoning mode.

Jul 16, 2026

Upload by Ava Thompson