When the Target Domain Changes: AI-Mediated Construct Drift in High-Stakes English Language Assessment

Jul 16, 2026

High-stakes English proficiency tests use standardized, unaided performance to interpret academic English readiness, a logic that weakens as generative AI reshapes real target language use. The paper frames AI as a score-interpretation validity problem rather than only an operational issue. It synthesizes relevant literature and defines AI-mediated construct drift: misalignment between AI-mediated communicative demands and test constructs anchored in unaided performance. It proposes bounded AI mediation, with controlled assistance boundaries, logged interactions, and task differentiation to support valid score interpretations when claims involve AI-mediated academic communication.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages29

Reading Time1 h 13 min

Abstract

Toward Bounded AI Mediation as a Validity-Oriented Design Principle

High-stakes English proficiency tests and extrapolation logic

Generative AI’s pressure on target domain interpretation

AI-mediated academic communication and conditional generalization

Key contributions: construct drift and bounded mediation

When the Target Domain Changes: AI-Mediated Construct Drift in High-Stakes English Language Assessment Yi Gui Measurement Incorporated [queenmg@gmail.com](queenmg@gmail.com) Abstract High-stakes English proficiency tests treat standardized, unaided performance as evidence for score interpretations about academic English proficiency. This interpretation remains meaningful, but as target language use domains increasingly involve generative AI, the extrapolation from unaided test performance to academic communicative readiness becomes less self-evident. This conceptual validity argument reframes AI as a score-interpretation problem in high-stakes language testing, not only an operational issue of scoring, feedback, security, or misconduct. Synthesizing current literature in three uneven layers, the paper shows that most work treats AI as assessment infrastructure, while far less theorizes its implications for construct validity and extrapolation warrants. It defines AI-mediated construct drift as the misalignment that arises when communicative abilities required in the target domain change through AI mediation while test constructs remain anchored to an unaided-performance model. It proposes bounded AI mediation as a validity-oriented design principle: a standardized condition in which all test takers access the same institutionally controlled AI assistant, with predefined assistance boundaries, logged interactions, and tasks that distinguish comprehension support from answer generation. The paper argues that score interpretations should be narrowed and supplemented when used to support claims about AI-mediated academic communication. Keywords: construct validity; English language assessment; artificial intelligence; large language models; target language use; extrapolation; authenticity; consequential validity; construct drift; bounded AI mediation Toward Bounded AI Mediation as a Validity-Oriented Design Principle High-stakes English language proficiency tests have historically played a gatekeeping role in international higher education. Universities, professional programs, and immigration systems use TOEFL, IELTS, the Duolingo English Test, Pearson PTE, and related assessments to support decisions about whether test takers have sufficient English proficiency to study, work, or participate in English-medium institutions. These tests differ in format, scoring, delivery, and theoretical tradition, but they share a broad interpretive logic: performance under standardized test conditions is used to support a score interpretation about academic English proficiency, and that interpretation is then extrapolated to English-medium academic or professional communication. The rapid spread of generative artificial intelligence (AI) and large language models (LLMs) has placed pressure on this interpretive logic. Much public and professional discussion has treated AI as a problem of cheating, plagiarism, automated scoring, test preparation, or test security. Those concerns are real. Test providers must attend to security, fairness, proctoring, data privacy, score comparability, and the reliability of AI-supported scoring systems. Yet for language testing, the sharper measurement problem is whether AI is changing the target language use (TLU) domain to which English proficiency scores are interpreted to generalize. In academic settings, students increasingly read with AI assistance, draft and revise with feedback tools, summarize complex texts, translate across languages, prepare oral presentations with language support, and collaborate with AI systems during writing, research, and study. These practices are unevenly distributed, variably regulated, and still evolving. They are not uniformly legitimate in all institutions or assignments. Nevertheless, when such practices become part of the communicative ecology of academic work, the extrapolation warrant linking unaided standardized performance to academic communicativ

FAQs

How does AI change the validity of using unaided English test scores for academic readiness claims?

AI-mediated communication changes the target language use domain. As AI assistance becomes part of academic practice, extrapolating from unaided standardized performance to AI-mediated readiness becomes less self-evident.

What is AI-mediated construct drift as defined in the paper?

AI-mediated construct drift is the misalignment that arises when communicative abilities required in the target domain change through AI mediation while test constructs remain anchored to an unaided-performance model.

What does bounded AI mediation propose for future validity-oriented design and research?

Bounded AI mediation is a validity-oriented design principle: a standardized condition where all test takers access the same institutionally controlled AI assistant, with predefined assistance boundaries, logged interactions, and tasks that distinguish comprehension support from answer generation.

Jul 16, 2026

Upload by Evangeline