Score Gains and Validity Evidence for English Learners Testing with Supports on the ACT - Working Paper 2021-R2124

Oct 9, 2026

Uploaded by Jacob

Study provides validity evidence for English learners (ELs) taking the ACT® with testing supports (accommodations). Three groups were compared: ELs retesting with supports, ELs testing twice without supports, and non-ELs testing twice without supports. ELs retesting with supports showed the lowest ACT performance yet the largest score gains, especially in reading. Links between high school grades and ACT scores were higher on the second attempt for supported ELs, mainly in reading and science. Psychometric checks across a second sample showed generally high classification accuracy and consistency, minimal total-score impact despite some DIF. Findings indicate supports reduce construct-irrelevant variance without creating unfair advantage.

LanguageEnglish

CategoryExam

FormatPDF

Pages39

Reading Time1 h 38 min

AccessFree to download

Objectives

Theoretical Framework

Testing Supports and Construct-Irrelevant Variance

Working Paper 2021-R2124 Score Gains and Validity Evidence for English Learners Testing with Supports on the ACT JOANN L. MOORE, PHD , DONGMEI LI, PHD, AND YANG LU, PHD [ACT.org](ACT.org) Score Gains and Validity Evidence for English Learners Testing with Supports on the ACT Joann L. Moore, PhD, Dongmei Li, PhD, and Yang Lu, PhD Abstract The purpose of this study was to provide validity evidence for English learners (ELs) taking the ACT® using testing supports. Three groups were compared to evaluate performance, score gains, and relationships between ACT scores and high school grades: ELs who took the ACT without testing supports and retested using testing supports, ELs who tested twice without supports, and non-ELs who tested twice without supports. ELs who retested with supports had the lowest ACT performance, followed by ELs who tested twice without supports, and non-ELs had the highest performance. Score gains were highest for ELs who retested with supports, especially in reading. Relationships between high school grades and ACT scores were higher on the second test attempt for ELs who retested with supports, particularly in reading and science. A second sample was used to evaluate psychometric properties of scores for ELs who tested with supports compared to ELs who tested without supports and non-ELs who tested without supports. Classification accuracy and consistency rates were generally high across groups. Some evidence of DIF was found between ELs who tested with supports and non-ELs, but about half favored the focal group and half favored the reference group; therefore, the impact on total scores is expected to be minimal. CSEM was virtually identical across groups, while score reliability and SEM were both lower for ELs who tested with supports compared to the other two groups. This study provides evidence that testing supports are providing a benefit to ELs and removing construct-irrelevant variance without conferring an unfair advantage. Future research is needed to compile additional validity evidence and examine the impact of the supports on predicting college performance. Keywords: English learners, accommodations, testing supports, standardized testing, admissions testing Copyright ® 2021 by ACT, Inc. All rights reserved. | R2124 Score Gains and Validity Evidence for English Learners Testing with Supports on the ACT Objectives In 2017, ACT began providing testing supports (or accommodations) to eligible English learners (ELs) in the US taking the ACT® test, including translated test instructions, use of word-to-word bilingual dictionaries, small-group testing, and extra time. The goal of these supports is to remove construct-irrelevant variance and allow students with limited English proficiency to more accurately demonstrate their true achievement level. The purpose of this study is to investigate the effect of these supports on removing construct-irrelevant variance, thus resulting in ACT scores that more accurately reflect ELs’ achievement level in the subject being assessed. Theoretical Framework The US Department of Education defines an EL as an individual between the ages of 3 and 21 who is enrolled or planning to enroll in elementary or secondary education; whose native language is not English; and whose lack of proficiency in reading, writing, speaking, or listening may deny them the ability to meet state academic standards; succeed in classrooms where instruction is in English; or be able to fully participate in society (US Department of Education, 2016). This is the definition of EL that is used by ACT and throughout this paper, although it should be noted that the procedures for identifying ELs (e.g., choice of English proficiency assessment, choice of cut scores) are determined at the district or state level. Approximately five million students in the US are identified as English learners (ELs), and the percentage of ELs has increased from 9.2% in 2010 to 10.2% in 20

FAQs

What testing supports were investigated for English learners on the ACT?

The study focuses on supports such as translated test instructions, word-to-word bilingual dictionaries, small-group testing, and extra time, designed to reduce construct-irrelevant variance for ELs.

How were groups compared in the study?

Three groups were compared: ELs who took the ACT without supports and retested with supports, ELs who tested twice without supports, and non-ELs who tested twice without supports.

What did the study find about score gains and relationships with high school grades?

Score gains were highest for ELs who retested with supports, particularly in reading. Relationships between high school grades and ACT scores were higher on the second test attempt for supported ELs, especially in reading and science.

Oct 9, 2026

Uploaded by Jacob