ARTIFICIAL INTELLIGENCE IN LARGE-SCALE ASSESSMENT PROGRAMS - Applications and Considerations for State Education Agencies

Oct 15, 2026

This paper examines how artificial intelligence, particularly generative AI, can be applied to large-scale assessment programs, with emphasis on statewide summative assessments used for federal accountability. It follows the large-scale assessment lifecycle, detailing how AI may influence each stage from construct definition through content development, field testing and equating, administration, scoring, reporting, and related uses. For each phase, it summarizes key activities, proposes current and potential AI applications, and highlights considerations for state education agencies, focusing on validity, fairness, standardization, security, and human oversight.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages47

Reading Time1 h 58 min

AccessFree to download

Executive Summary

The Large-Scale Assessment Lifecycle

AI and Generative AI

Why AI in Large-Scale Assessment?

Humans in the Loop

AI Applications and Considerations

Construct Definition

Content Development

Field Testing and Equating

Administration

Scoring

Reporting

Other Applications

Conclusion

Implications for State Education Agencies

References

ARTIFICIAL INTELLIGENCE IN LARGE-SCALE ASSESSMENT PROGRAMS: Applications and Considerations for State Education Agencies February 2026 Will Lorié & Nathan Dadey National Center for the Improvement of Educational Assessment, Inc. Dover, New Hampshire ARTIFICIAL INTELL IGENCE IN LARGE-SCALE ASSESSMENT PROGRAMS: Applications and Considerations for State Education Agencies The National Center for the Improvement of Educational Assessment, Inc. (the Center for Assessment) is a New Hampshire based not-for-profit (501(c)(3)) corporation. Founded in September 1998, the Center’s mission is to improve student learning by partnering with educational leaders to advance effective practicesand policies in support of high-quality assessment and accountability systems. The Center for Assessment does this by providing services directly to states, school districts, and partner organizations to support state and district assessment and accountability systems. This work is licensed under the Creative Commons Attribution 4.0 International License (CC BY) . To view a copy of this license, visit [http://creativecommons.org/licenses/by/4.0/](http://creativecommons.org/licenses/by/4.0/) . ACKNOWLEDGMENTS: We acknowledge the support and feedback from André Rupp, Sue Lottridge, Laura Hamilton, Ruhan Circi, Juan D’Brot, and other colleagues whose ideas and suggestions improved this paper. Special thanks to Catherine Gewertz, whose keen insights and thorough edits significantly improved the clarity of our work. Any errors and omissions are our own. SUGGESTED CITATION: Lorié, W., & Dadey, N. (2026) . Artificial intelligence in large-scale assessment programs: Applications and considerations for state education agencies. Dover, NH: The National Center for the Improvement of Educational Assessment. PAGE 2 ARTIFICIAL INTELL IGENCE IN LARGE-SCALE ASSESSMENT PROGRAMS: Applications and Considerations for State Education Agencies TABLE OF CONTENTS EXECUTIVE SUMMARY ....................................................4 INTRODUCTION ..............................................................6 • The Large-Scale Assessment Lifecycle ....................... 6 • AI and Generative AI..................................................... 7 WHY AI IN LARGE-SCALE ASSESSMENT? .......................9 • Humans in the Loop..................................................... 9 AI APPLICATIONS AND CONSIDERATIONS .................11 • Construct Definition ................................................... 11 • Content Development................................................ 15 • Field Testing and Equating ........................................ 21 • Administration ............................................................ 23 • Scoring ......................................................................... 26 • Reporting ..................................................................... 30 • Other Applications...................................................... 34 CONCLUSION ................................................................37 • Implications for State Education Agencies ..............37 REFERENCES ..................................................................39 PAGE 3 ARTIFICIAL INTELLIGENCE IN LARGE-SCALE ASSESSMENT PROGRAMS: Applications and Considerations for State Education Agencies EXECUTIVE SUMMARY This paper explores the potential of artificial intelligence (AI), especially generative AI (GenAI),1 in educational assessment, with a focus on large-scale assessment (LSA) programs—specifically statewide summative assessments designed to fulfill federal accountability requirements. We’ve structured the paper around the LSA life cycle, analyzing how AI currently influences, or could influence, each phase, from defining what we’re measuring to reporting, and how it could support processes that cross stages. For each phase of the life cycle, we outline the main activiti

FAQs

What parts of the large-scale assessment lifecycle does the paper cover?

It covers construct definition, content development, field testing and equating, administration, scoring, reporting, and other applications, aligned to the large-scale assessment lifecycle.

How can generative AI support content development for assessments?

GenAI systems can produce assessment items, synthesize passages and stimuli, and assist human authors by providing feedback on item quality and alignment.

Why does the paper emphasize human responsibility in construct definition?

It argues that defining and validating constructs must remain with human experts to preserve validity, fairness, and interpretive coherence of test scores.

Oct 15, 2026

Uploaded by Taylor Morgan