Spring 2021 NSCAS Phase I Pilot ELA - Mathematics - Science Technical Report - October 25, 2021

Oct 15, 2026

Technical report documenting the Spring 2021 NSCAS Phase I pilot across ELA, mathematics, and science, released on October 25, 2021. The report covers NWEA operational content and psychometrics, including the validity argument, test design and development (standards, blueprints, item types, DOK, ALD development), and administration and security processes. It details scoring and reporting, constraint-based engine simulations, and psychometric analyses such as classical item statistics and DIF, concluding with calibration and science field-test considerations.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages169

Reading Time7 h 6 min

AccessFree to download

Executive Summary

Introduction

NSCAS Overview

Background

Schedule of Major Events

Building a Validity Argument

Test Design and Development

Test Design

Academic Content Standards

Blueprints

Item Types

Depth of Knowledge (DOK)

ALD Development

ELA Passage Development

Item Development

Content Alignment

Universal Design

Sensitivity and Fairness

Test Construction

Data Review of Field Tested Items

Test Administration and Security

User Roles and Responsibilities

Administration Training

Item Type Samplers

Accommodations and Accessibility Features

User Acceptance Testing (UAT)

Student Participation

Test Security

Partner Support

Scoring and Reporting

Scoring Rules

Paper-Pencil Scoring

Score Reporting Methods

Report Summary

Report Process

Constraint-Based Engine

Overview

Engine Simulations and Evaluation

Psychometric Analyses

Number of Student Included in the Analyses

Classical Item Analyses

Differential Item Functioning (DIF)

IRT Calibration

Science Field Test

Spring 2021 NSCAS Phase I Pilot ELA, Mathematics, and Science Technical Report October 25, 2021 NWEA Operational Content and Psychometrics © 2021 Nebraska Department of Education. No part of this publication may be reproduced or distributed in any form or by any means, or stored in a database or retrieval system, without the prior written permission of the publisher. Table of Contents Executive Summary ......................................... 7 1. Introduction ............................................ 10 1.1 NSCAS Overview ..................................... 10 1.2 Background ......................................... 11 1.3 Schedule of Major Events ................................. 11 1.4 Building a Validity Argument ................................ 12 1.4.1 Intended Purposes and Uses of Test Results .................. 13 1.4.2 Theory of Action .................................. 14 2. Test Design and Development .................................. 16 2.1 Test Design ......................................... 16 2.2 Academic Content Standards ............................... 17 2.3 Blueprints .......................................... 17 2.4 Item Types ......................................... 18 2.5 Depth of Knowledge (DOK) ................................ 18 2.6 ALD Development ..................................... 19 2.6.1 Policy ALDs ..................................... 19 2.6.2 Range ALDs .................................... 20 2.6.2.1 ELA and Mathematics .......................... 20 2.6.2.2 Science .................................. 22 2.6.3 Reporting ALDs .................................. 23 2.7 ELA Passage Development ................................ 23 2.8 Item Development ..................................... 23 2.8.1 Item Speciﬁcations ................................. 24 2.8.2 Item Retirement .................................. 24 2.9 Content Alignment ..................................... 24 2.9.1 Alignment and Adaptive Testing .......................... 25 2.9.2 2019 Mathematics Alignment Study ....................... 26 2.10 Universal Design ...................................... 26 2.11 Sensitivity and Fairness .................................. 26 2.12 Test Construction ...................................... 27 2.13 Data Review of Field Tested Items ............................ 27 3. Test Administration and Security ................................ 30 3.1 User Roles and Responsibilities ............................. 30 3.2 Administration Training ................................... 31 3.3 Item Type Samplers .................................... 32 3.4 Accommodations and Accessibility Features ....................... 32 3.5 User Acceptance Testing (UAT) .............................. 35 3.6 Student Participation .................................... 36 3.6.1 Paper-Pencil Participation Criteria ........................ 37 3.6.2 Participation of English Language Learners (ELLs) ............... 37 3.6.3 Participation of Recently Arrived Limited English Proﬁcient Students ..... 37 3.7 Test Security ........................................ 38 3.7.1 Test Security .................................... 38 3.7.1.1 Physical Warehouse Security...................... 38 3.7.1.2 Secure Destruction of Test Materials.................. 38 3.7.1.3 Shipping Security............................. 38 3.7.1.4 Electronic Security of Test Materials and Data............. 39 3.7.2 Caveon Test Security ............................... 39 3.7.2.1 Monitoring for Disclosure of Test Content................ 39 3.7.2.2 Monitoring for Potential Test Security Violations............ 39 3.8 Partner Support ...................................... 41 4. Scoring and Reporting ...................................... 43 4.1 Scoring Rules ........................................ 43 4.2 Paper-Pencil Scoring ....................................

FAQs

What does the Spring 2021 NSCAS Phase I pilot technical report cover?

It covers technical aspects of the pilot for ELA, mathematics, and science, including operational content and psychometric methodology, test design and development, administration and security, scoring and reporting, and subsequent analyses.

How is test validity addressed in the report?

The report includes a section on building a validity argument, covering intended purposes and uses of test results as well as the theory of action.

Which psychometric analyses are described?

The report outlines classical item analyses (e.g., item difficulty and discrimination), differential item functioning using logistic regression and Mantel-Haenszel methods, and IRT calibration methods and results, followed by science field test elements.

Oct 15, 2026

Uploaded by Alex Sinclair