Identifying Rapid-Guessing Behaviors - Response Time Threshold - Item Response Theory - Machine Learning Methods

Aug 4, 2026

Low-stakes assessments can attract unmotivated test takers who exhibit disengaged patterns, including rapid responses with short response times, threatening the validity of score inferences. The dissertation compares multiple rapid-guessing detection approaches, including response time threshold methods, item response theory based methods, and a neural-network autoencoder using anomaly detection. A simulation study evaluates (a) classification accuracy and (b) motivation filtering effects on person and item parameter estimates, with extensions to PISA data and additional gradient boosting analyses for process variables.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages188

Reading Time7 h 54 min

AccessFree to download

Acknowledgements

Abstract

Table of Contents

Chapter 1 Introduction

Background

Statement of the Problem and Its Significance

Purpose of Study

Chapter 2 Literature Review

Rapid-Gue

Identifying Rapid-Guessing Behaviors: Comparison of Response Time Threshold, Item Response Theory, and Machine Learning Methods | Item Type | Dissertation (Open Access) | | --- | --- | | Authors | Lee , Minhyeong | | DOI | 10.7275/36471360 | | Rights | Attribution 4.0 International | | Download date | 2026-04-14 01:09:28 | | Item License | [http://creativecommons.org/licenses/by/4.0/](http://creativecommons.org/licenses/by/4.0/) | | Link to Item | [https://hdl.handle. net/20.500.14394/19461](https://hdl.handle. net/20.500.14394/19461) | Identifying Rapid-Guessing Behaviors: Comparison of Response Time Threshold, Item Response Theory, and Machine Learning Methods A Dissertation Presented by MINHYEONG LEE Submitted to the Graduate School of the University of Massachusetts Amherst in partial fulfillment of the requirements for the degree of DOCTOR OF PHILOSOPHY February 2024 College of Education Research, Educational Measurement, and Psychometrics © Copyright by Minhyeong Lee 2024 All Rights Reserved Identifying Rapid-Guessing Behaviors: Comparison of Response Time Threshold, Item Response Theory, and Machine Learning Methods A Dissertation Presented by MINHYEONG LEE Approved as to style and content by: | Scott Monroe, Chair | | --- | | Stephen G. Sireci, Member | Frederic Robin, Member Shane Hammond Associate Dean for Student Success College of Education ACKNOWLEDGEMENTS I extend my profound appreciation to my academic advisor and the chair of the committee, Dr. Scott Monroe. His guidance, mentorship, and support have played a crucial role in shaping the path of my research and academic pursuits. I am truly grateful to have had him as my advisor during my time at UMass. I express sincere gratitude to Dr. Steve Sireci for his invaluable insights and thoughtful contributions as a committee member. Beyond his academic guidance, Dr. Sireci has opened doors to great opportunities throughout my journey, and I am thankful for the doors he has helped me unlock. A special note of appreciation goes to Dr. Frederic Robin, my final committee member, whose guidance led me into the captivating field of rapid-guessing research. I deeply appreciate his expertise in aberrant testing behaviors and PISA data. My heartfelt gratitude goes out to my family—my mother, father, and sister. Their unwavering support, often expressed simply through their presence, has been my anchor throughout this long journey. To my husband, Tyler Seabury, I extend my deepest thanks for his unwavering support and constant encouragement. His steadfast belief in my capabilities has been my greatest source of strength. I extend my appreciation to my friends and REMP family for their camaraderie and support throughout this challenging yet rewarding journey. Truly, I feel fortunate to have each of you in my life. ABSTRACT Identifying Rapid-Guessing Behaviors: Comparison of Response Time Threshold, Item Response Theory, and Machine Learning Methods February 2024 MINHYEONG LEE, B.A., SUNGKYUNKWAN UNIVERSITY M.A., SUNGKYUNKWAN UNIVERSITY M. S., UNIVERSITY OF MASSACHUSETTS AMHERST Ph.D., UNIVERSITY OF MASSACHUSETTS AMHERST Directed by: Professor Scott Monroe On low-stake tests, unmotivated test takers may show disengaged behaviors, such as rapid responses with short response times and such non-effortful responses can be a threat to the test score validity. Several methods have been proposed to determine rapidguessing behavior, which can be classified as response time threshold methods and item response theory based methods. In addition, I suggest a machine learning method based on neural networks known as the autoencoder for anomaly detection. This research aims to compare the detection of rapid-guessing using various methods and their effects on inferences drawn from test data. In the simulation study,(a) the classification accuracy of the methods and (b) the effects of motivation filtering, i.e., removing r

FAQs

Why is detecting rapid-guessing important in low-stakes tests?

Rapid-guessing behaviors, such as fast responding with short response times, can reflect disengagement. These patterns can threaten the validity of test score inferences.

What classes of rapid-guessing detection methods does the research compare?

The study compares response time threshold methods, item response theory based methods, and a machine learning approach using a neural-network autoencoder for anomaly detection.

How does the dissertation evaluate the methods and their consequences for test inference?

A simulation study compares classification accuracy and assesses how motivation filtering—removing rapid guesses—affects person and item parameter estimates. The work further uses PISA data and gradient boosting to analyze predictive variables.

Aug 4, 2026

Upload by Emma Wilson