Comparison of Threshold Values of Three-Tier Diagnostic and Multiple-Choice Tests Based on Response Time

Oct 15, 2026

Uploaded by Dipper

Study investigates item Response Time Fidelity scores (RTFs), KR20, and Cronbach’s Alpha for threshold points between 1 and 30 seconds, using descriptive research with pre-service science teachers in Turkey (n=115). Two groups compared Three-Tier Diagnostic Test (3TDT) and Multiple-Choice Test (MCT) with 63 items on chemical concepts, evaluating validity, reliability, and threshold estimates, then performing risk analyses via Cox Proportional Hazards Model. Results indicate response times support measurement tool validity and reliability, and examinees spend less time in Tier-II of 3TDT.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages18

Reading Time45 min

AccessFree to download

INTRODUCTION

Purpose and research framing

Multiple-Choice Tests (MCT): strengths and limitations

Tier Diagnostic Tests (TDT): rationale and misconceptions

Study context and future directions

Anatolian Journal of Education e-ISSN: 2547-9652 October 2020 ● Vol.5, No.2 [www.e-aje.net](www.e-aje.net)[ ](www.e-aje.net)pp. 19-36 Comparison of Threshold Values of Three-Tier Diagnostic and Multiple-Choice Tests Based on Response Time Suat Türkoguz Assoc. Prof., Dokuz Eylül Universty, Turkey, [suat.turkoguz@gmail.com](suat.turkoguz@gmail.com) This study aimed to investigate the item Response Time Fidelity scores (RTFs), KuderRichardson Reliability (KR20) and Cronbach’s Alpha Reliability (α) coefficients, calculate KR20 coefficients with RTFs for 30 threshold points between 1 and 30 seconds and compare these values on plots, estimate the threshold point in the literature to support these plots and conduct risk analyses with Cox Proportional Hazards Model (Cox-HM) on the initial data and the new data determined by the threshold value. This study is a descriptive research. The participants were pre-service science teachers at a state university in Turkey (n:115) . The study was conducted in two groups. In Group-1 (n:57), a Three-Tier Diagnostic Test (3TDT) was performed, and in Group-2 (n:58), a Multiple-Choice Test (MCT) was applied. Tests consisted of 63 items and chemical concepts. As a result, it was found that item response times could be used in the validity and reliability studies of measurement tools. Moreover, examinees spent less time and effort in Tier-II of 3TDT. In future research, response times and RTFs may have roles in development of 3TDTs about different concepts. Moreover, these may be used in Rasch and Parametric Logistic Analysis for 3TDT. This study may be repeated using different psychometric scales. Keywords: tier-diagnostic, threshold, reliability, cox-hm, chemistry INTRODUCTION Measurement and assessment tools are used to determine the degree of realization of pre-determined learning objectives and outcomes as a result of the teaching process. Multiple-Choice Tests (MCTs) and open-ended questions are widely used testing and assessment tools. With these tests, information about the learning process cannot be determined while measuring the knowledge levels of the students as a result of the teaching process (Akınoğlu, 2011) . Therefore, it will be beneficial to make measurement and assessment based on the process, rather than result-oriented measurement and assessment (Bektaş & Kudubeş, 2014) . In addition to measuring how many questions students answered correctly in an examination, we need to know how much they have learned. MCTs and open-ended test have advantages and disadvantages. Tier Diagnostic Tests (TDTs), which are thought to overcome the disadvantageous aspects of these tests, have become widespread. Below, the causes and justifications of the advantageous and disadvantaged aspects of MCT and TDT are listed respectively. The reasons for the popular preference of MCTs are that they can be applied to a large number of people and it is easy to prepare and assess. As known, MCTs consist of questions and options one which is the correct one, while the others are distracters. However, these tests do not provide an idea ofwhy the examinee has chosen that option. The lack of reasons for choosing the options oftest items may constitute a problem in terms of understanding the ways in which concepts are constructed. Furthermore, it may prevent the determination of the student’s misconceptions. MCTs are fast in terms Citation: Türkoguz, S. (2020) . Comparison of Threshold Values of Three-Tier Diagnostic and Multiple-Choice Tests Based on Response Time. Anatolian Journal of Education, 5(2), 19-36. [https://doi.org/10.29333/aje.2020.522a](https://doi.org/10.29333/aje.2020.522a) 20 Comparison of Threshold Values of Three-Tier Diagnostic … of testing and assessment, but it is disadvantageous in terms of the chance factor and not explaining the reasons of choosing particular options. Recently, various studies have been carried out about preparation in a short time, assessment and appl

FAQs

What reliability measures and coefficients are analyzed in the study?

The study analyzes item Response Time Fidelity scores (RTFs), KuderRichardson Reliability (KR20), and Cronbach’s Alpha (α).

How are the participants divided and which test types are used?

Participants are pre-service science teachers (n=115) split into two groups: Group-1 performs the Three-Tier Diagnostic Test (3TDT, n=57) and Group-2 uses the Multiple-Choice Test (MCT, n=58).

What is the role of response time in the study’s conclusions?

Item response times are found usable in validity and reliability studies of measurement tools, and examinees spend less time and effort in Tier-II of 3TDT.

Oct 15, 2026

Uploaded by Dipper