Validity, Reliability, and Concordance of the Duolingo English Test - Technical Report - University of Pittsburgh May 2014

Aug 29, 2026

Upload by Rhys

Validity, reliability, and score concordance evidence for the Duolingo English Test (DET) is evaluated using an independently conducted study at the University of Pittsburgh in early 2014. Participants included Duolingo users learning English, international students at U.S. universities, and individuals who had taken the TOEFL in China, all aged 18+ with recent TOEFL iBT results. Surveys collected demographics and TOEFL data, followed by DET completion, with a two-week retest and second survey. Results show DET scores substantially correlate with TOEFL total scores, moderately with TOEFL sections (stronger for Speaking/Writing), demonstrate strong test-retest reliability, and enable equipercentile linking to comparable TOEFL iBT scores for practical score interpretation.

LanguageEnglish

CategoryExam

FormatPDF

Pages11

Reading Time28 min

AccessFree to download

Introduction

Method

Data collection

Sample description

Data analysis

Results

Descriptive statistics of the sample

External structure evidence for validity

Test-retest reliability

Equipercentile linking results

Conclusion

References

Appendix

Validity, Reliability, and Concordance of the Duolingo English Test Technical Report, University of Pittsburgh May 2014 (11 pages) [englishtest.duolingo.com/resources](englishtest.duolingo.com/resources) Feifei Ye This report refers to an older version of the Duolingo English Test (DET) . More recent research and information on the current test can be found here . Contents 1 Introduction 2 2 Method 3 2. 1 Data collection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 2.2 Sample description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 2.3 Data analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 3 Results 5 3.1 Descriptive statistics of the sample . . . . . . . . . . . . . . . . . . . . . . . 5 3.2 External structure evidence for validity . . . . . . . . . . . . . . . . . . . . . 6 3.3 Test-retest reliability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 3.4 Equipercentile linking results . . . . . . . . . . . . . . . . . . . . . . . . . . 8 4 Conclusion 9 References 10 Appendix 11 Corresponding author: Feifei Ye, PhD Assistant Professor, School of Education University of Pittsburgh, Pittsburgh PA USA Email: [feifeiye@pitt.edu](feifeiye@pitt.edu) © 2014 Feifei Ye. All rights Reserved. Executive Summary Duolingo has developed a computer adaptive test of English competency for non-native English learners . This research study of the validity and reliability of the Duolingo English Test was independently conducted from February-April of 2014 . The study lasted for approximately eight weeks . Participants were recruited from Duolingo users who studied English, international students in several United States universities, and people who took Test of English as Foreign Language (TOEFL) at several TOEFL centers in China . Participants were at least 18 years of age and had taken the TOEFL within the last 18 months . Participants ﬁlled out a survey in the beginning of the study to provide information of their demographics such as age, gender, native language, and education background, experience in learning English, and most importantly, the date and location of taking the TOEFL iBT as well al their TOEFL scores . The participants were directed to complete the Duolingo English Test at the end of the survey. Two weeks later, the participants were instructed via email to ﬁll out a second survey and the Duolingo English Test for the second time . Main results: • Participants’ scores on the Duolingo English Test correlated substantially with their TOEFL total scores, and moderately with TOEFL section scores, with higher correlations for the TOEFL Speaking and Writing sections . This provides criterionrelated evidence of validity of the Duolingo test scores . • Participants’ scores on the ﬁrst Duolingo English Test correlated highly with their scoreson a second test two weeks later, indicating that Duolingo test scores are reliable with a good test-retest reliability coefﬁcient. • Duolingo English Test scores were linked with TOEFL total scores to ﬁnd the comparable scores from the two tests that have similar percentile rank. Duolingo English Test scores are on a scale of 0–100 and TOEFL scores are on a scale of 0–120 . For international students to apply for studying in US universities, the minimum cut-off score of TOEFL iBT is 80 and a more selective cut-off score is 100, corresponding to scores 50 and 72 respectively on the Duolingo English Test. 1 Introduction Duolingo is a free language-learning website which became publicly available in 2012 . Duolingo has recently developed a computer adaptive test of English competency for nonnative English learners . The goal of the current research study was to independently evaluate some of the evidence for the validity and reliability of Duolingo English Test scores, and to link Duolingo English Test scores to TOEFL iBT scores . This study was designed and

FAQs

How was data collected and how did the retest work in the study?

Participants completed a first survey with demographics and TOEFL information, then took the Duolingo English Test. Two weeks later they received an email to complete a second survey and took the test again.

What evidence supports validity of DET scores?

DET scores showed substantial correlation with TOEFL total scores and moderate correlations with TOEFL section scores, providing criterion-related evidence for validity, with higher correlations for Speaking and Writing.

How were DET and TOEFL iBT scores made comparable?

The study used equipercentile linking to map DET 0–100 scores to comparable TOEFL iBT total scores on a 0–120 scale, enabling interpretation of cutoff-style score equivalents.

Aug 29, 2026

Upload by Rhys