Second Language Acquisition Modeling - Duolingo SLAM Report

Aug 29, 2026

This report addresses Second Language Acquisition Modeling by predicting which words a learner is likely to answer incorrectly and which words they are likely to get right. Using a Duolingo dataset from the first 30 days of learning, it presents three neural model implementations: a Deep Key-Value Memory Network (DKVMN), a Bi-directional LSTM CNN (BLSTM) model, and an Encoder-Decoder approach. Results are comparable to state-of-the-art performance shown on Duolingo’s leaderboard, indicating BLSTM sequence-to-sequence modeling can match and sometimes exceed more specialized methods.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages10

Reading Time25 min

AccessFree to download

Introduction

Task definition and dataset

Knowledge tracing background

Modeling design choices

Related Work

Item Response Theory (IRT)

Deep Knowledge Tracing (DKT)

Dynamic Key-Value Memory Networks (DKVMN)

Second Language Acquisition Modeling-Duolingo Hima Dureddy, George Larionov, Xin Qian Carnegie Mellon University fglariono,hdureddy,[xinq](xinqg@cs.cmu.edu)[g](xinqg@cs.cmu.edu)[@cs.cmu.edu](xinqg@cs.cmu.edu) Abstract This report tackles Second Language Acquisition Modeling, which is the task of learning to predict which words a student learning a new language will make mistakes on and which he or she is likely to get right. Using a dataset from Duolingo, a mobile language learning app, we describe three model implementations which have been developed to solve this task in a logical fashion: a Deep Key-Value Memory Network (DKVMN), a Bi-directional LSTM CNN (BLSTM), and an Encoder-Decoder model. We achieve comparable results to the state of the art (as seen on the Duolingo leaderboard), showing that a generic BLSTM sequenceto-sequence model performs roughly as well as, and in some cases outperforms, more complex models designed speciﬁcally for the task. 1 Introduction This paper focuses on the 2018 Duolingo Shared Task on Second Language Acquisition Modeling (SLAM) . Duolingo provides a large dataset which contains the results of about 6,000 students learning a second language over the course of their ﬁrst 30 days using the Duolingo app. From the ofﬁcial task site: “the goal of this task is to predict future mistakes that learners of English, Spanish, and French will make based on a history of the mistakes they have made in the past.” The related task of knowledge tracing has seen signiﬁcant study for some time. Several methods have been proposed to tackle the knowledge tracing problem, the most useful among them being Item Response Theory, Bayesian Knowledge Tracing, and various Neural Network architectures [8] [6][10] . We build on this research by utilizing more advanced neural network architectures than [6], who use a relatively simple LSTM model, tweaking existing models such as [10], and introducing a state-of-the-art sequence labeling model to see if it is up to the task. Most knowledge tracing tasks provide a dataset of assignments labeled with a single right/wrong answer each, whereas Duolingo provides full sentences translated by students where each word is labeled as right/wrong. Therefore to adapt KT algorithms we must treat each word as a separate assignment, which greatly increases the complexity of the problem as there are many more words in the Duolingo dataset than there are assignments in other datasets. 2 Related Work 2.1 Item Response Theory (IRT) IRT has been studied for decades, mostly in the context of standardized testing. IRT stores a single number for each student, which is the student's proﬁciency or ability over the course of completing several assessments. IRT assumes that this score does not change while taking the assessments. In the baseline version of IRT, each assignment is assigned a difﬁculty parameter and the probability that a student answers a given item correctly is determined by subtracting the difﬁculty of the item from the proﬁciency score of the student and then passing it through some function f , where f is a sigmoidal function, often the logistic function. This means that IRT often corresponds to structured Table 1: Training data statistics of different streams of Duolingo data | Track | | Users | Exercises | Unique tokens | Unique POS tags | Positive Label Ratio | | --- | --- | --- | --- | --- | --- | --- | | es | en | 2643 | 731896 | 2525 | 15 | 14.1 | | en es | | 2593 | 824012 | 1967 | 17 | 12.6 | | fr en | | 1213 | 326792 | 1941 | 17 | 16.2 | logistic regression. The parameters of the model are learned by maximizing the posterior log probability given the response data [8] . There has been a variety of augmented IRT models proposed, which attempt to take into account extra information such as groupings of assignments into topics or concepts (Hierarchical IRT), students forgetting things over time (Temporal IRT), etc. [8] . 2.2 Deep Knowled

FAQs

What problem does Second Language Acquisition Modeling (SLAM) target in this report?

It predicts which words a student learning a new language will likely get wrong and which they will likely get right, based on a history of past mistakes.

What dataset source is used for training and evaluation?

The models use a Duolingo dataset collected from about 6,000 students over the first 30 days, with word-level right/wrong labels derived from student translations.

Which models are implemented and compared in the report?

The report implements three approaches: DKVMN, a Bi-directional LSTM CNN (BLSTM), and an Encoder-Decoder model.

Aug 29, 2026

Upload by Alex Sinclair