Time as a geometric property of the latent space - An empirical study of Time-and-Order RoPE in Knowledge Tracing

Oct 15, 2026

Knowledge Tracing models estimate students’ evolving knowledge states, but attention-based architectures such as SAINT and SAKT often fail to extrapolate to longer sequences than seen during training. The study targets two root causes: absolute learnable positional embeddings that limit length generalization, and methods that assume uniformly spaced interactions instead of modeling continuous temporal dynamics of learning and forgetting. It proposes Time-and-Order RoPE (TO-RoPE) as geometric latent-space rotations capturing discrete order and real time, then evaluates on ASSISTments2017, EdNet, and a proprietary dataset, improving AUC over long windows up to 500 interactions.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages12

Reading Time30 min

AccessFree to download

Introduction

Background on Knowledge Tracing and Transformer Models

Limitations of Absolute Positional Embeddings and Uniform Timing

Motivation for Length Extrapolation

Proposed Solution: TO-RoPE

Time as a geometric property of the latent space: An empirical study of Time-and-Order RoPE in Knowledge Tracing William O. C. Silva UFRPE [williamcostac@gmail.com](williamcostac@gmail.com)[ ](williamcostac@gmail.com)André Nascimento UFRPE [andrecamara@gmail.com](andrecamara@gmail.com) Filipe Dwan Pereira UFRR [filipe.dwan@ufrr.br](filipe.dwan@ufrr.br) Luiz Rodrigues UTFPR [luizrodrigues@utfpr.edu.br](luizrodrigues@utfpr.edu.br) Gabriel Alves UFRPE gabriel.alves@ufrpe.br Mario A. P. Santos Cogna Educação S.A. [mario.a.santos@cogna.com.br](mario.a.santos@cogna.com.br) Tyagi M. Lima Cogna Educação S.A. [tyagi.lima@cogna.com.br](tyagi.lima@cogna.com.br) Rafael Ferreira Mello UFRPE CESAR School [rafael.mello@ufrpe.br](rafael.mello@ufrpe.br) ABSTRACT Knowledge Tracing (KT) models are essential for estimating student knowledge states, yet prominent attention-based architectures like SAINT and SAKT struggle with length extrapolation, often failing when sequence lengths exceed those seen during training. This limitation frequently arises from the use of absolute learnable positional embeddings. Furthermore, existing methods often treat interactions as uniformly spaced, failing to capture the continuous temporal dynamics of learning and forgetting. This paper investigates the incorporation of Time-and-Order rotation Positional Encoding (TO-RoPE), which treats both discrete interaction order and continuous real time as geometric properties of the latent space. By applying deterministic rotations to queries and keys, TO-RoPE allows for robust extrapolation to longer sequences while capturing heterogeneous temporal patterns. We conducted an empirical study across three datasets: ASSISTments2017, EdNet, and a Proprietary Dataset. The results demonstrate that replacing absolute embeddings with TO-RoPE significantly improves the AUC performance of SAINT and SAKT in extrapolated windows of up to 500 interactions. In particular, the SAINT + TOR configuration consistently outperformed the baselines, suggesting that explicit geometric modeling of continuous time is a viable approach to modeling long-range dependencies in digital education environments. Keywords Knowledge Tracing, Length Extrapolation, Time-aware Positional Encoding Willian Costa, Filipe Dwan Pereira, Gabriel Alves, Andr´e Nascimento, Luiz Rodrigues, Mario A. P. Santos, Tyagi M. Lima, and Rafael Ferreira Mello. Time as a geometric property of the latent space: An empirical study of Time-and-Order RoPE in Knowledge Tracing. In Anthony Botelho, Maria Mercedes T. Rodrigo, Adish Singla, Hiroaki Ogata, Hyojeong So, and Young Hoan Cho (eds.) Proceedings of the 19th International Conference on Educational Data Mining, Seoul, Republic of Korea, June, 2026, pp. 317–328. International Educational Data Mining Society (2026) . © 2026 Copyright is held by the author(s) . This work is distributed under the Creative Commons Attribution NonCommercial NoDerivatives 4.0 International (CC BY-NC-ND 4.0) license. [https://doi.org/10.5281/zenodo.21039935](https://doi.org/10.5281/zenodo.21039935) 1. INTRODUCTION The aim of personalized education at scale is fundamentally based on our ability to model learners’ evolving cognitive states [30] . Knowledge Tracing (KT), the computational task of predicting student future performance based on historical interactions, serves as the analytical engine for personalized solutions, such as Intelligent Tutoring Systems (ITS) and Adaptive Systems [2, 16] . By accurately mapping a student’s mastery over specific Knowledge Components (KCs), these systems can transition from static, one-size-fitsall curricula to adaptive environments that provide targeted feedback and optimized learning paths. Furthermore, educational platforms have begun to generate large volumes of data on student interactions, opening opportunities for analytical techniques to enhance the learning experience [23, 11, 3] . In this context, rece

FAQs

What problem does the paper address in Knowledge Tracing models like SAINT and SAKT?

It addresses poor length extrapolation when sequence lengths exceed training ranges, and it highlights that existing approaches do not capture continuous temporal learning and forgetting dynamics.

How does TO-RoPE represent time and order in the model?

TO-RoPE treats discrete interaction order and continuous real time as geometric properties in the latent space, using deterministic rotations applied to queries and keys.

Which datasets and evaluation metric are used to measure the proposed method?

The method is evaluated empirically on ASSISTments2017, EdNet, and a proprietary dataset, using AUC performance improvements on extrapolated windows up to 500 interactions.

Oct 15, 2026

Uploaded by Aria Callaghan