Analyzing the Equity of the Brazilian National High School Exam - Validating the Item Response Theory’s Invariance

Sep 5, 2026

Several studies examine how economic, racial, and gender circumstances affect student performance in large-scale entrance exams such as Brazil’s ENEM. This work uses Item Response Theory to test the invariance premise, checking whether item characteristic curves are similar across subpopulations defined by gender, race, and income irrespective of students’ true abilities. It analyzes each group’s observed curve properties and applies a nonparametric ranking test to compare equity per item. Results show Languages and Codes questions favor male, white, and high-income participants, while Mathematics, Natural Sciences, and Human Sciences are more egalitarian.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages5

Reading Time13 min

AccessFree to download

Abstract

Keywords

1. Introduction

2. Data

Analyzing the Equity of the Brazilian National High School Exam by Validating the Item Response Theory’s Invariance Vitoria Guardieiro1 , Marcos M. Raimundo1 ,2 , Jorge Poco1 1 Getulio Vargas Foundation; 2 Universidade Federal do Rio de Janeiro [vitoriaguardieiro@gmail.com](vitoriaguardieiro@gmail.com) ; [marcosmrai@gmail.com](marcosmrai@gmail.com) ; [jorge.poco@fgv.br](jorge.poco@fgv.br) ABSTRACT Several studies adopt different approaches to examining how economic, racial, and gender circumstances influence student performance in large-scale entrance exams, such as the National High School Exam (ENEM) . Using a methodology based on Item Response Theory, ENEM’s exam attempts to assess, for each item (question), the curve (function) that relates the participants’ abilities to their probabilities of correctly answering the item, which is assumed to hold whichever subgroup, a fundamental premise of IRT called invariance. This work analyzes whether the ENEM 2019 test presented similar curves for subpopulations defined by gender, race, and income, regardless of the participant’s actual abilities. Our approach is to analyze the properties of the observed curve for each group and then perform a nonparametric ranking test to compare the equity of each item (question) for each analyzed characteristic. We found that the ”Languages and Codes” questions consistently favored male, white, and high-income participants. At the sametime, the other three sets of questions (Mathematics, Natural Sciences, and Human Sciences) were considerably more egalitarian. Keywords Higher education entrance exams, Grading equity, Item Response Theory, Educational Data Mining 1. INTRODUCTION The Brazilian National High School Exam (ENEM, for its initials in Portuguese) is one of the most extensive entrance exams globally, having over 5 million participants registered in 2019 [11] . The exam has several functions; on an individual scale, it serves as an admission test to access the federal universities (through the Unified Selection System or SISU) and access to the federal scholarship programs (University for All Program or ProUni) . On a collective scale, this exam allows a comparison between schools and municipalities, and it also serves as an indicator for national public V. Guardieiro, M. M. Raimundo, and J. Poco. Analyzing the equity of the Brazilian national high school exam by validating the item response theory’s invariance. In A. Mitrovic and N. Bosch, editors, Proceedings of the 15th International Conference on Educational Data Mining, pages 583–587, Durham, United Kingdom, July 2022 . International Educational Data Mining Society. © 2022 Copyright is held by the author(s) . This work is distributed under the Creative Commons Attribution NonCommercial NoDerivatives 4.0 International (CC BY-NC-ND 4.0) license. [https://doi.org/10.5281/zenodo.6852946](https://doi.org/10.5281/zenodo.6852946) educational policies at the national level [10] . Several studies investigated how sensible characteristics such as income, race, gender, and locality, affect the participants’ score [13, 20, 19] . However, most studies use the grade obtained asa direct indicator of the participants’ ability without investigating whether exams’ grading methodologies are unfairly favoring or disfavoring specific subpopulations. Since 2009, ENEM’s participants’ grades have been assigned using Item Response Theory (IRT) methods, which consider the difficulty of each participant’s correct questions [7] to assign the grades, in contrast to the Classical Test Theory, where only the number of correct answers matter [4] . IRT creates a probability function that gives, for each question, the probability of a correct answer given the participant’s ability. Moreover, the primary assumption of IRT theory is that such function does not vary independently of subgroups of students [7, 16] . This assumption means that, given two groups based on a specific characteristic (e.g

FAQs

What invariance assumption does the paper validate in ENEM grading?

The study validates the Item Response Theory invariance premise, meaning item curve functions should not vary independently across subgroups. If subgroup-specific curves differ for many items, grading may systematically overestimate or underestimate groups’ abilities.

How does the methodology compare equity across gender, race, and income?

The method approximates item characteristic curves from ENEM’s IRT-evaluated grades and answers, then compares curve-related properties per group. It uses a nonparametric ranking test to check whether observed item behavior is statistically consistent across the whole test.

Which subjects showed the most consistent bias according to the findings?

The “Languages and Codes” questions consistently favored male, white, and high-income participants. The other three sets—Mathematics, Natural Sciences, and Human Sciences—were considerably more egalitarian.

Sep 5, 2026

Upload by Chumphorn