From Zero-shot to Self-generated References - Leveraging LLMs for Scoring ENEM Essays

Sep 6, 2026

Upload by Mia

This study investigates the application of Large Language Models (LLMs) to Automated Essay Scoring (AES) in the context of Brazil’s Exame Nacional do Ensino Médio (ENEM). Five state-of-the-art LLMs are evaluated across three prompting scenarios: zero-shot, one-shot using high-score references, and a self-generated reference approach where models create their own ideal reference before scoring. Using the Essay-BR corpus, performance is measured with classification and regression metrics. Results indicate one-shot prompting delivers the best overall metrics, while self-generated references remain a viable option when real references are unavailable.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages11

Reading Time28 min

AccessFree to download

Introduction

Automated Essay Scoring for ENEM

Study Objectives and Contributions

Related Work

From Zero-shot to Self-generated References: Leveraging LLMs for Scoring ENEM Essays Matheus Yasuo Ribeiro Utino 1 , Paulo Mann2 1 Institute of Mathematics and Computer Science, University of So Paulo 2Institute of Computing, Federal University of Rio de Janeiro [matheusutino@usp.br](matheusutino@usp.br) , [paulomannjr@gmail.com](paulomannjr@gmail.com) Abstract. This study investigates the application of Large Language Models (LLMs) to Automated Essay Scoring (AES) in the context of Brazil’s Exame Na cional do Ensino Me´dio (ENEM). We evaluate five state-of-the-art LLMs across three prompting scenarios: zero-shot, one-shot (with high-score references), and a novel self-generated reference approach, where the model generates its own ideal reference before evaluation. Using the Essay-BR corpus, we assess performance using both classification and regression metrics. Results show that one-shot prompting consistently yields the best metrics, while the self-generated reference method presents a viable alternative when no real references are avail able. Our findings highlight the promise of LLMs for educational scoring. 1. Introduction The Exame Nacional do Ensino M e´dio (ENEM) is Brazil’s primary educational assessment, playing a central role in public university admissions and access to government programs such as Sisu, Prouni, and FIES [Inep 2020] . With millions of participants annually, ENEM evaluates a broad range of competencies, including written production, which is graded by human evaluators based on objective criteria. The essay component, in particular, represents a critical stage, as it synthesizes students’ linguistic, argumentative, and socio-cognitive skills. More than a mere evaluation tool, ENEM functions as a mechanism for social mobility and the democratization of higher education access [Pires 2023] . Consequently, accurately predicting student performance, especially in the essay section, can inform the design of public education policies, the personalization of pedagogical interventions, and the early identification of risks related to dropout or poor academic achievement. With recent advances in Large Language Models (LLMs), it has become possible to explore novel approaches to complex tasks requiring sophisticated textual comprehension and production, such as Automated Essay Scoring (AES)[Atkinson and Palma 2025] . These models have demonstrated capabilities not only in classifying texts with high accuracy but also in providing structured textual justifications, making them particularly promising for educational applications where interpretability is a fundamental requirement. These outcomes are closely tied to the emergent abilities of LLMs, which enable them to perform complex reasoning and explanation tasks beyond their training objectives [Berti et al. 2025] . Within this context, the present study investigates the use of LLMs to predict ENEM essay scores, evaluating different usage configurations (zero-shot, one-shot, and a novel approach herein termed self-generated reference) . In addition to analyzing the predictive performance of the models, we explore their capacity to simulate human evaluative behavior, generate patterns of textual excellence, and provide formative feedback. Through this work, we aim to contribute to the advancement of AES in Portuguese, emphasizing scalable, transparent, and pedagogically valuable solutions. The main contributions of this work are: • The proposal of a novel self-generated reference strategy, in which the model creates its own ideal essay based solely on the theme and uses it as a benchmark for evaluation. • A comparative evaluation of five open-source LLMs across three usage configurations (zero-shot, one-shot with real reference, and self-generated reference) in the context of ENEM essay scoring. • The creation and public release of a new synthetic dataset containing LLMgenerated essays, predicted scores, and detailed justifications

FAQs

What prompting strategies are evaluated for ENEM essay scoring?

The study evaluates three strategies: zero-shot, one-shot with high-score references, and a self-generated reference method where the model generates an ideal reference before evaluation.

How is model performance measured in the study?

Performance is assessed on the Essay-BR corpus using both classification and regression metrics, aligned to the scoring task.

What does the study conclude about one-shot vs. self-generated references?

One-shot prompting consistently produces the best metrics, while the self-generated reference approach is a viable alternative when real references are not available.

Sep 6, 2026

Upload by Mia