IN-CONTEXT ITEMS IN A NATION WIDE EXAMINATION: WHICH KNOWLEDGE AND SKILLS ARE ACTUALLY ASSESSED?

Sep 6, 2026

Upload by Jacob

Over seven million Brazilian students took ENEM 2013, a high-stakes national test designed to select 170,000 places in tuition-free public universities. The assessment uses 180 multiple-choice items embedded in contextual information, relying on students to apply information in the item rather than on extensive prior knowledge. Two instruments based on real ENEM biology questions compared full-length context with abridged context while keeping identical stems and options. Results show no overall disadvantage for reduced context, with three items yielding significantly higher performance, suggesting reading and time-management matter more than prior knowledge, and any access democratization may be driven by test novelty.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages8

Reading Time20 min

AccessFree to download

Introduction

Background of ENEM and assessment design

Competencies, abilities, and item construction

Contextualization and prior research

IN-CONTEXT ITEMS INA NATION WIDE EXAMINATION: WHICH KNOWLEDGE AND SKILLS ARE ACTUALLY ASSESSED? Nelio Bizzo1, Ana Maria Santos Gouw1, Paulo Sérgio Garcia1, Paulo Henrique Nico Monteiro1 and Luiz Caldeira Brant de Tolentino Neto2 1 Faculty of Education and EDEVO Research Nucleus, São Paulo University, Brazil 2 Santa Maria Federal University, Brazil Abstract: Over seven million Brazilian youngsters enrolled for a National Test (ENEM 2013) aimed at those who are willing to get one of the 170,000 places available in public free universities (Jan 2014) . ENEM submits 180 multiple choice questions in a context with supposedly relevant information which should be applied, relying on few or no previous knowledge. It was originally presented as a new possibility for poor students finding a path leading to good quality universities. We prepared two instruments based on real ENEM questions focusing on the same subject matter (biology), which were presented to two randomized groups of high school students. One group received full length questions (n=1,631), and the other received the same questions (n=1,400) with abridged context information, but with the same stem and options. Performance analysis not only showed no statistical significance towards students who were answering full length questions, but also showed that students’ performance was significantly higher in three questions with abridged information. Conclusions show that students’ performance may rely more heavily on reading and time management skills rather than on previous knowledge or mental skills. Democratization on university access, if any, may be due to the novelty of the test. Keywords: students’ assessment; intelligence assessment, ENEM; time management skills; reading skills. INTRODUCTION The Brazilian Ministry of Education (MEC) organizes a National Test for students at the end of High School (“ENEM”) since the year of 1998. In 2009 major changes were introduced, which attracted a great number of students, not only those who are actually at the last year of High School. All people who aspire to a university degree seem to have been encouraged to pursue such testing, given the reward introduced for a good score, in the form of a place in a free public university. Students have been challenged to achieve the highest possible mark, which would enable them to apply for a place in a computerized system (“SISU”) provided by MEC, which compares ENEM scores of students and assigns seats in public universities all over the country. In the year 2013 over seven million students were enrolled in ENEM, competing for places in free public universities. In addition to SISU, students can compete for over 170,000 scholarships in private universities (“PROUNI”), which can be as high as 100% of the tuition fees, given some conditions related to students’ socioeconomic status. According to official MEC information, about 110,000 students enrolled in the first version of the then national test in 1998, and no one could believe that seven million people would be enrolled the same test fifteen years later (2013 exam), competing for about 170,000 places in public universities throughout Brazil (January 2014) . ENEM is known for avoiding traditional questions, which rely heavily on the recollection offactual knowledge. Since it was launched, it was presented as a new strategy to assess directly students’competencies, defined by an official document as “structural modalities of intelligence”(Franco and Bonamino, 1999:29). The new test was warmly welcomed by the Brazilian press and broadly marketed in “grey literature”, which is difficult to quote. Apparently it was taken as a strategy not only for a new assessment-based educational reform, but also for social reform, as it would help poor students to pursue a path to higher education and, in addition, was aimed explicitly at reaching the job market. MEC presented the test as an opportunity for youngsters to plan their

FAQs

What is ENEM 2013 and how is it structured?

ENEM 2013 is a Brazilian national test taken at the end of high school, using 180 multiple-choice items plus a compulsory written composition. Items provide visual and written context, followed by a stem and five options.

How did the study test the role of context length?

The study built two instruments from real ENEM biology questions for two randomized student groups. One group received full-length questions, while the other received the same stems and options but with abridged context information.

What did the performance comparison suggest about what students rely on?

Performance showed no statistical disadvantage for students answering abridged-context items, and was higher on three specific items. The conclusions indicate students’ performance may depend more on reading and time-management skills than on prior knowledge.

Sep 6, 2026

Upload by Jacob