Dinamika Ilmu - Vol. 17 No. 1 - An Evaluation Paradox - The Issues ofTest Validity in the Realm of Writing Test as the Final School Examination in the Indonesian Senior High School Milieu

Oct 15, 2026

Uploaded by dhado

Although Indonesia’s upper secondary curriculum includes four English language skills, national examinations formally test only reading and listening, creating unequal emphasis. Writing, though crucial for judging student achievement after three years of study, faces low test validity in existing writing assessments. A further study evaluates validity through theory-based, context, scoring, criterion-related, and consequential validity, using prior interviews and writing-test analysis to identify validity problems.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages21

Reading Time53 min

AccessFree to download

Abstract

1. Introduction

Educational assessment and national curriculum

Validity problems in national examinations

Ecological validity and writing assessment

DINAMIKA ILMU Vol. 17 No. 1, 2017 P-ISSN: 1411-3031; E-ISSN: 2442-9651 doi: [http://dx.doi.org/10.21093/di.v17i1.585](http://dx.doi.org/10.21093/di.v17i1.585) An Evaluation Paradox: The Issues ofTest Validity in the Realm of Writing Test as the Final School Examination in the Indonesian Senior High School Milieu1 David Imamyartha Universitas Negeri Jember, Indonesia e-mail: [imamyarthadavid@gmail.com](imamyarthadavid@gmail.com) Gunadi Harry Sulistyo Universitas Negeri Malang, Indonesia [e-mail: gunadi.hs@um.ac.id](e-mail: gunadi.hs@um.ac.id) Abstract Even though there are four English language skills in the Indonesia’s national curriculum at upper secondary schools, each of these skills is given an unequal emphasis since only reading and listening skills are formally tested in the national examination. Although writing competence possesses a particular stake as the determinant of students’ achievement after students undergo a three-year education at the upper secondary school level, it appears that the existing writing tests are low in terms of test validity, as demonstrated by a preliminary study. A further study is carried out to probe the issues of test validity by deploying the framework of test validity, which pertains to theory-based validity, context validity, scoring validity, criterion-related validity, and consequential validity in the scrutiny of the existing writing tests. It is revealed that the current writing tests are fraught with validity problems in all of these facets of test validity. This is corroborated by interview data in the preliminary study and the analysis of the existing writing tests. These particular issues obviously evoke an ambivalence between the exalted educational objectives in the national curricula and the edifice of English assessment. Based on the findings, several implications and directions rise for future praxis of writing assessment. Keywords: achievement, test validity, writing test, ambivalence 1 Several key ideas in this paper were presented at the 3rd Annual Conference of ALAA (Asian Association for Language Assessment), May 19-21, 2016, Sanur, Bali, Indonesia. Dinamika Ilmu, Volume 17 (1), 2017 1 A. Introduction As the endeavour to excel the life of a country, education and assessment are deemed to be a legitimate and fundamentally crucial arsenal in the improvement enterprise. In Indonesia, the government has decreed numerous provisos regarding the exigency to escalate the quality of education, one of which is the Republic of Indonesia’s Government’s Proviso no. 19, Chapter 2, Verse 2 pertaining to the National Standard of Education (Standar Nasional Pendidikan – henceforth SNP) . It is stated in the decree that in order to vouch and control the quality of education in accord with SNP, it is formally vital to conduct evaluation, accreditation, and certification (Government’s proviso number 19, 2005) . Another vital point stipulates that, on a yearly basis, every unit of education is to guarantee their education quality devoted to meeting or transcending SNP systematically by setting an exact and reasonable learning target and scheme to achieve. Despite the official decrees stipulating the need of assessment as a form of quality control, there is hardly any nationally standardized test measuring students’competence in English, in its entirety. National Examination (henceforth UN), the one devised by the Department of Education and Culture of Indonesia, entails only listening and reading (BNSP, 2013:24), which soundly delineates a mismatch between the dictated competences and the means through which they are assessed, analyzed, and interpreted (Sulistyo (2009), versing the omnipresent issues on UN, unearthed that many teachers consider the test developed by the government has better objectivity. However, it is possibly an explanation for another genuine response admitting the teachers’ inability to construct good tests and the avoidance of unfa

FAQs

Why does the paper discuss an “evaluation paradox” in Indonesian English assessment?

It highlights a mismatch between curriculum goals that emphasize broader language competences and national examinations that assess only reading and listening, leaving writing under-tested despite its educational importance.

How is test validity evaluated in the study?

The study uses a framework of test validity covering theory-based validity, context validity, scoring validity, criterion-related validity, and consequential validity to scrutinize existing writing tests.

What evidence supports the findings about validity issues?

The findings are corroborated by interview data from a preliminary study and analysis of the existing writing tests, showing validity problems across multiple validity facets.

Oct 15, 2026

Uploaded by dhado