QUALITY OF MARKING OF KS3 ENGLISH - INTRODUCTION - STANDARDISATION AND FIRST SAMPLE MARKS - RESULTS

Oct 20, 2026

Analysis examines the quality and consistency of marking in Key Stage 3 (KS3) English following the introduction of component marking in 2004. Three investigations from 2003 and 2004 compare supervising-marker and marker differences using standardisation and first sample checks, individual pupil review outcomes, and estimates of group reviews producing level changes. The study tracks agreement rates and absolute mark discrepancy metrics across reading, Shakespeare reading, writing, and Shakespeare writing papers.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages8

Reading Time20 min

AccessFree to download

Introduction

Standardisation and First Sample Marks

Background

Method

Results

Comparison between years

QUALITY OF MARKING OF KS3 ENGLISH 1. INTRODUCTION Lucy Royal-Dawson In 2004, Key Stage 3 English was component marked for the first time. Reading markers marked the two reading papers and writing markers marked the two writing papers. Prior to this, all four papers were marked by the same marker. There was a perceived expectation that component marking might increase marking reliability. To judge whether this was the case, three investigations of marking in 2003 and 2004 were undertaken. First, standardisation and first sample marks from three Senior Markers’ teams, one from 2003 and two from 2004, were explored to compare the mark differences between the supervising marker and the markers. Second, requests for individual reviews of pupils’ marks were analysed with regard to the proportion of mark changes that resulted from reviews. Third, the number of group reviews that resulted in a level change this year compared with last year was estimated from a sample of scripts. 2. STANDARDISATION AND FIRST SAMPLE MARKS Background Markers are trained to mark through standardisation. Immediately after training, they send scripts they have marked, the standardisation sample, to their team leader who checks the accuracy of the marking. Marker acceptability is judged by reference to the absolute mark difference (AMD) between the team leader’s and marker’s mark allocated to a pupil. The AMDs are calculated at strand level1 for the two writing papers (writing and Shakespeare writing) for each pupil and at total mark level for each of the two reading papers (reading and Shakespeare reading) for each pupil. The pupil AMDs are summed across all pupils in the sample and marking acceptability is graded according to the size of the sample AMD. Once a marker is deemed acceptable, he or she can start marking. If a marker does not meet the acceptability criterion, the team leader requests further samples for checking. The quality of marking is monitored in the same way during the marking period by the team leaders through two further samples of scripts from each marker, the first early on in the marking period and the final at about three-quarters of the way through the allocation. Method The current study used the standardisation and first sample forms from three Senior Markers’teams, one from 2003 and two from 2004 representing reading and writing. The teams were selected because their paperwork was readily available at short notice. Only markers who had forms for both the standardisation and first samples were used to provide consistency between standardisation and first sample. The size and composition of the teams used in the study are given in Table 1 below. The paperwork for the teams in 2003 was not complete in that not every marker had forms for both reading and writing scripts. Table 1 Size and composition of the marking teams in the study | 2003 | | | 2004 | | | --- | --- | --- | --- | --- | | | Reading | Writing | Reading | Writing | | Number of team leaders | 8 | 8 | 6 | 6 | | Number of markers | 53 | 57 | 49 | 38 | 1 The writing strands are: sentence structure and punctuation (SSP), text structure and organisation (TSO) and composition and effect (C&E); the Shakespeare writing strands are: sentence structure, punctuation and text organisation (SSPTO), composition and effect (C&E) and spelling. To compare the quality of marking between the two years, the following measures were calculated from the mark differences between the team leaders and markers recorded for both the standardisation and first samples: 􀂃 The proportion of agreements in marks between a team leader and a marker. 􀂃 The proportion of agreements within one mark (that is, the same mark or one mark either side) between a team leader and a marker. 􀂃 The mean pupil AMD and its standard deviation across all markers for each of the four papers. The agreement rate provides an indication of the consistency of marking between team leaders and marke

FAQs

What prompted the investigations into KS3 English marking quality?

Component marking was introduced in 2004 for KS3 English, creating an expectation that it might improve marking reliability. Three investigations were conducted in 2003 and 2004 to evaluate whether this occurred.

How is marker acceptability assessed during standardisation?

After training, markers submit standardisation sample scripts. Acceptability is judged using the absolute mark difference (AMD) between the team leader’s and marker’s allocated marks, calculated at strand level for writing papers and at total mark level for reading papers.

What indicators were used to compare marking quality between 2003 and 2004?

Agreement rates between team leaders and markers, agreement within one mark, and the mean pupil AMD (with standard deviation) across all markers for the four papers were calculated and compared across years and between standardisation and first sample.

Oct 20, 2026

Uploaded by Alex Sinclair