Fully automated speaking assessment - pronunciation and proficiency testing - changes, trade-offs, and machine scoring debates

Aug 29, 2026

Technological innovation has transformed speaking assessment by enabling recording, quantification, and scoring of oral performance beyond live test conditions, while also expanding user choice through fully automated delivery. This chapter examines machine scoring systems trained initially on human ratings but operating without a human component in scoring, emphasizing how pronunciation underpins performance evaluation. Key issues include accuracy and constraints, trade-offs between automation and fairness, prospects for hybrid machine-human scoring, and the possibility of prioritizing intelligibility over accent reduction.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages29

Reading Time1 h 13 min

AccessFree to download

Introduction

Historical context and conceptual foundations

Historical and current conceptualizations of speaking assessment

Indirect testing and its limitations

Fully automated assessment and its role in proficiency testing

Trade-offs, debates, and technological constraints

Pronunciation-centered machine scoring

Future directions: intelligibility prioritization and hybrid scoring

Fully automated speaking assessments: Changes to proficiency testing and the role of pronunciation Talia Isaacs UCL Institute of Education, University College London [Talia.isaacs@ucl.ac.uk](Talia.isaacs@ucl.ac.uk) Reference Isaacs, T. (in press). Fully automated speaking assessment: Changes to proficiency testing and the role of pronunciation. In O. Kang, R. I. Thomson, & J. Murphy (Eds.), The Routledge handbook of contemporary English pronunciation. Abingdon, UK: Routledge Abstract Technological innovation has revolutionized capability in assessing speech, making it possible to record, quantify, and score oral performances outside of live testing conditions. Technology has also increased test-user choice, with fully automated tests, which were initially trained on human ratings but devoid of a human component in test delivery and scoring, now available on the market. The goal of this chapter is to underscore the innovations, trade-offs, and debates in the machine scoring of speech, highlighting the central role of pronunciation. The consequences of machine-driven assessment and possibility of complementing machine scoring with human ratings will also be considered. Main text Introduction Technology has revolutionized capability in assessing speaking, making it possible to record, quantify, and score test-takers’ oral performances outside of live testing conditions. Technology has also opened the door to new possibilities in terms of the delivery and scoring of tests, resulting in greater choices for test-users in the second language (L2) speaking assessments available on the market in the 21st century. The focus of this chapter is on fully automated assessments ofL2 speech—that is, operational test delivery and scoring of speech production without any human intervention, achieved through both automatic speech recognition (ASR) technology, and an algorithmic score generator to optimize approximation to human ratings. As a preface to discussing how technology has transformed the L2 speaking assessment terrain and the centrality of pronunciation (particularly segmental features) to the state-of-the-art in machine scoring, it is important to situate automated assessment in its broader historical context. After outlining how age-old concerns about the lack of objectivity in human ratings of L2 speech have been addressed through the use of technology, the chapter will focus on operational machine scoring systems used for L2 speaking assessments, including feedback on segmental errors for low-stakes (e.g., research or pedagogical) purposes. The discussion will be underpinned by debates on trade-offs in using fully automated assessments, highlighting concerns about technological constraints dictating the nature of the assessment. Future directions will then be discussed, one of which is ways that technology can be used to prioritize the factors that are most important for intelligibility (i.e., being easily understandable to listeners), instead of simply promoting accent reduction (i.e., mastery of all L2 pronunciation features when some are more consequential for communication than others) . Finally, the chapter will conclude with the possibility of using hybrid machine-human scoring systems. Historical and current conceptualizations of the topic The L2 assessment literature has long distinguished between three modes of speaking assessment that are foundational to a discussion of the topic. A ‘direct test’ refers to a spoken assessment conducted face-to-face with an interlocutor (e.g., interviewer, examiner, testtaker), whereas a ‘semi-direct test’ is machine administered (e.g., computer-or phone-based), with the test-taker speaking into a recording device. Finally, an ‘indirect test’ denotes assessing speaking without eliciting any form of spoken production (Ginther, 2012) . An example of indirect test item is the following multiple-choice question: Which of the following words is unlike the others in ter

FAQs

What makes a speaking assessment “fully automated” in this chapter?

Fully automated assessment delivers and scores speech production without any human intervention, using automatic speech recognition (ASR) and an algorithmic score generator that approximates human ratings.

How does the chapter distinguish direct, semi-direct, and indirect speaking tests?

A direct test involves face-to-face assessment with an interlocutor, a semi-direct test is machine administered with the learner speaking into a recording device, and an indirect test assesses speaking without eliciting spoken production.

Why is pronunciation considered central to machine scoring?

The chapter frames pronunciation—especially segmental features—as a key factor in how machine scoring approximates performance evaluation and supports feedback, including for low-stakes contexts.

Aug 29, 2026

Uploaded by Nguyễn Văn Học