The Effect of Interlocutor and Assessment Mode Variables in Offshore Assessments of Speaking Skills in Occupational Settings

Sep 2, 2026

Upload by Jiven

The study investigates problems in offshore assessment of second-language speaking skills in work-related settings. It examines an approach used by the Australian Government in which native English interlocutors conduct recorded oral interactions with candidates, while trained raters evaluate audio centrally. Data from about 70 offshore health-professional candidates include ratings of performance, interlocutor competence, candidate–interlocutor rapport, and audibility. Statistical analysis shows that interlocutor variability and tape quality affect ratings.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages12

Reading Time30 min

AccessFree to download

Abstract

Introduction

Interlocutor and assessment mode variables in the Occupational English Test (OET)

ED 364066 AUTHOR TITLE PUB DATE NOTE PUB TYPE EDRS PRICE DESCRIPTORS IDENTIFIERS ABSTRACT DOCUMENT RESUYE MeNamara,T.F.;Lumley,Tom # The Effect of Interlocutor and Assessment ModeVariables in Offshore Assessments of Speaking Skillsin Occupational Settings. Aug 93 12p.;Paper presented at the Annual Language TestingResearch Colloquium(15th,Cambridge,England,andArnhem,The Netherlands,August 2-8,1993)。Reports -Research/Technical(143)--Speeches/Conference Papers(150) MF01/PC01 Plus Postage. *English for Special Purposes;Foreign Countries;Language Skil1s;*Language Tests;Statistical Analysis;Tape Recordings;*Testing Problems;*VerbalTests;*Vocational English(Second Language)Australia This study investigated potential problems associatedwit'one method of testing English-as-a-Second-Language(ESL)skillsin work settings in foreign countries.The method,used by theAustralian government to assess speech skills of individuals inlocations outside Australia,involves having native English-speakerscarry out a series of oral interactions with the examinee,that aretaped and later eve"uated by a small team of trained raters.Thestudy reported her nalyzed data from assessments of about 70examinees who were administered an advanced ESL test for healthprofessionals.Assessment of the recordings included multiple ratingsof examinee performance,competence.of the interlocutor,rapportestablished between examinees and interlocutor,and audibility of theinteraction.The data were subjected to statistical analysis.Resultssuggest that interlocutor variability and audiotape quality do affectratings,and that this method of analysis is usefu1 for examiningvariables in the testing situation.(Author/MSE) *********************************************************************** ED 364 066 THE EFFECT OF INTERLOCUTOR AND ASSESSMENT MODE VARIABLES INOFFSHORE ASSESSMENTS OF SPEAKING SKILLS IN OCCUPATIONALSETTINGS1 T.F.McNamara and Tom LumleyNLLIA Language Testing Centre,University of Melbourne Address for correspondence:NLLIA Language Testing Centre,Department of Applied Linguistics and Language Studies,The University of Melbourne,Parkville,Victoria 3052,AUSTRALIA Telephone: \#6133444207 Fax: \#6133445163 e-mail:tmac@unimelb.edu.au;tom_lumley@muwayf.unimelb.edu.au ## Abstract The increasing demand for performance assessment of speaking skills in second languages hasled to logistic complications,for example,the delivery of tests in offshore locations.Onesolution to the problem has been to train native speaker interlocutors to carry out a series of oralinteractions with the candidate,with assessment from audio recordings of the test sessionpostponed and conducted centrally by a small team of trained raters.This tech,ique is currentlyused in two large scale occupationally related ESP tests administered internationally on behalfof the Australian Government.But these procedures raise questions about the effect of suchfacets of the assessment situation as interlocutor variables and the quality of the audiotaperecording.Recent developments in multi-faceted Rasch measurement(Linacre,1989)havesignificantly broadened the possibilities for investigation of these issues. The research presented in this paper investigates potential problems associated with the aboveapproach to the offshore testing of speaking skills.Data from audiotape-based assessments ofapproximately 70 offshore candidates from two administrations of the Occupational EnglishTest,an advanced level ESP test for health professionals,are considered.In addition tomultiple ratings of candidate performance,each recording is rated for perceptions of thecompetence of the interlocutor,the rapport established between the candidate and theinterlocutor,and the audibility of the interaction.These aspects of the assessment situation aretreated as facets ina multi-faceted Rasch analysis of the data. The results of:he analysis reveal the effects of interlocutor variability and a

FAQs

What assessment method does the study examine for offshore speaking tests?

Native English-speaker interlocutors conduct oral interactions with candidates that are recorded, while trained raters later evaluate the recordings centrally.

What data and rating facets are included in the analysis?

Approximately 70 offshore candidates are analyzed with multiple ratings of candidate performance, plus ratings of interlocutor competence, rapport, and audibility.

Which factors are found to affect the rater ratings?

Interlocutor variability and audiotape quality significantly influence the ratings.

Sep 2, 2026

Upload by Jiven