Differences in Written Discourse in Independent and Integrated Prototype Tasks for Next Generation TOEFL

Sep 30, 2026

Assessments compare discourse produced for prototype integrated writing tasks and for independent TOEFL Essay prompts in a field test. Using 216 compositions by 36 examinees across three TOEFL Essay score levels, researchers code lexical and syntactic complexity, grammatical accuracy, argument structure, orientations to evidence, and verbatim use of source text. Non-parametric MANOVAs reveal significant differences across task types, including lexical complexity, syntactic complexity, rhetoric, and pragmatics, with additional level-specific effects on accuracy, claims, and certain voice and summary behaviors.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages52

Reading Time2 h 11 min

AccessFree to download

Abstract

Study focus and task types

Methods and scoring variables

Key findings across task types and proficiency levels

Assessment implications for writing tasks

Differences in Written Discourse in Independent and Integrated Prototype Tasks for Next Generation TOEFL Alister Cumming, Ontario Institute for Studies in Education of the University of Toronto Robert Kantor, Educational Testing Service Kyoko Baba, Ontario Institute for Studies in Education of the University of Toronto Usman Erdosy, Carleton University Keanre Eouanzoui, Niagara University Mark James, University of Puerto Rico Version of January 31, 2005 Contact for authors: Alister Cumming [acumming@oise.utoronto.ca](acumming@oise.utoronto.ca) Modern Language Centre Ontario Institute for Studies in Education University of Toronto 252 Bloor Street West Toronto, Ontario Canada M5S 1V6 Acknowledgements This research was funded by the TOEFL research program at Educational Testing Service. The opinions expressed are those of the authors rather than the policies of the TOEFL or of Educational Testing Service. An extended version of this article will appear as a TOEFL Research Report. We thank Dan Eignor for helpful suggestions on our reporting of statistics as well as Don Banh for designing the computer software program for analyses of verbatim strings of words from source texts, type-token ratios of lexical words, and word length. Differences in Written Discourse in Independent and Integrated Prototype Tasks for Next Generation TOEFL Abstract We assessed whether and how the discourse written for prototype integrated tasks (involving writing in response to print or audio source texts) field tested for Next Generation TOEFL® differs from the discourse written for independent essays (i.e., the TOEFL Essay®) . We selected 216 compositions written for 6 tasks by 36 examinees in a field test—representing score levels 3, 4, and 5 on the TOEFL Essay—then coded the texts for lexical and syntactic complexity, grammatical accuracy, argument structure, orientations to evidence, and verbatim uses of source text. Analyses with non-parametric MANOVAs followed a 3 (task type: TOEFL Essay, writing in response to a reading passage, writing in response to a listening passage) by 3 (English proficiency level: score levels 3, 4, and 5 on the TOEFL Essay) within-subjects factorial design. The discourse produced for the integrated writing tasks differed significantly from the discourse produced in the independent essay for the variables of: lexical complexity (text length, word length, ratio of different words to total words written), syntactic complexity (number of words per T-unit, number clauses per T-unit), rhetoric (quality of propositions, claims, data, warrants, and oppositions in argument structure), and pragmatics (orientations to source evidence in respect to self or others and to phrasing the message as either declarations, paraphrases, or summaries) . Across the three English proficiency levels, significant differences appeared for the variables of grammatical accuracy as well as all indicators of lexical complexity (text length, word length, ratio of different words to total words written), one indicator of syntactic complexity (words per T-unit), one rhetorical aspect (quality of claims in argument structure), and two pragmatic aspects (expression of self as voice, messages phrased as summaries) . A fundamental issue in the assessment of writing abilities through multiple tasks is establishing that the writing elicited in a test is, on the one hand, substantially different from one task type to another, yet on the other hand, is also consistent across writing tasks. This issue came to the fore in the present study, which we conducted as research preliminary to the design of new writing tasks for the TOEFL® (Test of English as a Foreign Language) test, known as the Next Generation TOEFL® and TOEFL ®/iBT (internet-based test) . In particular, we focused on task types that involve examinees writing to integrate information from reading and listening stimulus materials (i.e., integrated tasks) in addi

FAQs

What did the study compare in the TOEFL writing tasks?

The study compared discourse written for prototype integrated tasks (writing based on reading or listening sources) with discourse written for independent essay tasks (TOEFL Essay).

How were the compositions selected and analyzed?

Researchers selected 216 compositions written for six tasks by 36 examinees from a field test, then coded them for lexical and syntactic complexity, grammatical accuracy, argument structure, evidence orientations, and verbatim source use. Analyses used non-parametric MANOVAs with a task type by proficiency-level design.

Which discourse aspects differed most between integrated and independent tasks?

Differences appeared for lexical complexity, syntactic complexity, rhetoric (quality of propositions/claims/warrants/oppositions), and pragmatics (orientations to source evidence and message phrasing choices such as declarations, paraphrases, or summaries).

Sep 30, 2026

Upload by SundayMiles