The Short-Term and Long-Term Effects of AWE Feedback on ESL Students’ Development of Grammatical Accuracy

Oct 15, 2026

Uploaded by Fans

This classroom-based mixed-methods study investigates both short-term and long-term effects of Criterion feedback on ESL learners’ grammatical accuracy. Multilevel growth modeling shows that Criterion feedback reduces errors in eight of nine categories from first drafts to final drafts for intermediate-high and advanced-low groups. Long-term results show only one statistically significant improvement for Run-on Sentence across subsequent papers. Interviews further clarify learners’ perceptions, informing implications for more effective use of AWE tools in ESL writing instruction.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages21

Reading Time53 min

AccessFree to download

Abstract

Corrective Feedback and the Role of Technology

Literature Review

Accuracy and Effectiveness of AWE Feedback

Students’ Perceptions of AWE Feedback

calico journal (online) issn 2056–9017 Article The Short-Term and Long-Term Effects of AWE Feedback on ESL Students’ Development of Grammatical Accuracy Zhi Li1, Hui-Hsien Feng2 and Aysel Saricaoglu3 Abstract This classroom-based study employs a mixed-methods approach to exploring both short-term and long-term effects of Criterion feedback on ESL students’ development of grammatical accuracy. The results of multilevel growth modeling indicate that Criterion feedback helps students in both intermediate-high and advanced-low levels reduce errors in eight out of nine categories from first drafts to final drafts within the same papers (short-term effects). However, there is only one error reduction of statistical significance in the category of Run-on Sentence from the first drafts of the first paper to the first drafts in the subsequent papers for both levels of students (long-term effects). The findings from interviews with the participants reveal students’ perceptions of Criterion feedback and help us understand the feedback effect. Implications for a more effective use of AWE tools in ESL classrooms are discussed. Keywords: automated writing evaluation, corrective feedback, short-term effects, long-term effects, grammatical accuracy Corrective feedback is an important source of input for language learners as it creates teachable moments and actionable learning opportunities (Ferris, 2006; Hyland & Hyland, 2006) . While responses to student writing have drawn the attention of both teachers and researchers since the 1980s, research Affiliations 1Paragon Testing Enterprises, Inc.. [email: zlisu2010@gmail.com](email: zlisu2010@gmail.com) [2](2Iowa State University)[Iowa State University](2Iowa State University). email: [hhfeng@iastate.edu](hhfeng@iastate.edu) (corresponding author) 3TED University, Turkey. [email: aysel.saricaoglu@tedu.edu.tr](email: aysel.saricaoglu@tedu.edu.tr) calico journal vol 34.3 2017 355–375 [https://doi.org/10.1558/cj.26382](https://doi.org/10.1558/cj.26382) ©2017, equinox publishing 356 Short-Term and Long-Term Effects of AWE Feedback into corrective feedback in second-language (L2) writing still raises debatable questions. One such question is concerned with what role technology can play in delivering feedback and assisting English writing. The past decade has witnessed a noticeable transformation of automated scoring tools into feedback-generating systems for pedagogical purposes. For example, e-rater from the Educational Testing Service (ETS) is incorporated ina web-based instructional automated writing evaluation (AWE) system called Criterion, and IntelliMetric from Vantage Learning supports another instructional AWE system called MY Access! A comprehensive review of AWE software can be found in Elliot et al. (2013) . AWE tools present more opportunities for feedback provision in writing classrooms, and the use of AWE tools in instructional settings is increasing rapidly (Ware & Hellmich, 2014). Meanwhile, research on AWE tools raises some concerns, such as the practices of AWE tools in classrooms (Link, Dursun, Karakaya, & Hegelheimer, 2014), the timing and quality of AWE feedback (Ferris, 2012), and, more importantly, the effectiveness of automated feedback on students’ writing development (Dikli & Bleyle, 2014; Wang, 2013) . Our research questions deal with the last concern. Literature Review Accuracy and Effectiveness of AWE Feedback Research on AWE feedback has mainly focused on the accuracy and effectiveness of AWE feedback to help learners improve their writing. One of the common findings of such studies is that AWE feedback is not as accurate as human feedback. For example, Otoshi (2005) compared the sentence-level feedback from Criterion and two English-language instructors on 28 essays written by Japanese adults and found that human instructors detected more errors than Criterion. In a comprehensive analysis of My Access! and Criterion feedback

FAQs

What does the study examine about AWE feedback for ESL students?

It examines both short-term and long-term effects of Criterion feedback on students’ development of grammatical accuracy.

What do the short-term results show?

Multilevel growth modeling indicates students reduce errors in eight of nine grammatical categories from first drafts to final drafts within the same papers.

How do the long-term effects compare with short-term effects?

Long-term effects show only one statistically significant error reduction in the Run-on Sentence category across subsequent papers for both student levels.

Oct 15, 2026

Uploaded by Fans