This study investigated the effect of automated writing evaluation (AWE) feedback on Korean university students’ revision behavior. Specifically, it explored the extent to which the quality of essays differed between first drafts and second drafts. Furthermore, the way individual students dealt with incorrect feedback in revising essays was examined. In this study, ETS’ Criterion program, one of the most popular AWE programs, was employed. In cases where incorrect feedback was detected, the first and second drafts were closely compared to find out how students altered the text in response to the incorrect feedback, and this was classified into one of three categories: successful changes, unsuccessful changes, and text deletion. The results showed that students’ second drafts were rated significantly higher than first drafts by two NS raters, which indicates that many students followed the revision suggestions made by the Criterion program. As students’ Criterion scores increased, their ability to identify and make successful changes in response to incorrect feedback improved. The findings of the study can advance our understanding of AWE use in an EFL context and should contribute to broader examination of how Korean university students engage in revision of their essays.

This is a longitudinal case study using a mixed-methods research design to track how two Korean university students improved their English writing competence over one year with the aid of automated writing evaluation (AWE) program, Criterion. The participants wrote essays outside of class every month for one year, submitting first and later second drafts. The participants completed a TOEIC writing test at the beginning and end of the study; students’ reflections on their writing development, obtained through interviews and journal entries, were also examined. A comparison of scores, errors, and quantitative measures of fluency and grammatical complexity indicated writing improvement. Both participants used Criterion feedback effectively to render informed judgments and valid corrections. Essay revision based on Criterion feedback yielded more self-directed learning and greater comfort with writing in content courses. It is suggested that the effect of AWE feedback transfers to long-term improvement. The results point to the potential benefit of AWE use in individual out-of-class writing practices.