automated essay scoring and personalized feedback generation in EFL writing education

Abstract

Automated essay scoring (AES) is a useful tool in writing education, offering real-time essay scores for students and instructors. However, previous AES models do not provide specific rubric-based scores and actionable feedback for essay improvement, which are crucial for learning. Addressing this gap, we present FABRIC, a pipeline designed to enhance English writing classes by automatically generating 1) the overall scores, 2) detailed rubric-based scores, and 3) constructive feedback for essay improvement. The first component of the FABRIC is DREsS, a real-world Dataset for Rubric-based Essay Scoring (DREsS). DREsS includes EFL students’ written essays and scores annotated by instructors under three primary rubrics: content, organization, and language. The second component is CASE, a Corruption- based Augmentation Strategy for Essays, which improves the performance of the baseline model by 45.44%. The third element is EssayCoT, the Essay Chain-of-Thought prompting strategy which uses scores predicted from the AES model to generate more preferable feedback from instructors. We conduct a comprehensive evaluation of EssayCoT against standard prompting, involving 21 English education experts. The feedback generated by EssayCoT is 5.6 times more preferred for its quality and type of feedback. Lastly, we implement FABRIC in college English writing classes and evaluate its performance and students’ learning effect.