Rater Reliability in Peer Speech Assessment done by CEFR A2 EFL Learners

Creators Takahashi Saiji

This paper aims to examine to what extent peer speaking assessment, conducted by CEFR A2 EFL learners, aligns with their instructor’s assessment. It also investigates whether there are differences in the assessment approach between two learner subgroups with varying English proficiency within the same CEFR band. The participants in this study were 21 EFL university students, most of whom had English proficiency at the CEFR A2 level. These students were enrolled in a 15-lesson ESP course that focused on presentation and paragraph writing. Each participant’s presentation was evaluated by both the author and the other participants based on seven criteria, encompassing linguistic and non-linguistic qualities. The obtained scores were then compared. The results indicated that peer assessment of linguistic qualities showed a weak correlation with the instructor’s assessment, whereas assessment of non-linguistic qualities exhibited a medium-tostrong correlation. These findings suggest that CEFR A2 EFL learners may find it challenging to evaluate their peers’ speech as accurately as their instructor. Additionally, the relative English proficiency within the same CEFR band group had a minimal impact on achieving a more precise rating in speaking.

[]