Validating an L2 Academic Group Oral Assessment: Insights From a Spoken Learner Corpus

This study determines the fine-grained bottom-up linguistic features involved in successful second language (L2) English academic group oral tutorial discussion through the use of a spoken learner corpus composed of more than 20hrs of L2 production. Student performances were graded by teacher-raters using a can-do rating scale, which assessed students' ability to participate in a group academic oral discussion. The performances were transcribed and annotated for linguistic features such as L2 errors, a range of interactive and interpersonal metadiscourse features, and a range of temporal, prosodic, lexical, and syntactic markers (or fluencemes) of (dis)fluency. The results of the corpus study suggest that frequent use of metadiscourse is the primary indicator of raters' positive evaluation of student performance in L2 academic tutorial discussion, alongside frequent use of discourse markers, filled pauses, and a high speech rate per minute as fluencemes. Most L2 error types and syntactic fluencemes did not particularly feature in raters' positive (or negative) evaluations. The detailed cross-sectional data afforded by this corpus analysis serve as quantitative evidence of the linguistic features accompanying each grade awarded across the rating scale and contributing to the construct validity of the assessment.