Home > Research > Publications & Outputs > Using a logic model to evaluate rater training ...

Links

Text available via DOI:

View graph of relations

Using a logic model to evaluate rater training for EAP writing assessment

Research output: Contribution to Journal/MagazineJournal articlepeer-review

Published
  • J. O'Connell
Close
Article number101160
<mark>Journal publication date</mark>30/11/2022
<mark>Journal</mark>Journal of English for Academic Purposes
Volume60
Number of pages13
Publication StatusPublished
Early online date12/09/22
<mark>Original language</mark>English

Abstract

Assessment by written exams and coursework is common practice in pre-sessional and preliminary year EAP programmes, but the allocation of marks for written assessment is complex, as is training raters to apply specified assessment standards. This practitioner research uses a Logic Model, a visual diagram commonly used in programme evaluation, to evaluate the rater training procedure for writing assessment in an English-medium university department. This study integrates data from surveys, interviews and workshops with the stakeholders involved in the rater training procedure to develop a Logic Model as part of an ongoing theory of change evaluation. The final product is a Model that reveals the guiding principles of rater training in the department, text that describes the evaluation process, and a measurement plan. This paper showcases how practitioner research can enhance EAP practice by demonstrating how an essential component of EAP assessment, rater training, and the rationale behind it, can be made cogent to the various stakeholders involved in the procedure. This paper offers considerations for EAP practitioners, managers, and testing staff when developing or working with rater training, bridging the gap between EAP and language testing and assessment communities.