Home > Research > Publications & Outputs > Text authenticity in listening assessment

Electronic data

  • 2021_LAQ_RossiBrunfaut_ListeningAuthenticity_AuthorAcceptedManuscript

    Rights statement: This is an Accepted Manuscript of an article published by Taylor & Francis in Language Assessment Quarterly on 08/03/2021, available online: https://www.tandfonline.com/doi/abs/10.1080/15434303.2021.1895162

    Accepted author manuscript, 380 KB, PDF document

    Embargo ends: 8/09/22

    Available under license: CC BY-NC: Creative Commons Attribution-NonCommercial 4.0 International License

Links

Text available via DOI:

View graph of relations

Text authenticity in listening assessment: Can item writers be trained to produce authentic-sounding texts?

Research output: Contribution to Journal/MagazineJournal articlepeer-review

Published

Standard

Text authenticity in listening assessment : Can item writers be trained to produce authentic-sounding texts? / Rossi, Olena; Brunfaut, Tineke.

In: Language Assessment Quarterly, Vol. 18, No. 4, 11.11.2021, p. 398-418.

Research output: Contribution to Journal/MagazineJournal articlepeer-review

Harvard

APA

Vancouver

Author

Bibtex

@article{7da48f90527c4e8f93893113cc915dba,
title = "Text authenticity in listening assessment: Can item writers be trained to produce authentic-sounding texts?",
abstract = "A long-standing debate in the testing of listening concerns the authenticity of the listening input. On the one hand, listening texts produced by item-writers often lack spoken language characteristics. On the other hand, real-life recordings are often too context-specific to stand alone, or not suitable for item generation. In this study, we explored the effectiveness of an existing item-writing training course to produce authentic-sounding listening texts within the constraints of test specifications. Twenty-five trainees took an online item-writing course including training on creating authentic-sounding listening texts. Prior to and after the course, they developed a listening task. The resulting listening texts were judged on authenticity by three professional item reviewers and analysed linguistically by the researchers. Additionally, we interviewed the trainees following each item-writing event and analysed their online discussions from during the course. Statistical comparison of the pre-and post-course authenticity scores revealed a positive effect of the training on item-writers{\textquoteright} ability to produce authentic-sounding listening texts, while the linguistic analysis demonstrated that the texts produced after the training contained more instances of spoken language. The interviews and discussions revealed that item-writers{\textquoteright} awareness of spoken language features and their text production techniques influenced their ability to develop authentic-sounding texts.",
keywords = "testing listening, authenticity, item writing, item writer training",
author = "Olena Rossi and Tineke Brunfaut",
note = "This is an Accepted Manuscript of an article published by Taylor & Francis in Language Assessment Quarterly on 08/03/2021, available online: https://www.tandfonline.com/doi/abs/10.1080/15434303.2021.1895162",
year = "2021",
month = nov,
day = "11",
doi = "10.1080/15434303.2021.1895162",
language = "English",
volume = "18",
pages = "398--418",
journal = "Language Assessment Quarterly",
issn = "1543-4303",
publisher = "Routledge",
number = "4",

}

RIS

TY - JOUR

T1 - Text authenticity in listening assessment

T2 - Can item writers be trained to produce authentic-sounding texts?

AU - Rossi, Olena

AU - Brunfaut, Tineke

N1 - This is an Accepted Manuscript of an article published by Taylor & Francis in Language Assessment Quarterly on 08/03/2021, available online: https://www.tandfonline.com/doi/abs/10.1080/15434303.2021.1895162

PY - 2021/11/11

Y1 - 2021/11/11

N2 - A long-standing debate in the testing of listening concerns the authenticity of the listening input. On the one hand, listening texts produced by item-writers often lack spoken language characteristics. On the other hand, real-life recordings are often too context-specific to stand alone, or not suitable for item generation. In this study, we explored the effectiveness of an existing item-writing training course to produce authentic-sounding listening texts within the constraints of test specifications. Twenty-five trainees took an online item-writing course including training on creating authentic-sounding listening texts. Prior to and after the course, they developed a listening task. The resulting listening texts were judged on authenticity by three professional item reviewers and analysed linguistically by the researchers. Additionally, we interviewed the trainees following each item-writing event and analysed their online discussions from during the course. Statistical comparison of the pre-and post-course authenticity scores revealed a positive effect of the training on item-writers’ ability to produce authentic-sounding listening texts, while the linguistic analysis demonstrated that the texts produced after the training contained more instances of spoken language. The interviews and discussions revealed that item-writers’ awareness of spoken language features and their text production techniques influenced their ability to develop authentic-sounding texts.

AB - A long-standing debate in the testing of listening concerns the authenticity of the listening input. On the one hand, listening texts produced by item-writers often lack spoken language characteristics. On the other hand, real-life recordings are often too context-specific to stand alone, or not suitable for item generation. In this study, we explored the effectiveness of an existing item-writing training course to produce authentic-sounding listening texts within the constraints of test specifications. Twenty-five trainees took an online item-writing course including training on creating authentic-sounding listening texts. Prior to and after the course, they developed a listening task. The resulting listening texts were judged on authenticity by three professional item reviewers and analysed linguistically by the researchers. Additionally, we interviewed the trainees following each item-writing event and analysed their online discussions from during the course. Statistical comparison of the pre-and post-course authenticity scores revealed a positive effect of the training on item-writers’ ability to produce authentic-sounding listening texts, while the linguistic analysis demonstrated that the texts produced after the training contained more instances of spoken language. The interviews and discussions revealed that item-writers’ awareness of spoken language features and their text production techniques influenced their ability to develop authentic-sounding texts.

KW - testing listening

KW - authenticity

KW - item writing

KW - item writer training

U2 - 10.1080/15434303.2021.1895162

DO - 10.1080/15434303.2021.1895162

M3 - Journal article

VL - 18

SP - 398

EP - 418

JO - Language Assessment Quarterly

JF - Language Assessment Quarterly

SN - 1543-4303

IS - 4

ER -