Home > Research > Publications & Outputs > Standard setting anchor statements

Research

Links

Text available via DOI:

View graph of relations

Standard setting anchor statements: a double cross-over trial of two different methods

Research output: Contribution to Journal/Magazine › Journal article › peer-review

Published

Standard

Standard setting anchor statements: a double cross-over trial of two different methods. / Burr, Steven; Martin, Theresa; Edwards, James et al.
In: MedEDPublish, Vol. 10, 32, 03.02.2021.

Research output: Contribution to Journal/Magazine › Journal article › peer-review

Harvard

Burr, S, Martin, T, Edwards, J, Ferguson, C, Gilbert, K, Gray, C, Hill, A, Hosking, J, Jonhstone, K, Kisielewska, J, Milsom, C, Moyes, S, Rigby-Jones, A, Robinson, IM, Toms, N, Watson, H & Zahra, D 2021, 'Standard setting anchor statements: a double cross-over trial of two different methods', MedEDPublish, vol. 10, 32. https://doi.org/10.15694/mep.2021.000032.1

APA

Burr, S., Martin, T., Edwards, J., Ferguson, C., Gilbert, K., Gray, C., Hill, A., Hosking, J., Jonhstone, K., Kisielewska, J., Milsom, C., Moyes, S., Rigby-Jones, A., Robinson, I. M., Toms, N., Watson, H., & Zahra, D. (2021). Standard setting anchor statements: a double cross-over trial of two different methods. MedEDPublish, 10, Article 32. https://doi.org/10.15694/mep.2021.000032.1

Vancouver

Burr S, Martin T, Edwards J, Ferguson C, Gilbert K, Gray C et al. Standard setting anchor statements: a double cross-over trial of two different methods. MedEDPublish. 2021 Feb 3;10:32. doi: 10.15694/mep.2021.000032.1

Author

Burr, Steven ; Martin, Theresa ; Edwards, James et al. / Standard setting anchor statements : a double cross-over trial of two different methods. In: MedEDPublish. 2021 ; Vol. 10.

Bibtex

@article{0ffbaa374f4141ce95b42456cd3e7887,
title = "Standard setting anchor statements: a double cross-over trial of two different methods",
abstract = "Context: We challenge the philosophical acceptability of the Angoff method, and propose an alternative method of standard setting based on how important it is for candidates to know the material each test item assesses, and not how difficult it is for a subgroup of candidates to answer each item.Methods: The practicalities of an alternative method of standard setting are evaluated here, for the first time, with direct comparison to an Angoff method. To negate bias due to any leading effects, a prospective cross-over design was adopted involving two groups of judges (n=7 and n=8), both of which set the standards for the same two 100 item multiple choice question tests, by the two different methods.Results: Overall, we found that the two methods took a similar amount of time to complete. The alternative method produced a higher cut-score (by 12-14%), and had a higher degree of variability between judges' cut-scores (by 5%). When using the alternative method, judges reported a small, but statistically significant, increase in their confidence to decide accurately the standard (by 3%).Conclusion: This is a new approach to standard setting where the quantitative differences are slight, but there are clear qualitative advantages associated with use of the alternative method.",
author = "Steven Burr and Theresa Martin and James Edwards and Colin Ferguson and Kerry Gilbert and Christian Gray and Adele Hill and Joanne Hosking and Karen Jonhstone and Jolanta Kisielewska and Chloe Milsom and Siobhan Moyes and Ann Rigby-Jones and Robinson, {Iain M} and Nick Toms and Helen Watson and Daniel Zahra",
year = "2021",
month = feb,
day = "3",
doi = "10.15694/mep.2021.000032.1",
language = "English",
volume = "10",
journal = "MedEDPublish",
publisher = "Taylor and Francis",

}

RIS

TY - JOUR

T1 - Standard setting anchor statements

T2 - a double cross-over trial of two different methods

AU - Burr, Steven

AU - Martin, Theresa

AU - Edwards, James

AU - Ferguson, Colin

AU - Gilbert, Kerry

AU - Gray, Christian

AU - Hill, Adele

AU - Hosking, Joanne

AU - Jonhstone, Karen

AU - Kisielewska, Jolanta

AU - Milsom, Chloe

AU - Moyes, Siobhan

AU - Rigby-Jones, Ann

AU - Robinson, Iain M

AU - Toms, Nick

AU - Watson, Helen

AU - Zahra, Daniel

PY - 2021/2/3

Y1 - 2021/2/3

N2 - Context: We challenge the philosophical acceptability of the Angoff method, and propose an alternative method of standard setting based on how important it is for candidates to know the material each test item assesses, and not how difficult it is for a subgroup of candidates to answer each item.Methods: The practicalities of an alternative method of standard setting are evaluated here, for the first time, with direct comparison to an Angoff method. To negate bias due to any leading effects, a prospective cross-over design was adopted involving two groups of judges (n=7 and n=8), both of which set the standards for the same two 100 item multiple choice question tests, by the two different methods.Results: Overall, we found that the two methods took a similar amount of time to complete. The alternative method produced a higher cut-score (by 12-14%), and had a higher degree of variability between judges' cut-scores (by 5%). When using the alternative method, judges reported a small, but statistically significant, increase in their confidence to decide accurately the standard (by 3%).Conclusion: This is a new approach to standard setting where the quantitative differences are slight, but there are clear qualitative advantages associated with use of the alternative method.

AB - Context: We challenge the philosophical acceptability of the Angoff method, and propose an alternative method of standard setting based on how important it is for candidates to know the material each test item assesses, and not how difficult it is for a subgroup of candidates to answer each item.Methods: The practicalities of an alternative method of standard setting are evaluated here, for the first time, with direct comparison to an Angoff method. To negate bias due to any leading effects, a prospective cross-over design was adopted involving two groups of judges (n=7 and n=8), both of which set the standards for the same two 100 item multiple choice question tests, by the two different methods.Results: Overall, we found that the two methods took a similar amount of time to complete. The alternative method produced a higher cut-score (by 12-14%), and had a higher degree of variability between judges' cut-scores (by 5%). When using the alternative method, judges reported a small, but statistically significant, increase in their confidence to decide accurately the standard (by 3%).Conclusion: This is a new approach to standard setting where the quantitative differences are slight, but there are clear qualitative advantages associated with use of the alternative method.

U2 - 10.15694/mep.2021.000032.1

DO - 10.15694/mep.2021.000032.1

M3 - Journal article

VL - 10

JO - MedEDPublish

JF - MedEDPublish

M1 - 32

ER -