Home > Research > Publications & Outputs > Societal impact evaluation

Links

Text available via DOI:

View graph of relations

Societal impact evaluation: exploring evaluator perceptions of the characterization of impact under the REF2014

Research output: Contribution to Journal/MagazineJournal articlepeer-review

Published

Standard

Societal impact evaluation : exploring evaluator perceptions of the characterization of impact under the REF2014. / Samuel, Gabrielle N.; Derrick, Gemma E.

In: Research Evaluation, Vol. 24, No. 3, 07.2015, p. 229-241.

Research output: Contribution to Journal/MagazineJournal articlepeer-review

Harvard

APA

Vancouver

Samuel GN, Derrick GE. Societal impact evaluation: exploring evaluator perceptions of the characterization of impact under the REF2014. Research Evaluation. 2015 Jul;24(3):229-241. Epub 2015 Apr 16. doi: 10.1093/reseval/rvv007

Author

Bibtex

@article{0c3bf417dfaa435c8e266317160fbefe,
title = "Societal impact evaluation: exploring evaluator perceptions of the characterization of impact under the REF2014",
abstract = "The relative newness of 'impact' as a criterion for research assessment has meant that there is yet to be an empirical study examining the process of its evaluation. This article is part of a broader study which is exploring the panel-based peer and end-user review process for societal impact evaluation using the UK's national research assessment exercise, the Research Excellence Framework (REF) 2014, as a case study. In particular, this article explores the different perceptions REF2014 evaluators had regarding societal impact, preceding their evaluation of this measure as part of REF2014. Data are drawn from 62 interviews with evaluators from the health-related Panel A and its subpanels, prior to the REF2014 exercise taking place. We show how going into the REF exercise, evaluators from Panel A had different perceptions about how to characterize impact and how to define impact realization in terms of research outcomes and the research process. We conclude by discussing the implications of our findings for future impact evaluation frameworks, as well as postulating a series of hypotheses about the ways in which evaluators' different perceptions going into an impact assessment could potentially influence the evaluation of impact submissions. Using REF2014 as a case study, these hypotheses will be tested in interviews with REF2014 evaluators post-assessment.",
keywords = "societal impact, Research Excellence Framework, qualitative research, research evaluation, HEALTH TECHNOLOGY-ASSESSMENT, PRODUCTIVE INTERACTIONS, PAYBACK FRAMEWORK, DECISION-MAKING, POLICY, PROGRAM, SCIENCE, BIAS, CHALLENGES, INNOVATION",
author = "Samuel, {Gabrielle N.} and Derrick, {Gemma E.}",
year = "2015",
month = jul,
doi = "10.1093/reseval/rvv007",
language = "English",
volume = "24",
pages = "229--241",
journal = "Research Evaluation",
issn = "0958-2029",
publisher = "Oxford University Press",
number = "3",

}

RIS

TY - JOUR

T1 - Societal impact evaluation

T2 - exploring evaluator perceptions of the characterization of impact under the REF2014

AU - Samuel, Gabrielle N.

AU - Derrick, Gemma E.

PY - 2015/7

Y1 - 2015/7

N2 - The relative newness of 'impact' as a criterion for research assessment has meant that there is yet to be an empirical study examining the process of its evaluation. This article is part of a broader study which is exploring the panel-based peer and end-user review process for societal impact evaluation using the UK's national research assessment exercise, the Research Excellence Framework (REF) 2014, as a case study. In particular, this article explores the different perceptions REF2014 evaluators had regarding societal impact, preceding their evaluation of this measure as part of REF2014. Data are drawn from 62 interviews with evaluators from the health-related Panel A and its subpanels, prior to the REF2014 exercise taking place. We show how going into the REF exercise, evaluators from Panel A had different perceptions about how to characterize impact and how to define impact realization in terms of research outcomes and the research process. We conclude by discussing the implications of our findings for future impact evaluation frameworks, as well as postulating a series of hypotheses about the ways in which evaluators' different perceptions going into an impact assessment could potentially influence the evaluation of impact submissions. Using REF2014 as a case study, these hypotheses will be tested in interviews with REF2014 evaluators post-assessment.

AB - The relative newness of 'impact' as a criterion for research assessment has meant that there is yet to be an empirical study examining the process of its evaluation. This article is part of a broader study which is exploring the panel-based peer and end-user review process for societal impact evaluation using the UK's national research assessment exercise, the Research Excellence Framework (REF) 2014, as a case study. In particular, this article explores the different perceptions REF2014 evaluators had regarding societal impact, preceding their evaluation of this measure as part of REF2014. Data are drawn from 62 interviews with evaluators from the health-related Panel A and its subpanels, prior to the REF2014 exercise taking place. We show how going into the REF exercise, evaluators from Panel A had different perceptions about how to characterize impact and how to define impact realization in terms of research outcomes and the research process. We conclude by discussing the implications of our findings for future impact evaluation frameworks, as well as postulating a series of hypotheses about the ways in which evaluators' different perceptions going into an impact assessment could potentially influence the evaluation of impact submissions. Using REF2014 as a case study, these hypotheses will be tested in interviews with REF2014 evaluators post-assessment.

KW - societal impact

KW - Research Excellence Framework

KW - qualitative research

KW - research evaluation

KW - HEALTH TECHNOLOGY-ASSESSMENT

KW - PRODUCTIVE INTERACTIONS

KW - PAYBACK FRAMEWORK

KW - DECISION-MAKING

KW - POLICY

KW - PROGRAM

KW - SCIENCE

KW - BIAS

KW - CHALLENGES

KW - INNOVATION

U2 - 10.1093/reseval/rvv007

DO - 10.1093/reseval/rvv007

M3 - Journal article

VL - 24

SP - 229

EP - 241

JO - Research Evaluation

JF - Research Evaluation

SN - 0958-2029

IS - 3

ER -