Home > Research > Publications & Outputs > Conduct of a meta review of programme evaluations

Electronic data

View graph of relations

Conduct of a meta review of programme evaluations: a case study of the SEARCH Program

Research output: ThesisDoctoral Thesis

Published

Standard

Conduct of a meta review of programme evaluations : a case study of the SEARCH Program. / Dickson, Rumona.

Lancaster : Lancaster University, 2012. 166 p.

Research output: ThesisDoctoral Thesis

Harvard

Dickson, R 2012, 'Conduct of a meta review of programme evaluations: a case study of the SEARCH Program', PhD, Lancaster University, Lancaster.

APA

Vancouver

Dickson R. Conduct of a meta review of programme evaluations: a case study of the SEARCH Program. Lancaster: Lancaster University, 2012. 166 p.

Author

Dickson, Rumona. / Conduct of a meta review of programme evaluations : a case study of the SEARCH Program. Lancaster : Lancaster University, 2012. 166 p.

Bibtex

@phdthesis{f17e9fe562f148c4baf66700479d166e,
title = "Conduct of a meta review of programme evaluations: a case study of the SEARCH Program",
abstract = "This thesis presents a retrospective case study that critically examines the evaluations that were undertaken as part of a continuing professional development (CPD) programme for health care professionals. The case is the SEARCH Program, an innovative CPD programme, which was designed to promote the implementation of evidence based practice (EBP) within the existing health care system in Alberta, Canada.Two approaches from the {\textquoteleft}using{\textquoteright} branch of Alkin and Chrisite{\textquoteright}s evaluation theory tree are used in this research. The first employs a quantitative metaevaluation tool to retrospectively assess the quality of evaluations that were conducted from 2000 to 2005. The second is qualitative and explores the use of evaluations to inform programme development.The results of the quantitative analysis demonstrate that the evaluations scored poorly. In fact all evaluations failed to meet basic pass/fail criteria in three of the four standard categories. Reasons for this are explored and include the interdependence of criteria in the metaevaluation tool, the poor or incomplete quality of the reports and the retrospective nature of the process that did not allow for additional data collection. The apparent precision offered by the metaevaluation tool is questionable, as there is a lack of explanation regarding the weighting of the various items, the quantitative formulae used, and the criteria for classifying an evaluation as a failure. The tool is also limited by its focus on evaluation process with no consideration given to the results of the programme evaluations.The application of qualitative method was also time consuming but more fruitful. The results of the qualitative analysis demonstrate that the SEARCH Program was a complex, innovative and evolving programme functioning in a complex and changing health care system. Evaluation processes used within the programme were developmental in nature and informed substantive programme changes. The extent of the changes extend beyond what would be expected with standard formative or summative evaluation and fit with the concepts and use of developmental evaluation as articulated by Patton.The development of CPD programmes for health care professionals who are required to implement EBP is complex and requires collaboration between networks of professionals from institutions within health and higher education. Such programmes need to be reflective, innovative and flexible in nature due to the complex environments in which they are established and the complex outcomes that they wish to implement. This complexity and need for consistent re-evaluation of the goals of the programmes means that developmental evaluation may be an appropriate approach.It is acknowledged that developmental evaluation is difficult and requires both expertise and commitment of those involved. It is also acknowledged that such evaluation may be able to demonstrate immediate outcomes of the CPD programme for the participants and even the faculty but is much less likely to be able to demonstrate impact on the health care system in which it is used.",
author = "Rumona Dickson",
year = "2012",
language = "English",
publisher = "Lancaster University",
school = "Lancaster University",

}

RIS

TY - THES

T1 - Conduct of a meta review of programme evaluations

T2 - a case study of the SEARCH Program

AU - Dickson, Rumona

PY - 2012

Y1 - 2012

N2 - This thesis presents a retrospective case study that critically examines the evaluations that were undertaken as part of a continuing professional development (CPD) programme for health care professionals. The case is the SEARCH Program, an innovative CPD programme, which was designed to promote the implementation of evidence based practice (EBP) within the existing health care system in Alberta, Canada.Two approaches from the ‘using’ branch of Alkin and Chrisite’s evaluation theory tree are used in this research. The first employs a quantitative metaevaluation tool to retrospectively assess the quality of evaluations that were conducted from 2000 to 2005. The second is qualitative and explores the use of evaluations to inform programme development.The results of the quantitative analysis demonstrate that the evaluations scored poorly. In fact all evaluations failed to meet basic pass/fail criteria in three of the four standard categories. Reasons for this are explored and include the interdependence of criteria in the metaevaluation tool, the poor or incomplete quality of the reports and the retrospective nature of the process that did not allow for additional data collection. The apparent precision offered by the metaevaluation tool is questionable, as there is a lack of explanation regarding the weighting of the various items, the quantitative formulae used, and the criteria for classifying an evaluation as a failure. The tool is also limited by its focus on evaluation process with no consideration given to the results of the programme evaluations.The application of qualitative method was also time consuming but more fruitful. The results of the qualitative analysis demonstrate that the SEARCH Program was a complex, innovative and evolving programme functioning in a complex and changing health care system. Evaluation processes used within the programme were developmental in nature and informed substantive programme changes. The extent of the changes extend beyond what would be expected with standard formative or summative evaluation and fit with the concepts and use of developmental evaluation as articulated by Patton.The development of CPD programmes for health care professionals who are required to implement EBP is complex and requires collaboration between networks of professionals from institutions within health and higher education. Such programmes need to be reflective, innovative and flexible in nature due to the complex environments in which they are established and the complex outcomes that they wish to implement. This complexity and need for consistent re-evaluation of the goals of the programmes means that developmental evaluation may be an appropriate approach.It is acknowledged that developmental evaluation is difficult and requires both expertise and commitment of those involved. It is also acknowledged that such evaluation may be able to demonstrate immediate outcomes of the CPD programme for the participants and even the faculty but is much less likely to be able to demonstrate impact on the health care system in which it is used.

AB - This thesis presents a retrospective case study that critically examines the evaluations that were undertaken as part of a continuing professional development (CPD) programme for health care professionals. The case is the SEARCH Program, an innovative CPD programme, which was designed to promote the implementation of evidence based practice (EBP) within the existing health care system in Alberta, Canada.Two approaches from the ‘using’ branch of Alkin and Chrisite’s evaluation theory tree are used in this research. The first employs a quantitative metaevaluation tool to retrospectively assess the quality of evaluations that were conducted from 2000 to 2005. The second is qualitative and explores the use of evaluations to inform programme development.The results of the quantitative analysis demonstrate that the evaluations scored poorly. In fact all evaluations failed to meet basic pass/fail criteria in three of the four standard categories. Reasons for this are explored and include the interdependence of criteria in the metaevaluation tool, the poor or incomplete quality of the reports and the retrospective nature of the process that did not allow for additional data collection. The apparent precision offered by the metaevaluation tool is questionable, as there is a lack of explanation regarding the weighting of the various items, the quantitative formulae used, and the criteria for classifying an evaluation as a failure. The tool is also limited by its focus on evaluation process with no consideration given to the results of the programme evaluations.The application of qualitative method was also time consuming but more fruitful. The results of the qualitative analysis demonstrate that the SEARCH Program was a complex, innovative and evolving programme functioning in a complex and changing health care system. Evaluation processes used within the programme were developmental in nature and informed substantive programme changes. The extent of the changes extend beyond what would be expected with standard formative or summative evaluation and fit with the concepts and use of developmental evaluation as articulated by Patton.The development of CPD programmes for health care professionals who are required to implement EBP is complex and requires collaboration between networks of professionals from institutions within health and higher education. Such programmes need to be reflective, innovative and flexible in nature due to the complex environments in which they are established and the complex outcomes that they wish to implement. This complexity and need for consistent re-evaluation of the goals of the programmes means that developmental evaluation may be an appropriate approach.It is acknowledged that developmental evaluation is difficult and requires both expertise and commitment of those involved. It is also acknowledged that such evaluation may be able to demonstrate immediate outcomes of the CPD programme for the participants and even the faculty but is much less likely to be able to demonstrate impact on the health care system in which it is used.

M3 - Doctoral Thesis

PB - Lancaster University

CY - Lancaster

ER -