Accepted author manuscript, 367 KB, PDF document
Available under license: CC BY: Creative Commons Attribution 4.0 International License
Final published version
Licence: CC BY: Creative Commons Attribution 4.0 International License
Research output: Contribution to Journal/Magazine › Review article › peer-review
Research output: Contribution to Journal/Magazine › Review article › peer-review
}
TY - JOUR
T1 - Natural experiments for the evaluation of place-based public health interventions
T2 - a methodology scoping review
AU - Albers, Patricia
AU - Rinaldi, Chiara
AU - Brown, Heather
AU - Mason, Kate
AU - d’Apice, Katrina
AU - McGill, Elizabeth
AU - McQuire, Cheryl
AU - Craig, Peter
AU - A Laverty, Anthony
AU - Beeson, Morgan
AU - Campbell, Mhairi
AU - Egan, Matt
AU - Gibson, Marcia`
AU - Fuller, Maxwell
AU - Dillon, Amy
AU - Taylor-Robinson, David
AU - Jago, Russell
AU - Tilling, Kate
AU - Barr, Benjamin
AU - Sniehotta, Falko
AU - Hickman, Matthew
AU - Millet, Christopher
AU - de Vocht, Frank
PY - 2023/6/22
Y1 - 2023/6/22
N2 - Place-based public health evaluations are increasingly making use of natural experiments. This scoping review aimed to provide an overview of the design and use of natural experiment evaluations (NEEs), and an assessment of the plausibility of the randomization assumption. A systematic search of three bibliographic databases (Pubmed, Web of Science and Ovid-Medline) was conducted in January 2020 to capture publications that reported a natural experiment of a place-based public health intervention or outcome. For each, study design elements were extracted. An additional evaluation of randomization was conducted by 12 of this paper's authors who evaluated the same set of 20 randomly selected studies and assessed ' ' randomization for each. 366 NEE studies of place-based public health interventions were identified. The most commonly used NEE approach was a Difference-in-Differences study design (25%), followed by before-after studies (23%) and regression analysis studies. 42% of NEEs had likely or probable randomization of exposure (the intervention), while for 25% this was implausible. An inter-rater agreement exercise indicated poor reliability of randomization assignment. Only about half of NEEs reported some form of sensitivity or falsification analysis to support inferences. NEEs are conducted using many different designs and statistical methods and encompass various definitions of a natural experiment, while it is questionable whether all evaluations reported as natural experiments should be considered as such. The likelihood of randomization should be specifically reported, and primary analyses should be supported by sensitivity analyses and/or falsification tests. Transparent reporting of NEE designs and evaluation methods will contribute to the optimum use of place-based NEEs. [Abstract copyright: Copyright © 2023 Albers, Rinaldi, Brown, Mason, d'Apice, McGill, McQuire, Craig, Laverty, Beeson, Campbell, Egan, Gibson, Fuller, Dillon, Taylor-Robinson, Jago, Tilling, Barr, Sniehotta, Hickman, Millett and de Vocht.]
AB - Place-based public health evaluations are increasingly making use of natural experiments. This scoping review aimed to provide an overview of the design and use of natural experiment evaluations (NEEs), and an assessment of the plausibility of the randomization assumption. A systematic search of three bibliographic databases (Pubmed, Web of Science and Ovid-Medline) was conducted in January 2020 to capture publications that reported a natural experiment of a place-based public health intervention or outcome. For each, study design elements were extracted. An additional evaluation of randomization was conducted by 12 of this paper's authors who evaluated the same set of 20 randomly selected studies and assessed ' ' randomization for each. 366 NEE studies of place-based public health interventions were identified. The most commonly used NEE approach was a Difference-in-Differences study design (25%), followed by before-after studies (23%) and regression analysis studies. 42% of NEEs had likely or probable randomization of exposure (the intervention), while for 25% this was implausible. An inter-rater agreement exercise indicated poor reliability of randomization assignment. Only about half of NEEs reported some form of sensitivity or falsification analysis to support inferences. NEEs are conducted using many different designs and statistical methods and encompass various definitions of a natural experiment, while it is questionable whether all evaluations reported as natural experiments should be considered as such. The likelihood of randomization should be specifically reported, and primary analyses should be supported by sensitivity analyses and/or falsification tests. Transparent reporting of NEE designs and evaluation methods will contribute to the optimum use of place-based NEEs. [Abstract copyright: Copyright © 2023 Albers, Rinaldi, Brown, Mason, d'Apice, McGill, McQuire, Craig, Laverty, Beeson, Campbell, Egan, Gibson, Fuller, Dillon, Taylor-Robinson, Jago, Tilling, Barr, Sniehotta, Hickman, Millett and de Vocht.]
U2 - 10.3389/fpubh.2023.1192055
DO - 10.3389/fpubh.2023.1192055
M3 - Review article
C2 - 37427271
VL - 11
JO - Frontiers in Public Health
JF - Frontiers in Public Health
SN - 2296-2565
M1 - 1192055
ER -