Home > Research > Publications & Outputs > Reproducibility in forecasting research

Electronic data

  • Reproducibility IJF Boylan et al

    Rights statement: This is the author’s version of a work that was accepted for publication in International Journal of Forecasting. Changes resulting from the publishing process, such as peer review, editing, corrections, structural formatting, and other quality control mechanisms may not be reflected in this document. Changes may have been made to this work since it was submitted for publication. A definitive version was subsequently published in International Journal of Forecasting, 31, 2015 DOI: 10.1016/j.ijforecast.2014.05.008

    Accepted author manuscript, 1.01 MB, PDF document

    Available under license: CC BY-NC-ND: Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License

Links

Text available via DOI:

View graph of relations

Reproducibility in forecasting research

Research output: Contribution to journalJournal articlepeer-review

Published

Standard

Reproducibility in forecasting research. / Boylan, John; Goodwin, Paul; Mohammadipour, Maryam; Syntetos, Aris.

In: International Journal of Forecasting, Vol. 31, No. 1, 01.2015, p. 79-90.

Research output: Contribution to journalJournal articlepeer-review

Harvard

Boylan, J, Goodwin, P, Mohammadipour, M & Syntetos, A 2015, 'Reproducibility in forecasting research', International Journal of Forecasting, vol. 31, no. 1, pp. 79-90. https://doi.org/10.1016/j.ijforecast.2014.05.008

APA

Boylan, J., Goodwin, P., Mohammadipour, M., & Syntetos, A. (2015). Reproducibility in forecasting research. International Journal of Forecasting, 31(1), 79-90. https://doi.org/10.1016/j.ijforecast.2014.05.008

Vancouver

Boylan J, Goodwin P, Mohammadipour M, Syntetos A. Reproducibility in forecasting research. International Journal of Forecasting. 2015 Jan;31(1):79-90. https://doi.org/10.1016/j.ijforecast.2014.05.008

Author

Boylan, John ; Goodwin, Paul ; Mohammadipour, Maryam ; Syntetos, Aris. / Reproducibility in forecasting research. In: International Journal of Forecasting. 2015 ; Vol. 31, No. 1. pp. 79-90.

Bibtex

@article{919b86fdaa92477aa21433311789db3a,
title = "Reproducibility in forecasting research",
abstract = "The importance of replication has been recognised across many scientific disciplines. Reproducibility is a necessary condition for replicability because an inability to reproduce results implies that the methods have been insufficiently specified, thus precluding replication. This paper describes how two independent teams of researchers attempted to reproduce the empirical findings of an important paper, “Shrinkage estimators of time series seasonal factors and their effect on forecasting accuracy” (Miller & Williams, 2003, IJF). The teams of researchers proceeded systematically, reporting results before and after receiving clarifications from the authors of the original study. The teams were able to approximately reproduce each other{\textquoteright}s results but not those of Miller & Williams. These discrepancies led to differences in the conclusions on conditions under which seasonal damping outperforms Classical Decomposition. The paper specifies the forecasting methods employed using a flowchart. It is argued that this approach to method documentation is complementary to the provision of computer code, as it is accessible to a broader audience of forecasting practitioners and researchers. The significance of this research lies not only in its lessons for seasonal forecasting but, more generally, in its approach to the reproduction of forecasting research. ",
keywords = "Forecasting practice, Replication, Seasonal Forecasting, Empirical research ",
author = "John Boylan and Paul Goodwin and Maryam Mohammadipour and Aris Syntetos",
note = "This is the author{\textquoteright}s version of a work that was accepted for publication in International Journal of Forecasting. Changes resulting from the publishing process, such as peer review, editing, corrections, structural formatting, and other quality control mechanisms may not be reflected in this document. Changes may have been made to this work since it was submitted for publication. A definitive version was subsequently published in International Journal of Forecasting, 31, 2015 DOI: 10.1016/j.ijforecast.2014.05.008 ",
year = "2015",
month = jan,
doi = "10.1016/j.ijforecast.2014.05.008",
language = "English",
volume = "31",
pages = "79--90",
journal = "International Journal of Forecasting",
issn = "0169-2070",
publisher = "Elsevier Science B.V.",
number = "1",

}

RIS

TY - JOUR

T1 - Reproducibility in forecasting research

AU - Boylan, John

AU - Goodwin, Paul

AU - Mohammadipour, Maryam

AU - Syntetos, Aris

N1 - This is the author’s version of a work that was accepted for publication in International Journal of Forecasting. Changes resulting from the publishing process, such as peer review, editing, corrections, structural formatting, and other quality control mechanisms may not be reflected in this document. Changes may have been made to this work since it was submitted for publication. A definitive version was subsequently published in International Journal of Forecasting, 31, 2015 DOI: 10.1016/j.ijforecast.2014.05.008

PY - 2015/1

Y1 - 2015/1

N2 - The importance of replication has been recognised across many scientific disciplines. Reproducibility is a necessary condition for replicability because an inability to reproduce results implies that the methods have been insufficiently specified, thus precluding replication. This paper describes how two independent teams of researchers attempted to reproduce the empirical findings of an important paper, “Shrinkage estimators of time series seasonal factors and their effect on forecasting accuracy” (Miller & Williams, 2003, IJF). The teams of researchers proceeded systematically, reporting results before and after receiving clarifications from the authors of the original study. The teams were able to approximately reproduce each other’s results but not those of Miller & Williams. These discrepancies led to differences in the conclusions on conditions under which seasonal damping outperforms Classical Decomposition. The paper specifies the forecasting methods employed using a flowchart. It is argued that this approach to method documentation is complementary to the provision of computer code, as it is accessible to a broader audience of forecasting practitioners and researchers. The significance of this research lies not only in its lessons for seasonal forecasting but, more generally, in its approach to the reproduction of forecasting research.

AB - The importance of replication has been recognised across many scientific disciplines. Reproducibility is a necessary condition for replicability because an inability to reproduce results implies that the methods have been insufficiently specified, thus precluding replication. This paper describes how two independent teams of researchers attempted to reproduce the empirical findings of an important paper, “Shrinkage estimators of time series seasonal factors and their effect on forecasting accuracy” (Miller & Williams, 2003, IJF). The teams of researchers proceeded systematically, reporting results before and after receiving clarifications from the authors of the original study. The teams were able to approximately reproduce each other’s results but not those of Miller & Williams. These discrepancies led to differences in the conclusions on conditions under which seasonal damping outperforms Classical Decomposition. The paper specifies the forecasting methods employed using a flowchart. It is argued that this approach to method documentation is complementary to the provision of computer code, as it is accessible to a broader audience of forecasting practitioners and researchers. The significance of this research lies not only in its lessons for seasonal forecasting but, more generally, in its approach to the reproduction of forecasting research.

KW - Forecasting practice

KW - Replication

KW - Seasonal Forecasting

KW - Empirical research

U2 - 10.1016/j.ijforecast.2014.05.008

DO - 10.1016/j.ijforecast.2014.05.008

M3 - Journal article

VL - 31

SP - 79

EP - 90

JO - International Journal of Forecasting

JF - International Journal of Forecasting

SN - 0169-2070

IS - 1

ER -