Research output: Working paper
Research output: Working paper
}
TY - UNPB
T1 - Measuring the Accuracy of Judgmental Adjustments to SKU-level Demand Forecasts
AU - Davydenko, A
AU - Fildes, R A
AU - Trapero Arenas, J R
PY - 2010
Y1 - 2010
N2 - The paper shows that due to the features of SKU (stock-keeping unit) demand data wellknown error measures previously used to analyse the accuracy of adjustments are generally not advisable for the task. In particular, percentage errors are affected by outliers and biases arising from a large number of low actual demand values and correlation between forecast errors and actual outcomes. It is also shown that MASE is equivalent to the arithmetic average of relative mean absolute errors (MAEs) and inherently is biased towards overrating the benchmark method. Therefore existing measures cannot deliver easily interpretable and unambiguous results. To overcome the imperfections of existing schemes a new measure is introduced which indicates average relative improvement of MAE. In contrast to MASE the proposed scheme is based on finding the geometric average of relative MAEs. This allows objective evaluation of relative change in forecasting accuracy yielded by the use of adjustments. Empirical analysis employed a large number of observations collected from a company specialising on manufacturing of fast-moving consumer goods (FMCG). The results suggest that adjustments reduced MAE of baseline statistical forecast on average by approximately 10%. Using a binomial test it was confirmed that adjustments improved the accuracy of forecasts significantly more frequently rather than they reduced it.
AB - The paper shows that due to the features of SKU (stock-keeping unit) demand data wellknown error measures previously used to analyse the accuracy of adjustments are generally not advisable for the task. In particular, percentage errors are affected by outliers and biases arising from a large number of low actual demand values and correlation between forecast errors and actual outcomes. It is also shown that MASE is equivalent to the arithmetic average of relative mean absolute errors (MAEs) and inherently is biased towards overrating the benchmark method. Therefore existing measures cannot deliver easily interpretable and unambiguous results. To overcome the imperfections of existing schemes a new measure is introduced which indicates average relative improvement of MAE. In contrast to MASE the proposed scheme is based on finding the geometric average of relative MAEs. This allows objective evaluation of relative change in forecasting accuracy yielded by the use of adjustments. Empirical analysis employed a large number of observations collected from a company specialising on manufacturing of fast-moving consumer goods (FMCG). The results suggest that adjustments reduced MAE of baseline statistical forecast on average by approximately 10%. Using a binomial test it was confirmed that adjustments improved the accuracy of forecasts significantly more frequently rather than they reduced it.
KW - judgmental adjustments
KW - forecasting support systems
KW - forecast accuracy
KW - forecast evaluation
KW - forecast error measures.
M3 - Working paper
T3 - Management Science Working Paper Series
BT - Measuring the Accuracy of Judgmental Adjustments to SKU-level Demand Forecasts
PB - The Department of Management Science
CY - Lancaster University
ER -