TY - UNPB

T1 - Measuring the Accuracy of Judgmental Adjustments to SKU-level Demand Forecasts

AU - Davydenko, A

AU - Fildes, R A

AU - Trapero Arenas, J R

PY - 2010

Y1 - 2010

N2 - The paper shows that due to the features of SKU (stock-keeping unit) demand data wellknown error measures previously used to analyse the accuracy of adjustments are generally not advisable for the task. In particular, percentage errors are affected by outliers and biases arising from a large number of low actual demand values and correlation between forecast errors and actual outcomes. It is also shown that MASE is equivalent to the arithmetic average of relative mean absolute errors (MAEs) and inherently is biased towards overrating the benchmark method. Therefore existing measures cannot deliver easily interpretable and unambiguous results. To overcome the imperfections of existing schemes a new measure is introduced which indicates average relative improvement of MAE. In contrast to MASE the proposed scheme is based on finding the geometric average of relative MAEs. This allows objective evaluation of relative change in forecasting accuracy yielded by the use of adjustments. Empirical analysis employed a large number of observations collected from a company specialising on manufacturing of fast-moving consumer goods (FMCG). The results suggest that adjustments reduced MAE of baseline statistical forecast on average by approximately 10%. Using a binomial test it was confirmed that adjustments improved the accuracy of forecasts significantly more frequently rather than they reduced it.

AB - The paper shows that due to the features of SKU (stock-keeping unit) demand data wellknown error measures previously used to analyse the accuracy of adjustments are generally not advisable for the task. In particular, percentage errors are affected by outliers and biases arising from a large number of low actual demand values and correlation between forecast errors and actual outcomes. It is also shown that MASE is equivalent to the arithmetic average of relative mean absolute errors (MAEs) and inherently is biased towards overrating the benchmark method. Therefore existing measures cannot deliver easily interpretable and unambiguous results. To overcome the imperfections of existing schemes a new measure is introduced which indicates average relative improvement of MAE. In contrast to MASE the proposed scheme is based on finding the geometric average of relative MAEs. This allows objective evaluation of relative change in forecasting accuracy yielded by the use of adjustments. Empirical analysis employed a large number of observations collected from a company specialising on manufacturing of fast-moving consumer goods (FMCG). The results suggest that adjustments reduced MAE of baseline statistical forecast on average by approximately 10%. Using a binomial test it was confirmed that adjustments improved the accuracy of forecasts significantly more frequently rather than they reduced it.

KW - judgmental adjustments

KW - forecasting support systems

KW - forecast accuracy

KW - forecast evaluation

KW - forecast error measures.

M3 - Working paper

T3 - Management Science Working Paper Series

BT - Measuring the Accuracy of Judgmental Adjustments to SKU-level Demand Forecasts

PB - The Department of Management Science

CY - Lancaster University

ER -