The paper shows that due to the features of SKU (stock-keeping unit) demand data wellknown
error measures previously used to analyse the accuracy of adjustments are
generally not advisable for the task. In particular, percentage errors are affected by
outliers and biases arising from a large number of low actual demand values and
correlation between forecast errors and actual outcomes. It is also shown that MASE is
equivalent to the arithmetic average of relative mean absolute errors (MAEs) and
inherently is biased towards overrating the benchmark method. Therefore existing
measures cannot deliver easily interpretable and unambiguous results.
To overcome the imperfections of existing schemes a new measure is introduced which
indicates average relative improvement of MAE. In contrast to MASE the proposed
scheme is based on finding the geometric average of relative MAEs. This allows
objective evaluation of relative change in forecasting accuracy yielded by the use of
adjustments. Empirical analysis employed a large number of observations collected from
a company specialising on manufacturing of fast-moving consumer goods (FMCG). The
results suggest that adjustments reduced MAE of baseline statistical forecast on average
by approximately 10%. Using a binomial test it was confirmed that adjustments improved
the accuracy of forecasts significantly more frequently rather than they reduced it.