Intermittent demand time series involve items that are requested infrequently, resulting in sporadic demand. Croston's method and its variants have been proposed in the literature to address this forecasting problem. Recently other novel methods have appeared. Although the literature provides guidance on the suggested range for model parameters, a consistent and valid optimisation methodology is lacking. Growing evidence in the literature points against the use of conventional accuracy error metrics for model evaluation for intermittent demand time series. Consequently these may be inappropriate for parameter or model selection. This paper contributes to the discussion by evaluating a series of conventional time series error metrics, along with two novel ones for parameter optimisation for intermittent demand methods. The proposed metrics are found to not only perform best, but also provide consistent parameters with the literature, in contrast to conventional metrics. Furthermore, this work validates that employing different parameters for smoothing the non-zero demand and the inter-demand intervals of Croston's method and its variants is beneficial. The evaluated error metrics are considered for automatic model selection for each time series. Although they are found to perform similarly to theory driven model selection schemes, they fail to outperform single models substantially. These findings are validated using both out-of-sample forecast evaluation and inventory simulations.