There is decades long research interest in artificial neural networks (ANNs) that has led to several successful applications. In forecasting, both in theoretical and empirical works, ANNs have shown evidence of good performance, in many cases outperforming established benchmark models. However, our understanding of their inner workings is still limited, which makes it difficult for academicians and practitioners alike to use them. Furthermore, while there is a growing literature supporting their good performance in forecasting, there is also a lot of scepticism whether ANNs are able to provide reliable and robust forecasts. This analysis presents the advances of ANNs in the time series forecasting field, highlighting the current state of the art, which modelling issues have been solved and which are still critical for forecasting with ANNs, indicating future research directions.