Neural networks are one of the first major milestones in developing artificial intelligence systems. The utilisation of integrated photonics in neural networks offers a promising alternative approach to microelectronic and hybrid optical-electronic implementations due to improvements in computational speed and low energy consumption in machine-learning tasks. However, at present, the optical realisation of the nonlinear activation function is a tremendous challenge. In my talk, I will discuss the challenges and novel approaches for implementing an all-optical neural nonlinear activation function based on utilising unique light-matter interactions with new class of 2D materials towards the realization of all-optically implemented deep neural networks.