Attacks on critical infrastructures’ Supervisory Control and Data Acquisition (SCADA) systems are beginning to increase. They are often initiated by highly skilled attackers, who are capable of deploying sophisticated attacks to exfiltrate data or even to cause physical damage. In this paper, we rehearse the rationale for protecting against cyber attacks and evaluate a set of Anomaly Detection (AD) techniques in detecting attacks by analysing traffic captured in a SCADA network. For this purpose, we have implemented a tool chain with a reference implementation of various state-of-the-art AD techniques to detect attacks, which manifest themselves as anomalies. Specifically, in order to evaluate the AD techniques, we apply our tool chain on a dataset created from a gas pipeline SCADA system in Mississippi State University’s lab, which include artefacts of both normal operations and cyber attack scenarios. Our evaluation elaborate on several performance metrics of the examined AD techniques
such as precision; recall; accuracy; F-score and G-score. The results indicate that detection rate may change significantly when considering various attack types and different detections modes (i.e., supervised and unsupervised), and also provide indications that there is a need for a robust, and preferably real-time AD technique to introduce resilience in critical infrastructures.