TY - GEN

T1 - Anomaly detection based on eccentricity analysis

AU - Angelov, Plamen

PY - 2014/12

Y1 - 2014/12

N2 - In this paper, we propose a new eccentricity- based anomaly detection principle and algorithm. It is based on a further development of the recently introduced data analytics framework (TEDA – from typicality and eccentricity data analytics). We compare TEDA with the traditional statistical approach and prove that TEDA is a generalization of it in regards to the well-known “ns” analysis (TEDA gives exactly the same result as the traditional “ns” analysis but it does not require the restrictive prior assumptions that are made for the traditional approach to be in place). Moreover, it offers a nonparametric, closed form analytical descriptions (models of the data distribution) to be extracted from the real data realizations,not to be pre-assumed. In addition to that, for several types of proximity/similarity measures (such as Euclidean, cosine, Mahalonobis) it can be calculated recursively, thus, computationally very efficiently and is suitable for real time and online algorithms. Building on the per data sample, exactinformation about the data distribution in a closed analytical form, in this paper we propose a new less conservative and more sensitive condition for anomaly detection. It is quite different from the traditional “ns” type conditions. We demonstrate example where traditional conditions would lead to an increasedamount of false negatives or false positives in comparison with the proposed condition. The new condition is intuitive and easy to check for arbitrary data distribution and arbitrary small (but not less than 3) amount of data samples/points. Finally, because the anomaly/novelty/change detection is very important and basic data analysis operation which is in the fundament of suchhigher level tasks as fault detection, drift detection in data streams, clustering, outliers detection, autonomous video analytics, particle physics, etc. we point to some possible applications which will be the domain of future work.

AB - In this paper, we propose a new eccentricity- based anomaly detection principle and algorithm. It is based on a further development of the recently introduced data analytics framework (TEDA – from typicality and eccentricity data analytics). We compare TEDA with the traditional statistical approach and prove that TEDA is a generalization of it in regards to the well-known “ns” analysis (TEDA gives exactly the same result as the traditional “ns” analysis but it does not require the restrictive prior assumptions that are made for the traditional approach to be in place). Moreover, it offers a nonparametric, closed form analytical descriptions (models of the data distribution) to be extracted from the real data realizations,not to be pre-assumed. In addition to that, for several types of proximity/similarity measures (such as Euclidean, cosine, Mahalonobis) it can be calculated recursively, thus, computationally very efficiently and is suitable for real time and online algorithms. Building on the per data sample, exactinformation about the data distribution in a closed analytical form, in this paper we propose a new less conservative and more sensitive condition for anomaly detection. It is quite different from the traditional “ns” type conditions. We demonstrate example where traditional conditions would lead to an increasedamount of false negatives or false positives in comparison with the proposed condition. The new condition is intuitive and easy to check for arbitrary data distribution and arbitrary small (but not less than 3) amount of data samples/points. Finally, because the anomaly/novelty/change detection is very important and basic data analysis operation which is in the fundament of suchhigher level tasks as fault detection, drift detection in data streams, clustering, outliers detection, autonomous video analytics, particle physics, etc. we point to some possible applications which will be the domain of future work.

KW - TEDA

KW - typicality

KW - eccentricity

KW - data density

U2 - 10.1109/EALS.2014.7009497

DO - 10.1109/EALS.2014.7009497

M3 - Conference contribution/Paper

SN - 9781479944958

SP - 1

EP - 8

BT - 2014 IEEE Symposium on Evolving and Autonomous Learning Systems (EALS)

PB - IEEE Press

CY - Orlando, FL, USA

T2 - 2014 IEEE Symposium Series on Computational Intelligence

Y2 - 9 December 2014 through 12 December 2014

ER -