Accepted author manuscript, 3.42 MB, PDF document
Available under license: CC BY: Creative Commons Attribution 4.0 International License
Submitted manuscript
Licence: CC BY: Creative Commons Attribution 4.0 International License
Final published version
Research output: Contribution to Journal/Magazine › Journal article › peer-review
Research output: Contribution to Journal/Magazine › Journal article › peer-review
}
TY - JOUR
T1 - A Log-Linear Non-Parametric Online Changepoint Detection Algorithm based on Functional Pruning
AU - Romano, Gaetano
AU - Eckley, Idris
AU - Fearnhead, Paul
PY - 2024/1/31
Y1 - 2024/1/31
N2 - Online changepoint detection aims to detect anomalies and changes in real time within high frequency data streams, sometimes with limited available computational resources. This is an important task that is rooted in many real-world applications including, but not limited to, cybersecurity, medicine and astrophysics. While fast and efficient online algorithms have been recently introduced, these rely on parametric assumptions which are often violated in practical applications. Motivated by data streams from the telecommunications sector, we build a flexible nonparametric approach to detect a change in the distribution of a sequence. Our procedure, NP-FOCuS, builds a sequential likelihood ratio test for a change, across at a set of points, of the empirical cumulative density function of our data. This is achieved by keeping track of the number of observations above or below those points. Thanks to functional pruning ideas, NP-FOCuS has a computational cost that is log-linear in the number of observations and is suitable for high-frequency data streams. In terms of detection power, NP-FOCuS is seen to outperform current nonparametric online changepoint techniques in a variety of settings. We demonstrate the utility of the procedure on both simulated and real data.
AB - Online changepoint detection aims to detect anomalies and changes in real time within high frequency data streams, sometimes with limited available computational resources. This is an important task that is rooted in many real-world applications including, but not limited to, cybersecurity, medicine and astrophysics. While fast and efficient online algorithms have been recently introduced, these rely on parametric assumptions which are often violated in practical applications. Motivated by data streams from the telecommunications sector, we build a flexible nonparametric approach to detect a change in the distribution of a sequence. Our procedure, NP-FOCuS, builds a sequential likelihood ratio test for a change, across at a set of points, of the empirical cumulative density function of our data. This is achieved by keeping track of the number of observations above or below those points. Thanks to functional pruning ideas, NP-FOCuS has a computational cost that is log-linear in the number of observations and is suitable for high-frequency data streams. In terms of detection power, NP-FOCuS is seen to outperform current nonparametric online changepoint techniques in a variety of settings. We demonstrate the utility of the procedure on both simulated and real data.
KW - Changepoint
KW - anomaly detection
KW - non-parametric
KW - online
KW - real-time analysis
KW - telecommunications
U2 - 10.1109/TSP.2023.3343550
DO - 10.1109/TSP.2023.3343550
M3 - Journal article
VL - 72
SP - 594
EP - 606
JO - IEEE Transactions on Signal Processing
JF - IEEE Transactions on Signal Processing
SN - 1053-587X
ER -