Systems Engineering and Electronics ›› 2019, Vol. 41 ›› Issue (5): 964-971.doi: 10.3969/j.issn.1001-506X.2019.05.05

Previous Articles     Next Articles

Nonparametric detection of anomalous data with given constant false alarm rate

ZHANG Yidi, WANG Peizhi, LU Qiyong, ZHANG Jianqiu   

  1. Research Center of Smart Networks and Systems, Department of Electronics Engineering, School of
    Information Science and Technology, Fudan University, Shanghai 200433, China
  • Online:2019-04-30 Published:2019-04-26

Abstract:

In order to detect the outliers and/or anomalous data streams in data, a nonparametric method with a given constant false alarm rate under the maximum mean discrepancy and the mean embedding of distributions of the reproducing kernel Hilbert space is proposed. The normal data streams are first described as a probability distribution. Then, analyses show that the Neyman-Pearson hypothesis test via the described distribution can be exploited to do anomalous hypothesis tests with a given constant false alarm rate for data. It is also shown that the bootstrap resampling technique and/or expectation maximization algorithm can be used to estimate the distribution of the data streams. The judgment threshold required by the anomalous hypothesis test for the given constant false alarm rate can conveniently be obtained by the Monte-Carlo method so that the complex calculation for getting the threshold from the estimated distribution is simplified. Numerical simulation results verify the effectiveness of the proposed method and its superiority over other reported methods.

Key words: maximum mean discrepancy (MMD), constant false alarm rate (CFAR), anomaly detection, bootstrap resampling, xpectation-maximization (EM) algorithm, Monte-Carlo method

[an error occurred while processing this directive]