approach Non-parametric in detection problem
Zhang Benhong
Outline
Introduction Parametric approach: T2 test Non-parametric approach: bootstrap, empirical likelihood test Bai's test (suitable for high dimensional problems) Conclusion
Introduction
Normally the distribution of the measurement should be known to achieve optimal detection performance In practice the exact distribution may not be available Non-parametric solutions are introduced and their performances are evaluated asymptotically
Problem and Notations
Let x = ( x1 , x2 ,..., xN )T be N i.i.d. measurements, each xi is p 1 random vector with mean and variance , denote: N N S = i =1 ( xi - x )( xi - x )T / N - 1 x = i =1 xi / N be the sample mean and be the sample covariance Our goal is to test two hypotheses: H 0 : = 0 H1 : 0 using Neyman-Pearson criterion: Fix PF= and try to achieve largest available PD
T2 test
The test statistic is set to be: T 2 = N ( x - 0 )T S - 1 ( x - 0 ) and it is well known that when = 0, and the data xi is normally distributed,
N-p T 2 ~ Fp , N - p p( N -1)
Therefore, the following rule is applied to achieve maximum PD at PF=:
T 2 < Fp ,( N - p ), H 0 2 T > Fp ,( N - p ), H1
T2 test (cont.)
T2 test is exact ML test under normal data T2 is still asymptotically F distributed when p<<N, without the requirement of data distribution When p is comparable with N, or both p and N are small, the distribution of T2 can not be decided and the performance T2 test will degrade seriously. Especially, when p>N, T2 test can not be used since the sample covariance matrix S is not invertible
Bootstrap test
Still set the test statistic to be T2 We simulate the distribution of T2 by performing the following steps: 1. Generate resample of size N, by uniformly sampling {x, 0-x} with replacement, denote such resample as x* = {xi*}iN=1 2. Compute T2 based on x*, denote as T*2 {Ti*2 }iB=1 3. Repeat 1,2 B (big) times, the histogram of asymptotically represents the pdf of T2 *2 T(1) T(*2 ... T(*2) {Ti*2 }iB=1 2) B Sort as , bootstrap test is performed as:
*2 T 2 < T([ B (1- )]) H 0 2 *2 T > T([ B (1- )]) H1
Bootstrap test (cont.)
Bootstrap use Monte Carlo approach to estimate the -quantile of the test statistic, thus is more accurate than original T2 test when T2 is not F distributed When computing T*2 in step 2, S* may not be invertible even if p<N, due to repeated resamples It is still not able to handle p>N case Bootstrap can be considered as a calibration method
Empirical likelihood (EL) test
In EL test, each xi is assigned multinomial probability pi for i=1,2,...,N. The log likelihood function of mean 0 is:
L( 0 ) = max i =1 ln( Npi )
N p
subject to the following constraint: N
i =1
pi ( xi - 0 ) = 0,
N
pi 0, i =1 pi = 1
and the log likelihood ratio is defined ^as:
l ( 0 ) = -2( L( 0 ) - L( ))
EL test (cont.)
where L( ) = max L( ) 0 N i=1 pi ( xi - 0 ) = 0 , It is clear that without the constraint
L = max i =1 ln Npi = 0
N p
^
at pi=1/N, (i=1,2,...,N), resulting = x
^
To compute L(0), we apply Lagrange multipliers method:
f (p) = i =1 ln Npi - N i =1 pi ( xi - 0 ) - (i =1 pi - 1),
N N N
f (p) / p = 1 / p - N ( x - ) - i i i 0 N g (p) = i =1 pi ( xi - 0 ) = 0 N h(p) = i =1 pi - 1 = 0
EL test (cont.)
The group equation has solutions at:
= N,
xi - 0 1 1 = 0, and pi = i =1 1 + ( x - ) N 1 + ( xi - ) i 0
N
The solution of may be found by numerical searching the second equation. Note that the second equation does not have solution when 0 is not located within the convex hull of x, therefore, EL method is not able to give the exact likelihood under such scenario
EL test (cont.)
In the previous Lagrange multipliers method, the restriction of pi is not considered and therefore the final solution may not be valid. We introduce pseudo-logarithm function to resolve this issue:
log( z ), log* ( z ) = log(1 / N ) - 1.5 + 2 Nz - ( Nz ) 2 / 2 if z 1 / N if z < 1 / N
Using the pseudo-log function to replace the ordinary log function in the likelihood expression, the value of L(0) does not change at the true solution of pi while 0 pi 1 is guaranteed
EL test (cont.)
2 p, It can be proved that l(0) is asymptotically distributed as
therefore EL test is performed as the following:
2 l ( 0 ) < p , H 0 2 l ( 0 ) > p , H 1
EL test does not require the measurement distribution 2 p For small N, l(0) may not be well approximated by distribution Bootstrap method can be used here for threshold calibration
Bai's test
Ordinary T2 test degrades drastically with the increase of dimension p, when p / N = y (0,1) . This is due to the inaccurate estimation of , which is composed of p2 nuance parameters Bai's test totally discards the information of covariance by using the following simple test statistic:
M N = ( x - 0 )T ( x - 0 ) - 1 trS N
it is proved that MN is asymptotically Gaussian distributed with 0 mean and variance:
2 M =
2 tr ( 2 ) ( N - 1) 2
Bai's test (cont.)
where a ratio-consistent estimator of tr(2) is given as:
( N -1) 2 1 B = ( trS 2 - ( trS ) 2 ) ( N +1)( N - 2) N -1
2
The test is formulated as:
M N < z H 0 M M N > z H 1 M
Bai's test does not require data distribution Bai's test is robust with the inc...