This preview shows pages 1–2. Sign up to view the full content.
This preview has intentionally blurred sections. Sign up to view the full version.View Full Document
Unformatted text preview: STAT 100B Note 2 1 Topics (1) Maximum likelihood estimation. (2) Likelihood ratio test and Bayes rule for classification. (3) Generalized likelihood ratio test. 2 Mathematical preparation (1) Let x i = ( x i 1 ,...,x ip ) T and = ( 1 ,..., p ) T be two column vectors. Then x i 1 1 + ... + x ip p = p X j =1 x ij j = h x i , i = x T i . (2) Let f ( ) = x T i , then f ( ) / j = x ij for j = 1 ,...,p . Define f ( ) = ( f ( ) / 1 ,...,f ( ) / p ) T to be a column vector, then f ( ) = ( x i 1 ,...,x ip ) T = x i . (3) Let g ( ) = h ( x T i ), then according to the chain rule, g ( ) / j = h ( x T i ) ( x T i ) / j = h ( x T i ) x ij . g ( ) = ( g ( ) / 1 ,...,g ( ) / p ) T = ( h ( x T i ) x i 1 ,...,h ( x T i ) x ip ) T = h ( x T i ) x i . (4) h x i , i = | x i || | cos , where is the angle between x i and . So x T i / | | = | x i | cos can be considered the projection of the vector x i onto the vector . 3 Maximum likelihood estimation Let x 1 ,...,x n p ( x, ) independently, where p ( x, ) is a probability mass function (if x is discrete) or a probability density function (if x is continuous), and is the parameter. The maximum likelihood estimation consists of the following steps. (0) Write down the likelihood function: L ( ) = Q n i =1 p ( x i , ). (1) Take log to get the log-likelihood function: l ( ) = log L ( ) = n i =1 log p ( x i , ). (2) Take derivative: l ( ) = n i =1 log p ( x i , ). (3) Solve from the maximum likelihood estimating equation: l ( ) = 0. Example 1: Bernoulli. Let x 1 ,...,x n Bernoulli( p ) independently. The likelihood L ( p ) = n Y i =1 p x i (1- p ) 1- x i = p n i =1 x i (1- p ) n i =1 (1- x i ) . The log-likelihood l ( p ) = log L ( p ) = n X i =1 x i log p + ( n- n X i =1 x i )log(1- p ) ....
View Full Document
This note was uploaded on 09/20/2011 for the course STAT 100B taught by Professor Wu during the Winter '11 term at UCLA.
- Winter '11