This preview shows pages 1–2. Sign up to view the full content.
This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Unformatted text preview: STAT 100B Note 2 1 Topics (1) Maximum likelihood estimation. (2) Likelihood ratio test and Bayes rule for classification. (3) Generalized likelihood ratio test. 2 Mathematical preparation (1) Let x i = ( x i 1 ,...,x ip ) T and = ( 1 ,..., p ) T be two column vectors. Then x i 1 1 + ... + x ip p = p X j =1 x ij j = h x i , i = x T i . (2) Let f ( ) = x T i , then f ( ) / j = x ij for j = 1 ,...,p . Define f ( ) = ( f ( ) / 1 ,...,f ( ) / p ) T to be a column vector, then f ( ) = ( x i 1 ,...,x ip ) T = x i . (3) Let g ( ) = h ( x T i ), then according to the chain rule, g ( ) / j = h ( x T i ) ( x T i ) / j = h ( x T i ) x ij . g ( ) = ( g ( ) / 1 ,...,g ( ) / p ) T = ( h ( x T i ) x i 1 ,...,h ( x T i ) x ip ) T = h ( x T i ) x i . (4) h x i , i =  x i   cos , where is the angle between x i and . So x T i /   =  x i  cos can be considered the projection of the vector x i onto the vector . 3 Maximum likelihood estimation Let x 1 ,...,x n p ( x, ) independently, where p ( x, ) is a probability mass function (if x is discrete) or a probability density function (if x is continuous), and is the parameter. The maximum likelihood estimation consists of the following steps. (0) Write down the likelihood function: L ( ) = Q n i =1 p ( x i , ). (1) Take log to get the loglikelihood function: l ( ) = log L ( ) = n i =1 log p ( x i , ). (2) Take derivative: l ( ) = n i =1 log p ( x i , ). (3) Solve from the maximum likelihood estimating equation: l ( ) = 0. Example 1: Bernoulli. Let x 1 ,...,x n Bernoulli( p ) independently. The likelihood L ( p ) = n Y i =1 p x i (1 p ) 1 x i = p n i =1 x i (1 p ) n i =1 (1 x i ) . The loglikelihood l ( p ) = log L ( p ) = n X i =1 x i log p + ( n n X i =1 x i )log(1 p ) ....
View
Full
Document
This note was uploaded on 09/20/2011 for the course STAT 100B taught by Professor Wu during the Winter '11 term at UCLA.
 Winter '11
 Wu

Click to edit the document details