100Bnote2

100Bnote2 - STAT 100B Note 2 1 Topics (1) Maximum...

Info iconThis preview shows pages 1–2. Sign up to view the full content.

View Full Document Right Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: STAT 100B Note 2 1 Topics (1) Maximum likelihood estimation. (2) Likelihood ratio test and Bayes rule for classification. (3) Generalized likelihood ratio test. 2 Mathematical preparation (1) Let x i = ( x i 1 ,...,x ip ) T and = ( 1 ,..., p ) T be two column vectors. Then x i 1 1 + ... + x ip p = p X j =1 x ij j = h x i , i = x T i . (2) Let f ( ) = x T i , then f ( ) / j = x ij for j = 1 ,...,p . Define f ( ) = ( f ( ) / 1 ,...,f ( ) / p ) T to be a column vector, then f ( ) = ( x i 1 ,...,x ip ) T = x i . (3) Let g ( ) = h ( x T i ), then according to the chain rule, g ( ) / j = h ( x T i ) ( x T i ) / j = h ( x T i ) x ij . g ( ) = ( g ( ) / 1 ,...,g ( ) / p ) T = ( h ( x T i ) x i 1 ,...,h ( x T i ) x ip ) T = h ( x T i ) x i . (4) h x i , i = | x i || | cos , where is the angle between x i and . So x T i / | | = | x i | cos can be considered the projection of the vector x i onto the vector . 3 Maximum likelihood estimation Let x 1 ,...,x n p ( x, ) independently, where p ( x, ) is a probability mass function (if x is discrete) or a probability density function (if x is continuous), and is the parameter. The maximum likelihood estimation consists of the following steps. (0) Write down the likelihood function: L ( ) = Q n i =1 p ( x i , ). (1) Take log to get the log-likelihood function: l ( ) = log L ( ) = n i =1 log p ( x i , ). (2) Take derivative: l ( ) = n i =1 log p ( x i , ). (3) Solve from the maximum likelihood estimating equation: l ( ) = 0. Example 1: Bernoulli. Let x 1 ,...,x n Bernoulli( p ) independently. The likelihood L ( p ) = n Y i =1 p x i (1- p ) 1- x i = p n i =1 x i (1- p ) n i =1 (1- x i ) . The log-likelihood l ( p ) = log L ( p ) = n X i =1 x i log p + ( n- n X i =1 x i )log(1- p ) ....
View Full Document

This note was uploaded on 09/20/2011 for the course STAT 100B taught by Professor Wu during the Winter '11 term at UCLA.

Page1 / 6

100Bnote2 - STAT 100B Note 2 1 Topics (1) Maximum...

This preview shows document pages 1 - 2. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online