Jia li httpwwwstatpsuedujiali classificationdecision 2

Info iconThis preview shows page 1. Sign up to view the full content.

View Full Document Right Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: signed to node t T is denoted by (t). For 0-1 loss, the class assignment rule is: (t) = arg max p(j | t) . j Jia Li http://www.stat.psu.edu/jiali Classification/Decision Trees (I) The resubstitution estimate r (t) of the probability of misclassification, given that a case falls into node t is r (t) = 1 - max p(j | t) = 1 - p((t) | t) . j Denote R(t) = r (t)p(t). The resubstitution estimate for the overall misclassification rate R(T ) of the tree classifier T is: R(T ) = R(t) . ~ tT Jia Li http://www.stat.psu.edu/jiali Classification/Decision Trees (I) Proposition: For any split of a node t into tL and tR , R(t) R(tL ) + R(tR ) . Proof: Denote j = (t). p(j | t) = p(j , tL | t) + p(j , tR | t) = p(j | tL )p(tL | t) + p(j | tR )p(tR | t) = pL p(j | tL ) + pR p(j | tR ) j j pL max p(j | tL ) + pR max p(j | tR ) Jia Li http://www.stat.psu.edu/jiali Classification/Decision Trees (I) Hence, r (t) = 1 - p(j | t) 1 - pL max p(j | tL ) + pR max p(j | tR ) j j j j = pL (1 - max p(j | tL )) + pR (1 - max p(j | tR )) = pL r (tL ) + pR r (tR ) Finally, R(t) = p(t)r (t) p(t)pL r (tL ) + p(t)pR r (tR ) = p(tL )r (tL ) + p(tR )r (tR ) = R(tL ) + R(tR ) Jia Li http://www.stat.psu.edu/jiali Classification/Decision Trees (I) Digit Recognition Example (CART) The 10 digits are shown by different on-off combinations of seven horizontal and vertical lights. Each digit is represented by a 7-dimensional vector of zeros and ones. The ith sample is xi = (xi1 , xi2 , ..., xi7 ). If xij = 1, the jth light is on; if xij = 0, the jth light is off. http://www.stat.psu.edu/jiali Jia Li Classification/Decision Tr...
View Full Document

This note was uploaded on 02/04/2012 for the course STAT 557 taught by Professor Jiali during the Fall '09 term at Penn State.

Ask a homework question - tutors are online