Lecture23HO - CS440/ECE448: Intro to Articial Intelligence!...

Info iconThis preview shows pages 1–3. Sign up to view the full content.

View Full Document Right Arrow Icon
Lecture 23: Decision Trees Prof. Julia Hockenmaier juliahmr@illinois.edu http://cs.illinois.edu/fa11/cs440 CS440/ECE448: Intro to ArtiFcial Intelligence Decision trees Decision trees 3 CS440/ECE448: Intro AI drink? milk? milk? coffee tea yes no no sugar sugar yes no sugar no sugar Decision tree learning Training data D = {( x 1 , y 1 ),…, ( x N , y N )} each x i = ( x 1 i ,…., x d i ) is a d -dimensional feature vector each y i is the target label (class) of the i-th data point Training algorithm: Initial tree = the root, corresponding to all items in D A node is a leaf if all its data items have the same y At each non-leaf node: Fnd the feature x i with the highest information gain, create a new child for each value of x i , distribute the items accordingly. 4 CS440/ECE448: Intro AI
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Information Gain How much information are we gaining by splitting node S on attribute A with values V(A) ? Information required before the split: H(S parent ) Information required after the split: ! i ˥ V(A) P(S child_i )H(S child_i ) Gain ( S parent , A ) = H ( S parent ) ! H ( S child i ) i " V ( A ) N # S child i S parent Dealing with numerical attributes Many attributes are not boolean (0,1) or nominal (classes) Number of times a word appears in a text RGB values of a pixel height, weight, ….
Background image of page 2
Image of page 3
This is the end of the preview. Sign up to access the rest of the document.

Page1 / 7

Lecture23HO - CS440/ECE448: Intro to Articial Intelligence!...

This preview shows document pages 1 - 3. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online