# 7.0 Decision Trees.ppt - Decision Trees Decision tree...

• 34

This preview shows page 1 - 12 out of 34 pages.

Decision Trees Decision tree representation Iterative Dichotomiser 3 (ID3) learning algorithm Entropy, information gain Overfitting
2 Supplimentary material www /dtrees/4_dtrees1.html
3 Decision Tree for PlayTennis Outlook Sunny Overcast Rain Humidity High Normal Wind Strong Weak No Yes Yes Yes No
4 Decision Tree for PlayTennis Outlook Sunny Overcast Rain Humidity High Normal No Yes Each internal node tests an attribute Each branch corresponds to an attribute value node Each leaf node assigns a classificatio
5 No Decision Tree for PlayTennis Outlook Sunny Overcast Rain Humidity High Normal Wind Strong Weak No Yes Yes Yes No Outlook Temperature Humidity Wind PlayTennis Sunny Hot High Weak ?
6 Decision Tree for Conjunction Outlook Sunny Overcast Rain Wind Strong Weak No Yes No Outlook=Sunny Wind=Weak No
7 Decision Tree for Disjunction Outlook Sunny Overcast Rain Yes Outlook=Sunny Wind=Weak Wind Strong Weak No Yes Wind Strong Weak No Yes
8 Decision Tree for XOR Outlook Sunny Overcast Rain Wind Strong Weak Yes No Outlook=Sunny XOR Wind=Weak Wind Strong Weak No Yes Wind Strong Weak No Yes
9 Decision Tree Outlook Sunny Overcast Rain Humidity High Normal Wind Strong Weak No Yes Yes Yes No decision trees represent disjunctions of conjunctions (Outlook=Sunny Humidity=Normal) (Outlook=Overcast) (Outlook=Rain Wind=Weak)
10 When to consider Decision Trees Instances describable by attribute-value pairs Target function is discrete valued Disjunctive hypothesis may be required Possibly noisy training data Missing attribute values Examples: Medical diagnosis Credit risk analysis Object classification for robot manipulator (Tan 1993)
11 Top-Down Induction of Decision Trees ID3 1. A the “best” decision attribute for next node 2. Assign A as decision attribute for node 3. For each value of A create new descendant 4. Sort training examples to leaf node according to the attribute value of the branch 5. If all training examples are perfectly In decision tree learning, ID3 ( Iterative Dichotomiser 3 ) is an algorithm used to generate a decision tree invented by Ross Quinlan