{[ promptMessage ]}

Bookmark it

{[ promptMessage ]}


decision-tree-learning - Decision Tree Learning CS 464...

Info iconThis preview shows pages 1–3. Sign up to view the full content.

View Full Document Right Arrow Icon
1 CS 464: Introduction to Machine Learning Decision Tree Learning Slides adapted from Chapter 3 Machine Learning by Tom M. Mitchell http://www-2.cs.cmu.edu/afs/cs.cmu.edu/user/mitchell/ftp/mlbook.html 2 Decision Tree Learning Decision tree learning is a method for approximating discrete-valued target functions. The learned function is represented by a decision tree . A learned decision tree can also be re-represented as a set of if-then rules. It is robust to noisy data and capable of learning disjunctive expressions. Decision tree learning is one of the most widely used and practical methods for inductive inference. 3 Decision Tree for PlayTennis Outlook Sunny Overcast Rain Humidity High Normal Wind Strong Weak No Yes Yes Yes No 4 Decision Tree Decision trees represent a disjunction of conjunctions of constraints on the attribute values of instances. Each path from the tree root to a leaf corresponds to a conjunction of attribute tests, and The tree itself is a disjunction of these conjunctions. (Outlook = Sunny Humidity = Normal) (Outlook = Overcast) (Outlook = Rain Wind = Weak) Outlook Sunny Overcast Rain Humidity High Normal Wind Strong Weak No Yes Yes Yes No 5 Decision Tree Decision trees classify instances by sorting them down the tree from the root to some leaf node, which provides the classification of the instance. Each node in the tree specifies a test of some attribute of the instance. Each branch descending from a node corresponds to one of the possible values for the attribute. Each leaf node assigns a classification. The instance (Outlook=Sunny, Temperature=Hot, Humidity=High, Wind=Strong) is classified as a negative instance. 6 Continuous (real-valued) features can be handled by allowing nodes to split a real valued feature into two ranges based on a threshold (e.g. length < 3 and length 3) Classification trees have discrete class labels at the leaves, regression trees allow real-valued outputs at the leaves. Algorithms for finding consistent trees are efficient for processing large amounts of training data for data mining tasks. Methods developed for handling noisy training data (both class and feature noise). Methods developed for handling missing feature values. Properties of Decision Tree Learning
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
7 Top-Down Decision Tree Induction (ID3) Recursively build a tree top-down by divide and conquer. Main loop: 1. A the “best” decision attribute for next node 2. Assign A as decision attribute for node 3. For each value of A , create new descendant of node 4. Sort training examples to leaf nodes 5. If training examples perfectly classified, Then STOP, Else iterate over new leaf nodes 8 Which Attribute is Best? S: [29+,35-] Attributes: A and B possible values for A: a,b possible values for B: c,d Entropy([29+,35-]) = -29/64 log 2 29/64 – 35/64 log 2 35/64 = 0.99 A?
Background image of page 2
Image of page 3
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}