{[ promptMessage ]}

Bookmark it

{[ promptMessage ]}

solutions2

# solutions2 - Assignment#2 Solutions(Chapter 4 3 Consider...

This preview shows pages 1–2. Sign up to view the full content.

Assignment#2 Solutions (Chapter 4) 3. Consider the training examples shown in Table 4.8 for a binary classification problem. a) What is the entropy of this collection of training examples with respect to the positive class? Answer: There are four positive examples and five negative examples. Thus, P (+) = 4 / 9 and P ( ) = 5 / 9. The entropy of the training examples is 4 / 9 log 2 (4 / 9) 5 / 9 log 2 (5 / 9) = 0 . 9911 . b) What are the information gains of a 1 and a 2 relative to these training examples? Answer: For attribute a 1 , the corresponding counts and probabilities are: a 1 + - T 3 1 F 1 4 The entropy for a 1 is given by: 4/9 [ -(3/4)log(3/4) (1/4)log(1/4)] + 5/9[ -(1/5)log(1/5) (4/5)log(4/5)] = 0.7616. Therefore, the information gain for a 1 is 0 . 9911 0 . 7616 = 0 . 2294. For attribute a 2 , the corresponding counts and probabilities are: A 2 + - T 2 3 F 2 2 The entropy for a 2 is given by: 5/9[ -(2/5)log(2/5) (3/5)log(3/5)] + 4/9 [-(2/4)log(2/4) (2/4)log(2/4)] = 0.9839. Therefore, the information gain for a 2 is 0 . 9911 0 . 9839 = 0 . 0072. c) For a 3 , which is a continuous attribute, compute the information gain for every possible split. Answer: a 3 Class label Split point Entropy Info Gain 1.0 + 2.0 0.8484 0.1427 3.0 - 3.5 0.9885 0.0026 4.0 + 4.5 0.9183 0.0728 5.0 - 5.0 - 5.5 0.9839 0.0072 6.0 + 6.5 0.9728 0.0183 7.0 + 7.0 - 7.5 0.8889 0.1022 The best split for a 3 occurs at split point equals to 2. d) What is the best split (among a 1 , a 2 , and a 3 ) according to the infor-mation gain? Answer: According to information gain, a 1 produces the best split. e) What is the best split (between a 1 and a 2 ) according to the classification error rate? Answer: For attribute a 1 , the error rate = 2 / 9. For attribute a 2 , the error rate = 4 / 9. Therefore, according to error rate, a 1 produces the best split.

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

### What students are saying

• As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

Kiran Temple University Fox School of Business ‘17, Course Hero Intern

• I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

Dana University of Pennsylvania ‘17, Course Hero Intern

• The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

Jill Tulane University ‘16, Course Hero Intern