This preview shows pages 1–3. Sign up to view the full content.
This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Unformatted text preview: CS446: Pattern Recognition and Machine Learning Fall 2009 Problem Set 1 Handed Out: September 1, 2009 Due: September 10, 2009 Feel free to talk to other members of the class in doing the homework. I am more concerned that you learn how to solve the problem than that you demonstrate that you solved it entirely on your own. You should, however, write down your solution yourself. Please try to keep the solution brief and clear. Feel free to send me email or come to ask questions. Please, no handwritten solutions. Please present your algorithms in both pseudocode and English. That is, give a precise formulation of your algorithm as pseudocode and also explain in one or two concise paragraphs what your algorithm does. Be aware that pseudocode is much simpler and more abstract than real code. Take a look at the textbook pseudocode (e.g. Table 2.5 on page 33) to get an idea about the appropriate level of abstraction. The homework is due at 4:00 pm on the due date. Please email an electronic copy of your writeup and any code to the TA: mchang21@uiuc.edu. 1. [Learning as a Search  20 points] (Based on Mitchell, exercise 2.4) Let X be an instance space consisting of all of the points ( x, y ) in the plane with integer (not real) coordinates.108642 2 4 6 8 10108642 2 4 6 8 10 Let H be a set of hypotheses consisting of all the origin centered bagels. Formally, hypotheses are of the form ( a < radicalbig x 2 + y 2 < b ), where a < b , and a, b Z (the set of nonnegative integers). a. Consider the version space with respect to the set of positive (+) and negative ( ) training examples shown above: + : (5 , 5) , ( 6 , 4) , ( 3 , 4) , (2 , 4) 1 : ( 1 , 2) , ( 2 , 0) , (6 , 7) , (8 , 8) What is the S boundary set of the version space in this case? Write out the hypotheses in the form given above and draw them in on the diagram. [5 points] b. What is the G boundary set of this version space? Write out the hypotheses and draw them in. [5 points] c. Suppose the learner may now suggest a new ( x, y ) instance and ask the trainer for its classification. Suggest a query guaranteed to reduce the size of the version space, regardless of how the trainer classifies it. Suggest one that will not reduce the size of the version space, regardless of how the trainer classifies it. [5 points] d. There are many other possible hypothesis spaces that can explain this data. Pro pose one alternate hypothesis space and explicitly define its parameters as we did with a, b in the bagel space above. Choose an instance from your hypothesis space that separates the given data. Write out this hypothesis and sketch it. What can you say about the number of parameters in your space compared to the bagel space and the given number of training points. [5 points] 2. [Learning Conjunction  40 points] (Based on Mitchell, exercise 2.9) Consider a learning problem where each instance is a Boolean vector over n variables ( x 1 , . . . , x n ) and is labeled either positive (1) or negative (1). Thus a typical instance) and is labeled either positive (1) or negative (1)....
View
Full
Document
This note was uploaded on 04/23/2010 for the course COMPUTER S cs 446 taught by Professor Ahuja during the Fall '08 term at University of Illinois at Urbana–Champaign.
 Fall '08
 ahuja
 Machine Learning

Click to edit the document details