Course Hero - We put you ahead of the curve!

You have requested the below document.

Coursehero >> Maryland >> UMBC >> CS 671

Learning Concept and Version Spaces Based Ch.2 of Tom Mitchell s Machine Learning and lecture slides by Uffe Kjaerulff 1 Presentation Overview Concept learning as boolean function approximation Ordering of hypothesis Version spaces and candidateelimination algorithm The role of bias 2 A Concept Learning Task 1 2 3 4 Inferring boolean-valued functions from training examples; Inductive learning. Sky Sunny Sunny Rainy Sunny AirTemp Warm Warm Cold Warm Humidity Normal High High High Wind Strong Strong Strong Strong Water Warm Warm Warm Cool Forecast Same Same Change Change EnjoySport Yes Yes No Yes Example Example Given: Instances X: Possible days described by Sky, AirTemp, Humidity, Wind, Water, Forecast; Target concept c: Enjoy-Sport: Day {Yes,No}; Hypothesis H: described by a conjunction of attributes, e.g. Water=Warm Sky=Sunny; Training examples D: positive and negative examples of target function, <x1, c(x1), , xm, c(xm)>. 3 Determine: A hypothesis h from H such that h(x)=c(x) for all x in X. The Inductive Learning Hypothesis Note: the only information available about c is c(x) for each <x, c(x)> in D. Any hypothesis found to approximate the target function well over a sufficiently large set of training examples will also approximate the target function well over other observed example. 4 Concept Learning as Search Some notation for hypothesis representation: ? means that any value is acceptable as an attribute; 0 means that no value is acceptable. In our example Sky {Sunny, Cloudy, Rainy}; AirTemp {Warm, Cold}; Humidity {Normal, High}; Wind {Strong, Weak}; Water {Warm, Cold}; Forecast {Same, Change}. The instance space contains 3*2*2*2*2*2=96 distinct instances. The hypothesis space contains 5*4*4*4*4*4=5120 syntactically distinct hypothesis More realistic learning tasks contain much larger H. Efficient strategies are crucial. 5 More-General-Than Let hj and hk be boolean functions over X, then More-General-Than-Or-Equal(hj,hk) ( x X) [hk(x) Establishes partial order on the hypothesis space. P6.ps hj(x)] 6 Find-S Algorithm Initialize h to the most specific hypothesis in H; For each positive training instance x For each attribute ai in h If the constraint ai in h is not satisfied by x then replace ai in h by the most general constraint that is satisfied by x Output hypothesis h. Note: Assume that H contains c and that D contains no errors; Otherwise this technique does not work. Limitations: Can t tell if it s learned the concept: Other consistent hypothesis? Fails if training data is inconsistent; Picks maximally specific h; Depending on H there might be several. 7 Version Spaces A hypothesis h is consistent with a set of training examples D of target concept if and only if h(x)=c(x) for each training example <x, c(x)> in D: Consistent(h,D) ( <x, c(x)> D) [ h(x) = c(x) ] A version space VSH,D wrt H and D is the subset of hypothesis from H consistent with all training examples in D: VSH,D { h H: Consistent(h, D) } 8 The List-Then-Eliminate Algorithm VersionSpace a list containing every hypothesis in H; For each training example <x, c(x)> in D Remove from VersionSpace any h for which h(x) c(x) Output the list of hypothesis. Maintains a list of all hypothesis in VSH,D. Unrealistic for most H. More compact (regular) representation of VSH,D is needed. 9 Example Version Space Idea: VSH,D can be represented by the set of most general and most specific consistent hypothesis. P9.ps 10 Representing Version Spaces The general boundary G of version space VSH,D is the set of its most general members. The specific boundary S of version space VSH,D is set the of its most specific members. Version Space Representation Theorem Let X be an arbitrary set of instances and let H be a set of boolean-valued hypothesis defined over X. Let c: X {0,1} be an arbitrary target concept defined over X, and let D be an arbitrary set of training examples {<x, c(x)>}. For all X, H, c, and D such that S and G are well defined VSH,D { h HsSg Gg h s }. 11 Candidate-Elimination Algorithm G maximally general hypothesis in H S maximally specific hypothesis in H For each training example d If d is a positive example Remove from G any hypothesis that does not cover d For each hypothesis s in S that does not cover d Remove s from S Add to S all minimal generalizations h of s such that h covers d and some member of G is more general than h Remove from S any hypothesis that is more general than another hypothesis in S If d is a negative example Remove from S any hypothesis that covers d For each hypothesis g in G that covers d Remove g from G Add to G all minimal specializations h of g such that h does not cover d and some member of S is more specific than h Remove from G any hypothesis that is more specific than another 12 hypothesis in G Some Notes on CandidateElimination Algorithm Positive examples make S become increasingly general. Negative examples make G become increasingly specific. Candidate-Elimination algorithm will converge toward the hypothesis that correctly describes the target concept provided that There are no errors in the training example; There is some hypothesis in H that correctly describes the target concept. The target concept is exactly learned when the S and G boundary sets converge to a single identical hypothesis. Under the above assumptions, new training data can be used to resolve ambiguity. The algorithm beaks down if the data is noisy(inconsistent); Inconsistency can be eventually detected given sufficient training data is given: S and G converge to an empty version space. The target concept is a disjunction of feature attributes. 13 A Biased Hypothesis Space Bias: Each h H given by a conjunction of attribute values Unable to represent disjunctive concepts: Sky=Sunny Sky=Cloudy Example 2 3 4 Sky Sunny Cloudy Rainy AirTemp Warm Warm Warm Humidity Normal Normal Normal Wind Strong Strong Strong Water Cool Cool Cool Forecast Change Change Change EnjoySport Yes Yes No Most specific hypothesis consistent with 1 and 2 and representable in H is (?,Warm, Normal, Strong, Cool, Change). But it is too general: Covers 3. 14 Unbiased Learner Idea: Choose H that expresses every teachable concept; H is is a power set of X; Allow disjunction and negation. For our example we get 296 possible hypothesis. What is G and S? S becomes a disjunction of positive examples; G becomes a negated disjunction of negative examples. Only training examples will be unambiguously classified. The algorithm cannot generalize! 15 Inductive Bias Let L be a concept learning algorithm; X be a set instances; c be the target concept; Dc={<x, c(x)>} be the set of training examples; L(xi,Dc) denote the classification assigned to the instance xi by L after training on Dc. The inductive bias of L is any minimal set of assertions B such that for the target concept c and corresponding training examples Dc: xi X: (B Dc xi) L(xi, Dc) Inductive bias of Candidate-Elimination algorithm: The target concept c is contained in the given hypothesis space H. 16 Summary Points Concept learning as search through H Partial ordering of H Version space candidate elimination algorithm S and G characterize learner s uncertainty Inductive leaps are possible only if the learner is biased 17

Find millions of documents here - Study Guides, Homework Solutions, Papers, Exam Answer Keys and more. Course Hero has millions of course related materials that will enable you to learn better, faster and get an A in all your courses.
Below is a small sample set of documents:

k-nn.ppt
Path: UMBC >> CS >> 671 Fall, 2001

backprop.ppt
Path: UMBC >> CS >> 671 Fall, 2001
cascade-corr.ppt
Path: UMBC >> CS >> 671 Fall, 2001
amtimss.pdf
Path: Southeastern Bible >> TIMSSANDPI >> 1995 Fall, 2008
amtimss.pdf
Path: BC >> TIMSSANDPI >> 1995 Fall, 2008
amtoc.pdf
Path: Southeastern Bible >> TIMSSANDPI >> 1995 Fall, 2008
amtoc.pdf
Path: BC >> TIMSSANDPI >> 1995 Fall, 2008
amexsum.pdf
Path: Southeastern Bible >> TIMSSANDPI >> 1995 Fall, 2008
amexsum.pdf
Path: BC >> TIMSSANDPI >> 1995 Fall, 2008
amintro.pdf
Path: Southeastern Bible >> TIMSSANDPI >> 1995 Fall, 2008
amintro.pdf
Path: BC >> TIMSSANDPI >> 1995 Fall, 2008
am1chap.pdf
Path: Southeastern Bible >> TIMSSANDPI >> 1995 Fall, 2008
am1chap.pdf
Path: BC >> TIMSSANDPI >> 1995 Fall, 2008
am2chap.pdf
Path: Southeastern Bible >> TIMSSANDPI >> 1995 Fall, 2008
am2chap.pdf
Path: BC >> TIMSSANDPI >> 1995 Fall, 2008
am3chap.pdf
Path: Southeastern Bible >> TIMSSANDPI >> 1995 Fall, 2008
am3chap.pdf
Path: BC >> TIMSSANDPI >> 1995 Fall, 2008
am4chap.pdf
Path: Southeastern Bible >> TIMSSANDPI >> 1995 Fall, 2008
am4chap.pdf
Path: BC >> TIMSSANDPI >> 1995 Fall, 2008
am5chap.pdf
Path: Southeastern Bible >> TIMSSANDPI >> 1995 Fall, 2008
am5chap.pdf
Path: BC >> TIMSSANDPI >> 1995 Fall, 2008
amAapx.pdf
Path: Southeastern Bible >> TIMSSANDPI >> 1995 Fall, 2008
amAapx.pdf
Path: BC >> TIMSSANDPI >> 1995 Fall, 2008
amBapx.pdf
Path: Southeastern Bible >> TIMSSANDPI >> 1995 Fall, 2008
amBapx.pdf
Path: BC >> TIMSSANDPI >> 1995 Fall, 2008
amCapx.pdf
Path: Southeastern Bible >> TIMSSANDPI >> 1995 Fall, 2008
amCapx.pdf
Path: BC >> TIMSSANDPI >> 1995 Fall, 2008
amDapx.pdf
Path: Southeastern Bible >> TIMSSANDPI >> 1995 Fall, 2008
amDapx.pdf
Path: BC >> TIMSSANDPI >> 1995 Fall, 2008
BMathAll.pdf
Path: Southeastern Bible >> TIMSSANDPI >> 1995 Fall, 2008
BMathAll.pdf
Path: BC >> TIMSSANDPI >> 1995 Fall, 2008
BMathTOC.pdf
Path: Southeastern Bible >> TIMSSANDPI >> 1995 Fall, 2008
BMathTOC.pdf
Path: BC >> TIMSSANDPI >> 1995 Fall, 2008
BMathExc.pdf
Path: Southeastern Bible >> TIMSSANDPI >> 1995 Fall, 2008
BMathExc.pdf
Path: BC >> TIMSSANDPI >> 1995 Fall, 2008
BMathInt.pdf
Path: Southeastern Bible >> TIMSSANDPI >> 1995 Fall, 2008
BMathInt.pdf
Path: BC >> TIMSSANDPI >> 1995 Fall, 2008
BMathCh1.pdf
Path: Southeastern Bible >> TIMSSANDPI >> 1995 Fall, 2008
BMathCh1.pdf
Path: BC >> TIMSSANDPI >> 1995 Fall, 2008
BMathCh2.pdf
Path: Southeastern Bible >> TIMSSANDPI >> 1995 Fall, 2008
BMathCh2.pdf
Path: BC >> TIMSSANDPI >> 1995 Fall, 2008
BMathCh3.pdf
Path: Southeastern Bible >> TIMSSANDPI >> 1995 Fall, 2008
BMathCh3.pdf
Path: BC >> TIMSSANDPI >> 1995 Fall, 2008
BMathCh4.pdf
Path: Southeastern Bible >> TIMSSANDPI >> 1995 Fall, 2008
BMathCh4.pdf
Path: BC >> TIMSSANDPI >> 1995 Fall, 2008
BMathCh5.pdf
Path: Southeastern Bible >> TIMSSANDPI >> 1995 Fall, 2008
BMathCh5.pdf
Path: BC >> TIMSSANDPI >> 1995 Fall, 2008
BMathApA.pdf
Path: Southeastern Bible >> TIMSSANDPI >> 1995 Fall, 2008
BMathApA.pdf
Path: BC >> TIMSSANDPI >> 1995 Fall, 2008
BMathApB.pdf
Path: Southeastern Bible >> TIMSSANDPI >> 1995 Fall, 2008
BMathApB.pdf
Path: BC >> TIMSSANDPI >> 1995 Fall, 2008
BMathApC.pdf
Path: Southeastern Bible >> TIMSSANDPI >> 1995 Fall, 2008
BMathApC.pdf
Path: BC >> TIMSSANDPI >> 1995 Fall, 2008
BMathApD.pdf
Path: Southeastern Bible >> TIMSSANDPI >> 1995 Fall, 2008
BMathApD.pdf
Path: BC >> TIMSSANDPI >> 1995 Fall, 2008
BMathApE.pdf
Path: Southeastern Bible >> TIMSSANDPI >> 1995 Fall, 2008
BMathApE.pdf
Path: BC >> TIMSSANDPI >> 1995 Fall, 2008
BMathApF.pdf
Path: Southeastern Bible >> TIMSSANDPI >> 1995 Fall, 2008
BMathApF.pdf
Path: BC >> TIMSSANDPI >> 1995 Fall, 2008
t95_gender_all.pdf
Path: Southeastern Bible >> TIMSSANDPI >> 1995 Fall, 2008
t95_gender_all.pdf
Path: BC >> TIMSSANDPI >> 1995 Fall, 2008
T95_GExecSumm.pdf
Path: Southeastern Bible >> TIMSSANDPI >> 1995 Fall, 2008
T95_GExecSumm.pdf
Path: BC >> TIMSSANDPI >> 1995 Fall, 2008
T95_GChapter 1.pdf
Path: Southeastern Bible >> TIMSSANDPI >> 1995 Fall, 2008
T95_GChapter 1.pdf
Path: BC >> TIMSSANDPI >> 1995 Fall, 2008
T95_GChapter 2.pdf
Path: Southeastern Bible >> TIMSSANDPI >> 1995 Fall, 2008
T95_GChapter 2.pdf
Path: BC >> TIMSSANDPI >> 1995 Fall, 2008
T95_GChapter 3.pdf
Path: Southeastern Bible >> TIMSSANDPI >> 1995 Fall, 2008
T95_GChapter 3.pdf
Path: BC >> TIMSSANDPI >> 1995 Fall, 2008
T95_GChapter 4.pdf
Path: Southeastern Bible >> TIMSSANDPI >> 1995 Fall, 2008