1
CS 464: Introduction to
Machine Learning
Concept Learning
Slides adapted from Chapter 2,
Machine Learning
by Tom M. Mitchell
http://www2.cs.cmu.edu/afs/cs.cmu.edu/user/mitchell/ftp/mlbook.html
2
Concept Learning
circle6
Acquiring the definition of a general
category from given sample positive and
negative training examples of the category.
circle6
An example:
learning
“bird”
concept from
the given examples of birds (positive
examples) and nonbirds (negative
examples).
circle6
Inferring
a booleanvalued function
from
training examples of
its input and output.
3
Classification (Categorization)
•
Given
–
A fixed set of categories:
C=
{
c
1
,
c
2
,…
c
k
}
–
A description of an instance
x = (x
1
, x
2
,…x
n
)
with
n
features,
x
∈
X
, where
X
is the
instance space
.
•
Determine:
–
The category of
x
:
c
(
x
)
∈
C,
where
c
(
x
) is a
categorization function whose domain is
X
and whose
range is
C
.
–
If
c
(
x
) is a binary function
C
={1,0} ({true,false},
{positive, negative}) then it is called a
concept
.
4
Learning for Categorization
•
A training example is an instance
x
∈
X,
paired with its correct category
c
(
x
):
<
x
,
c
(
x
)> for an unknown categorization
function,
c
.
•
Given a set of training examples,
D
.
•
Find a hypothesized categorization function,
h
(
x
), such that:
)
(
)
(
:
)
(
,
x
c
x
h
D
x
c
x
=
∈
>
<
∀
Consistency
5
A Concept Learning Task – Enjoy Sport
Training Examples
Example
Sky
AirTemp
Humidity Wind
Water
Forecast
EnjoySport
1
Sunny
Warm
Normal
Strong
Warm
Same
YES
2
Sunny
Warm
High
Strong
Warm
Same
YES
3
Rainy
Cold
High
Strong
Warm
Change
glyph1197O
4
Sunny
Warm
High
Strong
Warm
Change
YES
• A set of example days, and each is described by six attributes.
• The task is to learn to predict the value of EnjoySport for arbitrary day,
based on the values of its attribute values.
ATTRIBUTES
COglyph1197CEPT
6
Hypothesis Space
•
Restrict learned functions to a given
hypothesis space
,
H
,
of functions
h
(
x
) that can be considered as definitions of
c
(
x
).
•
For learning concepts on instances described by
n
discrete
valued features, consider the space of conjunctive
hypotheses represented by a vector of
n
constraints
<
c
1
,
c
2
, …
c
n
> where each
c
i
is either:
–
?, a wild card indicating no constraint on that feature
–
A specific value from the domain of that feature
–
Ø indicating no value is acceptable
•
Sample conjunctive hypotheses are
–
<Sunny, ?,
?, Strong, ?, ?>
–
<?, ?, ?, ?, ?, ?> (
most general hypothesis
)
–
< Ø, Ø, Ø,
Ø, Ø, Ø> (
most specific hypothesis
)
This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
7
EnjoySport Concept Learning Task
Given
circle6
Instances X
: set of all possible days, each described by the attributes
circle6
Sky – (values: Sunny, Cloudy, Rainy)
circle6
AirTemp – (values: Warm, Cold)
circle6
Humidity – (values: Normal, High)
circle6
Wind – (values: Strong, Weak)
circle6
Water – (values: Warm, Cold)
circle6
Forecast – (values: Same, Change)
circle6
Target Concept (Function) c
:
EnjoySport :
X
barb2right
{0,1}
circle6
Hypotheses H
: Each hypothesis is described by a conjunction of constraints
on the attributes.
Example hypothesis: <Sunny, ?,
?, ?, ?, ?>
circle6
Training Examples D
This is the end of the preview.
Sign up
to
access the rest of the document.
 Spring '11
 NoProfessor
 Machine Learning, version space, Tom M. Mitchell, algo., EnjoySport

Click to edit the document details