1
CS 464: Introduction to
Machine Learning
Concept Learning
Slides adapted from Chapter 2,
Machine Learning
by Tom M. Mitchell
http://www-2.cs.cmu.edu/afs/cs.cmu.edu/user/mitchell/ftp/mlbook.html
2
Concept Learning
c
Acquiring the definition of a general
category from given sample positive and
negative training examples of the category.
c
An example:
learning
“bird”
concept from
the given examples of birds (positive
examples) and non-birds (negative
examples).
c
Inferring
a boolean-valued function
from
training examples of
its input and output.
3
Classification (Categorization)
•
Given
–
A fixed set of categories:
C=
{
c
1
,
c
2
,…
c
k
}
–
A description of an instance
x = (x
1
, x
2
,…x
n
)
with
n
features,
x
∈
X
, where
X
is the
instance space
.
•
Determine:
–
The category of
x
:
c
(
x
)
∈
C,
where
c
(
x
) is a
categorization function whose domain is
X
and whose
range is
C
.
–
If
c
(
x
) is a binary function
C
={1,0} ({true,false},
{positive, negative}) then it is called a
concept
.
4
Learning for Categorization
•
A training example is an instance
x
∈
X,
paired with its correct category
c
(
x
):
<
x
,
c
(
x
)> for an unknown categorization
function,
c
.
•
Given a set of training examples,
D
.
•
Find a hypothesized categorization function,
h
(
x
), such that:
)
(
)
(
:
)
(
,
x
c
x
h
D
x
c
x
=
∈
>
<
∀
Consistency
5
A Concept Learning Task – Enjoy Sport
Training Examples
Example
Sky
AirTemp Humidity Wind
Water
Forecast EnjoySport
1
Sunny
Warm
Normal
Strong
Warm
Same
YES
2
Sunny
Warm
High
Strong
Warm
Same
YES
3
Rainy
Cold
High
Strong
Warm
Change
-O
4
Sunny
Warm
High
Strong
Warm
Change
YES
• A set of example days, and each is described by six attributes.
• The task is to learn to predict the value of EnjoySport for arbitrary day,
based on the values of its attribute values.
ATTRIBUTES
CO-CEPT
6
Hypothesis Space
•
Restrict learned functions to a given
hypothesis space
,
H
,
of functions
h
(
x
) that can be considered as definitions of
c
(
x
).
•
For learning concepts on instances described by
n
discrete-
valued features, consider the space of conjunctive
hypotheses represented by a vector of
n
constraints
<
c
1
,
c
2
, …
c
n
> where each
c
i
is either:
–
?, a wild card indicating no constraint on that feature
–
A specific value from the domain of that feature
–
Ø indicating no value is acceptable
•
Sample conjunctive hypotheses are
–
<Sunny, ?,
?, Strong, ?, ?>
–
<?, ?, ?, ?, ?, ?> (
most general hypothesis
)
–
< Ø, Ø, Ø,
Ø, Ø, Ø> (
most specific hypothesis
)