CS 478 Machine Learning: Homework 4
Suggested Solutions
1
The Na¨
ıve Bayes Independence Assumption
(a) For case (i)
P
(
X
= (0
,
1
,
1
,
1
,
1)

Y
= +1)
=
P
(
X
1
= 0
,X
2
= 1
,X
3
= 1
,X
4
= 1
,X
5
= 1

Y
= +1)
=
P
(
X
1
= 0
,X
2
= 1

Y
= +1)
=
P
(
X
1
= 0

Y
= +1)
P
(
X
2
= 1

Y
= +1)
= 0
.
2
×
0
.
6
= 0
.
12
(1)
For case (ii)
P
(
X
= (0
,
1
,
1
,
1
,
1)

Y
= +1)
=
P
(
X
1
= 0
,X
2
= 1
,X
3
= 1
,X
4
= 1
,X
5
= 1

Y
= +1)
=
P
(
X
1
= 0

Y
= +1)
P
(
X
2
= 1

Y
= +1)
P
(
X
3
= 1

Y
= +1)
P
(
X
4
= 1

Y
= +1)
P
(
X
5
= 1

Y
= +1)
= 0
.
2
×
0
.
6
4
= 0
.
02592
(2)
(b) Due to the Naive Bayes assumption, the multivaraite Naive Bayes estimate of the
probability is the same as the computation of case (ii) in part (a) above, and is
0.02592. So the two cases cannot be distinguished.
(c) For case (i), using the true probability, we have
P
(
X
= (0
,
1
,
1
,
1
,
1)
,Y
= +1)
=
P
(
X
= (0
,
1
,
1
,
1
,
1)

Y
= +1)
P
(
Y
= +1)
= 0
.
12
×
0
.
5
= 0
.
06
(3)
And
P
(
X
= (0
,
1
,
1
,
1
,
1)
,Y
=

1)
=
P
(
X
= (0
,
1
,
1
,
1
,
1)

Y
=

1)
P
(
Y
=

1)
= 0
.
8
×
0
.
4
×
0
.
5
= 0
.
16
(4)
Therefore by Bayes’ rule the example
X
= (0
,
1
,
1
,
1
,
1) will be classiﬁed as

1.
1