CS573: Homework 3 Solution
Part of the solution is compiled from Jaewoo L.’s and Pedro P.’s submissions
December 11, 2010
Tree Augmented Naive Bayes
In this assignment we investigate a graphical model that extends Naive Bayes (NB) for
classification.
NB uses a generative model which assumes conditional independence be
tween the attributes (
X
=
{
X
1
, ..., X
n
}
) given the class (
C
). This can be represented using
a directed graphical model where the nodes are
V
=
{
X
i

1
≤
i
≤
n
} ∪ {
C
}
and the edges
are
E
=
{
(
C, X
i
)

1
≤
i
≤
n
}
. Tree Augmented Naive Bayes (TAN) augments this graphical
model with a set of edges
E
0
⊂
X
×
X
. The restriction on
E
0
is that every
X
i
has exactly
one parent from
X
(in addition to
C
), except for one
X
i
that has no parents other than
C
. Figure 1 gives a general description of Naive Bayes versus TAN.
To estimate a TAN model, the learning algorithm needs to search over the structure of
the model (i.e., which edges to add among the attributes) and then estimate the parameters
of the conditional probability distributions (CPDs). Learning the structure of TAN model
requires a procedure that consists of five main steps:
1. Compute
I
P
(
X
i
;
X
j

C
) between each pair of attributes,
i
6
=
j
, where
I
P
(
X
;
Y

z
) =
X
x,y,z
P
(
x, y, z
)
log
P
(
x, y

z
)
P
(
x

z
)
P
(
y

z
)
2. Build a complete undirected graph in which the vertices are the attributes
X
1
,
....
, X
n
.
Annotate the weight of an edge connecting
X
i
to
X
j
by
I
P
(
X
i
;
X
j

C
).
This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
This is the end of the preview.
Sign up
to
access the rest of the document.
 Fall '08
 Staff
 Laplace, Maximum likelihood, Estimation theory, Likelihood function

Click to edit the document details