X
H
E
Y
pa
Y
B
A
X
pa
Y
X
H
E
X
Y
pa
B
A
X
pa
}
,
{
)
(
},
,
{
)
(
}
,
{
)
(
},
,
,
{
)
(
}
,
,
{
)
(
},
,
{
)
(
3
3
2
2
1
1
My method
Mainly two steps
Insertion and deletion steps
e
f
g
h
c
d
a
b
e
f
g
h
c
d
a
b
= 0.5
e
f
g
h
c
d
a
b
e
f
g
h
d
= 0.4
c
a
b
e
f
g
h
d
c
a
b
e
f
g
h
d
c
a
b
= 0.3
Continue up to t = 0.
…
5
T
6
T
7
T
1
0
T
…
…
Insertion step
Deletion step
My method
Advantage
Based on Biological facts (Scale Free Network)
No need of thresholds
Online approach
Scalability
Easy to explore subnetworks
Fast computation
My method
Disadvantage
Input order dependency
Risky in exploring parents in data with big noise values.
(It can be overfitted to training data)
61 % edges are
less order
dependent
(in part B)
Bayesian network with MCMC
Bayesian network
Problem 1
Problem 2
A
B
C
D
E
i
i
i
X
pa
X
P
E
D
C
B
A
P
))
(

(
)
,
,
,
,
(
)

(
)

(
)

(
)

(
)
(
B
E
P
B
D
P
A
C
P
A
B
P
A
P
)
(
)
(
)

(
)

(
D
P
M
P
M
D
P
D
M
P
M
M
P
M
D
P
M
P
M
D
P
)
(
)

(
)
(
)

(
Left: in large data set.
Right: in small data set.
Bayesian network with MCMC
MCMC (Markov Chain Monte Carlo)
Inference rule
for Bayesian Network
Sample from the posterior distribution
Proposal Move :
Given M_old, propose a new network M_new with proba
bility
Acceptance and Rejection :
)

(
old
new
M
M
Q
)

(
)

(
)
(
)

(
)
(
)

(
,
1
min
old
new
new
old
old
old
new
new
accept
M
M
Q
M
M
Q
M
P
M
D
P
M
P
M
D
P
P
Bayesian network with MCMC
MCMC in Bayes Net toolbox
Hasting factor
The proposal probability is calculated from the numb
er of neighbours of the model.
Improvement of MCMCs
Fanin
The sparse data leads the prior probability to have a
nonnegligible influence on the posterior
P
(
M

D
).
Limit the maximum number of edges converging on a
node, fanin.
If
FI
(
M
) > a, P(M)=0. Otherwise, P(M)=1.
The time complexity reduced largely
Acceptable configuration of child and parents in fanin 3
A
A
A
A
B
B
B
C
C
D
A
B
C
D
E
Improvement of MCMCs
DAG to CPDAG
(DAG : Directed Acyclic Graph, CPDAG : Completed Partially Directed Acyclic
Graph)
X
Y
X
Y
P(X, Y) = P(X)P(XY) = P(YP(YX)
Set of all equivalent DAGs
DAG to CPDAG
D
E
is reversible
others are compelled.
Improvement of MCMCs
This CPDAG concept bring several
advantages:
The space of equivalent classes is more reduced.
It is easy to trap in local optimum in moving DAG spaces.
Incorporating CPDAG to MCMC
MCMCMC
Trapping
A
B
A : global optima
B : local optima
Easy to be trapped in local optima B.
Multi chains with different temperatures will be useful to escape from it.
MCMCMC
Trapping
MCMCMC
A super chain, S
Acceptance ratios of a super chain
j
T
i
j
i
j
i
j
M
P
M
D
P
D
S
P
1
)
(
)
(
)
(
)

(
)

(
)

(
)

(
1
1
i
i
i
i
S
S
Q
S
S
Q
l
k
k
l
T
l
l
T
k
k
T
l
l
T
k
k
i
accept
M
P
M
D
P
M
P
M
D
P
M
P
M
D
P
M
P
M
D
P
P
1
1
1
1
)
(
)

(
)
(
)

(
)
(
)

(
)
(
)

(
,
1
min
Importance Sampling
.
You've reached the end of your free preview.
Want to read all 43 pages?
 Fall '19
 MCMC, Bayesian network, Directed acyclic graph