Neural Networks
Homework 3, Due 2 February 2011
1.
On page 148149, Haykin discusses how to select initial weights. He notes that LeCun suggested
the standard deviation of the weights for all syanpses coming into a particular node
j
, namely
u
w
j
, should be chosen such that
u
w
j
=
m
j
m
1
/
2
where
m
j
is the number of input synapses to
node
j
.
Find the region [
k,k
] such that the uniform distribution over this region has standard deviation
m
j
m
1
/
2
.
To solve this, first note that
σ
2
=
E
[
(
x
mμ)
2
]
=
∫
m∞
∞
p
(
x
)(
x
mμ)
2
dx
and that for the uniform
distribution on interval [
k
,
k
], this gives
∫
m
k
k
x
2
2
k
dx
=
x
3
6
k
U
m
k
k
=
k
3
6
k
±
k
3
6
k
=
k
2
3
, thus
σ=
k
√
3
. So
if we want to satisfy
u
w
j
=
m
j
m
1
/
2
, we must have
k
²
3
=
m
j
m
1
/
2
, or
k
=
²
3
m
j
m
1
/
2
and our interval
is
[m
²
3
m
j
m
1
/
2
,
²
3
m
j
m
1
/
2
]
.
2.
Consider the following MLP discussed in class.
Use back propagation to find updated values for weights
This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
This is the end of the preview.
Sign up
to
access the rest of the document.
 Spring '08
 Staff
 Derivative, Normal Distribution, Neural Networks, neural network, momentum term

Click to edit the document details