Homework3-solutions

Homework3-solutions - Neural Networks Homework 3 Due 2...

Info iconThis preview shows pages 1–2. Sign up to view the full content.

View Full Document Right Arrow Icon
Neural Networks Homework 3, Due 2 February 2011 1. On page 148-149, Haykin discusses how to select initial weights. He notes that LeCun suggested the standard deviation of the weights for all syanpses coming into a particular node j , namely u w j , should be chosen such that u w j = m j m 1 / 2 where m j is the number of input synapses to node j . Find the region [ -k,k ] such that the uniform distribution over this region has standard deviation m j m 1 / 2 . To solve this, first note that σ 2 = E [ ( x mμ) 2 ] = m∞ p ( x )( x mμ) 2 dx and that for the uniform distribution on interval [- k , k ], this gives m k k x 2 2 k dx = x 3 6 k U m k k = k 3 6 k ± k 3 6 k = k 2 3 , thus σ= k 3 . So if we want to satisfy u w j = m j m 1 / 2 , we must have k ² 3 = m j m 1 / 2 , or k = ² 3 m j m 1 / 2 and our interval is [m ² 3 m j m 1 / 2 , ² 3 m j m 1 / 2 ] . 2. Consider the following MLP discussed in class. Use back propagation to find updated values for weights
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Image of page 2
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

Page1 / 3

Homework3-solutions - Neural Networks Homework 3 Due 2...

This preview shows document pages 1 - 2. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online