Homework3-solutions

Homework3-solutions - Neural Networks Homework 3 Due 2...

This preview shows pages 1–2. Sign up to view the full content.

Neural Networks Homework 3, Due 2 February 2011 1. On page 148-149, Haykin discusses how to select initial weights. He notes that LeCun suggested the standard deviation of the weights for all syanpses coming into a particular node j , namely u w j , should be chosen such that u w j = m j m 1 / 2 where m j is the number of input synapses to node j . Find the region [ -k,k ] such that the uniform distribution over this region has standard deviation m j m 1 / 2 . To solve this, first note that σ 2 = E [ ( x mμ) 2 ] = m∞ p ( x )( x mμ) 2 dx and that for the uniform distribution on interval [- k , k ], this gives m k k x 2 2 k dx = x 3 6 k U m k k = k 3 6 k ± k 3 6 k = k 2 3 , thus σ= k 3 . So if we want to satisfy u w j = m j m 1 / 2 , we must have k ² 3 = m j m 1 / 2 , or k = ² 3 m j m 1 / 2 and our interval is [m ² 3 m j m 1 / 2 , ² 3 m j m 1 / 2 ] . 2. Consider the following MLP discussed in class. Use back propagation to find updated values for weights

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

Page1 / 3

Homework3-solutions - Neural Networks Homework 3 Due 2...

This preview shows document pages 1 - 2. Sign up to view the full document.

View Full Document
Ask a homework question - tutors are online