lecture_14 - 2.160 System Identification, Estimation, and...

Info iconThis preview shows pages 1–3. Sign up to view the full content.

View Full Document Right Arrow Icon
2.160 System Identification, Estimation, and Learning Lecture Notes No. 1 4 April 10, 2006 8.4 The Error Back Propagation Algorithm The Multi-Layer Perception is a universal approximation function that can approximate an arbitrary (measurable) function to any accuracy. Unit i z i i g Unit j ) ( ) 1 ( m i m i x y = i j w z j g j Unit 1 Layer m Layer 2 Unit 4 Layer 1 Layer 0 Unit 3 Unit 2 Layer M Input Layer Output Layer Hidden Layers = ) ( m ji w weight of the connection from unit i to unit j in layer m = ) ( m j y output from unit j in layer m = ) ( m i x input to a unit in layer m from unit i Forward computation ) ( ) ( ) ( m i i m ji m j x w z = (19) ) 1 ( ) ( ) ( ) ( + = = m j m j j m j x z g y (20) m = 0, 1, 2, … 1
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Starting from m = 0, all the units can be computed recursively until m = M , output layer. How do we train the multi-layer perceptron, given training data presented sequentially? Note: Multi-Layer Perceptrons with nonlinear activation functions, g ( z ), are nonlinear in parameters w . A single-layer neural net is essentially linear in w , although g ( z ) is nonlinear. If two consecutive layers have linear activation functions, they can be combined and replaced by a single layer network. To be able to deal with nonlinear problems, such as the XOR problem, we now focus on a multi-layer perceptron with nonlinear activation functions.
Background image of page 2
Image of page 3
This is the end of the preview. Sign up to access the rest of the document.

This note was uploaded on 02/27/2012 for the course MECHANICAL 2.160 taught by Professor Harryasada during the Spring '06 term at MIT.

Page1 / 7

lecture_14 - 2.160 System Identification, Estimation, and...

This preview shows document pages 1 - 3. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online