Pg 4863 – Constructive Learning/Cascade Correlation
•
Cascade problem fixes many problems apparent in BP – it was invented to
solve 2 problems – learning speed an ability to learn difficulty problems that
BP could not solve
Curvature and stepsize problem
•
CC deals with step size problem in weight adjustment by using information
on curvature and slope or the error surface. Slope is rate of change of error
with respect to weight
•
Curvature taken as ratio of difference in slope of two points to change in
weight of two points
•
Use of this deals with step size problem, ensuring that weight changes in cc
can be larger and moe decisive, leading to faster learning than standard BP
Unit recruitment and the moving target problem
•
Another difficulty with learning speed is the moving target problem –
tendency for hidden units to all run off in the same direction at once (herd
effect) before finding their useful role in learning the reduce error
•
Solution is to allow only one hidden unit to evolve at a time – done by starting
with 0 hidden units and then recruiting only one hidden unit at a time, as
This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
This is the end of the preview.
Sign up
to
access the rest of the document.
 Fall '10
 Shultz
 Input/output, Toshiko Sato, BP Unit

Click to edit the document details