Unsupervised Learning
Unsupervised Learning
So far, we have only looked at supervised learning, in
which an external teacher improves network
performance by comparing desired and actual outputs
and mo
12/9/2010
A Hierarchical Neural Model for the Detection of
Motion Patterns in Optical Flow Fields
Data Flow Diagram
of Visual Areas in
Macaque Brain
Blue:
motion perception
pathway
Overview
1. The Str
12/8/2010
Adaptive Resonance Theory (ART)
Adaptive Resonance Theory (ART)
Adaptive Resonance Theory (ART) networks perform
completely unsupervised learning.
Their competitive learning algorithm is sim
Exemplar Analysis
Ensuring Coverage
When building a neural network application, we must
make sure that we choose an appropriate set of
exemplars (training data):
The entire problem space must be cove
K-Class Classification Problem
K-Class Classification Problem
Let us denote the kk-th class by Ck, with nk exemplars or
training samples, forming the sets Tk for k = 1, , K:
cfw_(
)
Tk = x kp , d kp |
Associative Networks
Interpolative Associative Memory
Associative networks are able to store a set of
patterns and output the one that is associated with the
current input.
For heterohetero-associatio
Adaptive Networks
Adaptive Networks
As you know, there is no equation that would tell you
the ideal number of neurons in a multimulti-layer network.
So far, we have determined the number of hiddenhidd
About Assignment #3
About Assignment #3
Two approaches to backpropagation learning:
1. PerPer-pattern learning:
Update weights after every exemplar presentation.
2. PerPer
Per-epoch
epoch (batch
(batc
Creating Data Representations
Creating Data Representations
On the other hand, sets of orthogonal vectors (such
as 100, 010, 001) can be processed by the network
more easily.
This becomes clear when w
Radial Basis Functions
Radial Basis Functions
If we are using such linear interpolation, then our
radial basis function (RBF) 0 that weights an input
vector based on its distance to a neurons referenc
Cascade Correlation
Cascade Correlation
Weights to each new hidden node are trained to
maximize the covariance of the nodes output with the
current network error.
wi =
Covariance:
K
P
S(wnew ) = ( xn
Example I: Predicting the Weather
Example I: Predicting the Weather
We decide (or experimentally determine) to use a
hidden layer with 42 sigmoidal neurons.
The next thing we need to do is collecting
The Hopfield Network
Asynchronous Hopfield Network
The nodes of a Hopfield network can be updated
synchronously or asynchronously.
Current network state O, attractors (stored patterns)
X and Y:
Synchr
Assignment #3 Question 2
Assignment #3 Question 2
Regarding your cascade correlation projects, here are
a few tips to make your life easier.
This will require more training, but it may find a better
(
Sigmoidal Neurons
f i (net i (t ) =
Sigmoidal Neurons
1
1 + e ( net i ( t ) ) /
fi(neti(t)
1
This leads to a simplified form of the sigmoid function:
S (net ) =
= 0.1
=1
We do not need a modifiable t
Adaline Schematic
The Adaline uses gradient descent to determine the
weight vector that leads to minimal error.
i1
i2
in
The Adaline Learning Algorithm
w0 + w1i1 + + wnin
Adjust
weights
Output
The ide
Capabilities of Threshold Neurons
Capabilities of Threshold Neurons
By choosing appropriate weights wi and threshold
we can place the line dividing the input space into
regions of output 0 and output
Visual Illusions
Visual Illusions
Visual Illusions demonstrate how we perceive an interpreted version of
the incoming light pattern rather that the exact pattern itself.
September 14, 2010
Neural Netw
Supervised Function Approximation
Supervised Function Approximation
In supervised learning, we train an ANN with a set of
vector pairs, soso-called exemplars
exemplars.
Each pair (x
(x, y) consists of
Refresher: Perceptron Training Algorithm
Algorithm Perceptron
Perceptron;
Another Refresher: Linear Algebra
How can we visualize a straight line defined by an
equation such as w0 + w1i1 + w2i2 = 0?
St
CS 672 Neural Networks Fall 2010
Instructor: Marc Pomplun
Assignment #1
Posted on September 26 due by October 7 at 4pm
Question 1: Perceptron Learning
The chart below shows a set of two-dimensional in
CS 672 Neural Networks Fall 2010
Instructor: Marc Pomplun
Assignment #3
Posted on November 7
Due by November 23 (Question 1)
and December 9 (Question 2)
Question 1: A Backpropagation Network for Digit
CS 672 Neural Networks Fall 2010
Instructor: Marc Pomplun
Assignment #2
Posted on October 10 due by October 19, 4pm
Question 1: A Perceptron Network for Digit Recognition
There is an excellent online
CS 672 Neural Networks Fall 2010
Instructor: Marc Pomplun
Practice Midterm Exam
Duration: 75 minutes
No calculators, no books, and no notes are allowed (in the actual exam).
Question 1: _ out of _ poi
CS 672 Neural Networks Fall 2010
Instructor: Marc Pomplun
Practice Exam
Duration: 2:30 hours
No calculators, no books, and no notes are allowed (in the actual exam).
Question 1: _ out of _ points
Ques
CS 672 Neural Networks Fall 2010
Instructor: Marc Pomplun
Practice Exam Sample Solutions
Duration: 2:30 hours
No calculators, no books, and no notes are allowed (in the actual exam).
Question 1: _ out
CS 672 Neural Networks Fall 2010
Instructor: Marc Pomplun
Assignment #4
Sample Solutions
Question 2: A Hopfield Network for Beginners
(a) Determine the weight matrix for an auto-associative, discrete
CS 672 Neural Networks Fall 2010
Instructor: Marc Pomplun
Assignment #1 Sample Solutions
Question1:
a) w0 = 0.2, w1 = 1, w2 = -1
Let = 0.5.
The input coordinates:
For Class 1:
cfw_(0.08,0.72), (0.26,0