and Relaxation Hopfield Networks
Neural Networks
Neural Networks - Hopfield
Bibliography
Hopfield, J. J., "Neural networks and physical systems with emergent
collective computational abilities," Proceedings of the National Academy of
Sciences 79:2554-2558, 1982.
Hopfield, J. J., "Neurons with graded response have collective
computational properties like those of two-state neurons." Proceedings of
the National Academy of Sciences 81: 3088-3092, 1984.
Abu-Mostafa, and J. St. Jacques, Information Capacity of the Hopfield
Model, IEEE Trans. on Information Theory, Vol. IT-31, No. 4, 1985.
Neural Networks - Hopfield
Neural Networks - Hopfield
Neural Networks - Hopfield
Neural Networks - Hopfield
Neural Networks - Hopfield
Neural Networks - Hopfield
Neural Networks - Hopfield
Neural Networks - Hopfield
Neural Networks - Hopfield
Limited by Lower order constraints
Has no hidden nodes, higher order units
All nodes visible
i.e. Program as CAM
000
011
101
110
However, auto-association relaxing allows a
garbled input to return a clean output
Assume two patterns trained
A -> X
B -> Y
Now enter the example with .6A and .4B
Result in a Backprop model?
Result in the Hopfield autoassociator: X
Neural Networks - Hopfield
Hopfield as a Computation Engine
Optimization
Travelling Salesman Problem (TSP)
NP-Complete
"Good" vs. Optimal Solutions
Very Fast Processing
Neural Networks - Hopfield
Neural Networks - Hopfield
Derive Energy equation for TSP
1. Legal State
2. Good State
Set weights accordingly
How would we do it
Neural Networks - Hopfield
Neural Networks - Hopfield
Neural Networks - Hopfield
Neural Networks - Hopfield
Summary
Much Current Work
Saturation and No Convergence
For Optimization, Saturation is moot
Many important Optimization problems
Non learning, but reasonably intuitive
programming - extensions to learning
Highly Parallel
Expensive Interconnect
Lots of Physical Implementation work, optics
Neural Networks - Hopfield