Principal Component
Analysis
!
!
Lecture 11
!
1
!
Eigenvectors and Eigenvalues!
g
Consider this problem of spreading butter on
a bread slice!
2
!
Eigenvectors and Eigenvalues!
g
Consider this problem of stretching cheese
on a bread slice!
3
!
Eigenvectors
Synapses & Dendritic
Computation
!
!
Lecture 8
!
1
!
Neural Communication!
g
Synapse!
Communication of information between neurons is
accomplished by movement of chemicals across a
small gap called the synapse"
n Synapses play a role in all the operations
Parameter!
Estimation
!
!
Lecture 16
!
1
!
Maximum Likelihood Estimation!
g
Suppose we consider estimating a density function p(x) which
depends on a number of parameters =[1, 2,M]T"
n
n
g
For a Gaussian pdf 1=, 2=2 and p(x)=N(, 2)%
To make the dependence
Classication
Algorithms
!
!
Lecture 17
!
1
!
Probability Theory!
Apples and Oranges
!
(Red Box)!
2 Apples!
6 Oranges!
From Bishop, PRML!
(Blue Box)!
3 Apples!
1 Orange!
Pick red box!
(40%)!
Pick blue box!
(60%)!
any piece of fruit in the boxes is equally
Independent
!
Component Analysis
!
!
Lecture 14
!
1
!
ICA: Motivation!
g
Cocktail party problem:!
Imagine you are in a room where two people are
speaking simultaneously. "
n You have two microphones placed in two different
locations."
n
g
Microphones will
Dimensionality
Reduction Algorithms
!
!
Lecture 15
!
1
!
High-dimensional Spaces!
g
Counter-intuitive properties!
n
Volume of a d dimensional sphere"
of radius r"
"
"
2" d / 2 r d
Vd (r) =
$ d'
d#& )
% 2(
where () is the gamma function. "
!
2
!
High-dimen
Multilayer Perceptrons
!
!
Lecture 11
!
1
!
Chain Rule!
g
Lets say we have two functions f(x) and g(x)"
f (x) = x 5
g(x) = (x 2 +1)
f (g(x) = (x 2 +1) 5
g
What is the derivative of f(g(x)?"
!
df (x)
= 5x 4
dx
df (g)
= 5g 4
dg
dg(x)
= 2x
dx
df (g(x) df (g)
Phenomenological
Models of Neurons
!
!
Lecture 5
!
1
!
Some Linear Algebra First!
Notes from Eero Simoncelli
2
!
Vector Addition!
Notes from Eero Simoncelli
3
!
Scalar Multiplication of a Vector!
4
!
Vector Norm!
5
!
Unit Vector!
6
!
Inner Product of Vect
Competitive Learning
!
!
Lecture 10
!
1
!
Competitive Learning!
g
A form of unsupervised training where output units are said to
be in competition for input patterns!
n
n
n
During training, the output unit that provides the highest activation to a
given i
Articial Neural
Networks
!
!
Lecture 9
!
1
!
Some History!
2
!
Some History!
3
!
Some History!
4
!
Some History!
5
!
Some History!
6
!
Perceptron!
g
Perceptron is a algorithm for learning linear
threshold function that takes an input X =
[x1,x2, , xd]T an