Another approach is to train some algorithm using a

Another approach is to train some algorithm
using a lot of labelled examples
Here are some pictures of lower-case R, and here are
some of lower-case N, you work it out!

38
Supervised learning
The trendiest supervised approach at the
moment is the artificial neural network (ANN)
Artificial Neural Networks are a software
construct loosely similar to the structure of the
human brain
They are biologically inspired , rather than actually
simulating real brains

39
Artificial Neural Networks
One of the earliest versions of ANN was called the
perceptron , developed in 1958
A perceptron contains one neuron, which has
many inputs but one output, and each input has
an importance, or weight
To calculate the output of a perceptron, the inputs
are multiplied by their weights, and if the weighted
sum is high enough, the neuron output is on

40
Single-neuron perceptron
So to learn, we find the set of weights that give
the correct output for each set of inputs
Sum input *
weights
Activation function
Activation In
Activation Out
Activation In

41
Perceptron learning
How do you find the right weights?
Start with any (random!) weights.
One by one, run examples through the perceptron and
change the weights until they no longer need changing
•
If the output is correct don’t change the weights
•
If the output is 0 and should be 1 then increase the
weight by a constant d
•
if the output is 1 and should be 0 then decrease the
weight by d

42
Perceptron learning
Perceptrons are able to learn some simple
input-output relationships
Any linearly separable relationships
Single perceptrons are unable to learn
functions like XOR
You can't draw a straight line to separate the 1 and 0
output areas

43
Linearly separable
Here are four points
representing the OR
relationship, X OR Y
Pink means the output
should be true for that
input
A perceptron can learn
this "function"
Y
X
On
Off
Off
On

44
Linearly separable
How about X AND Y ?
Y
X
On
Off
Off
On

45
Linearly separable
It's quite a bit harder
for XOR !
There's nowhere to
put a straight line that
will separate the
inputs that should give
true from those that
should give false
Y
X
On
Off
Off
On

Input layer
46
Networks of neurons
This limitation can be resolved by using
multiple layers of perceptrons, feeding into
each other
A
B
D
C
E
w
AC
w
CE
w
AD
w
BC
w
DE
Output layer Hidden layer

47
Artificial Neural Networks
These multi-layer perceptrons are more
commonly known as Artificial Neural Networks
They learn in more-or-less the same way, but
the errors have to propagate backwards
through the network layers

48
Neural Networks
O
O
O
O
O
10
20
30
35
40
45
50
55
60
65
70
80
O
O
O
O
O
O
O
O
O
O
O
O
O
O
O
O
O
O
O
O
O
O
O
far less than
less than
about the same as
more than
much more than
The husband needs ........ than the wife
O
O
O
O
O
90
The husband is likely to be
awarded ...... percentage of the assets
has few assets
is less than average
is about average
is wealthy
is very wealthy
The marriage...