lecture6-annotated - Machine Learning 10-701/15-781 Fall...

Info iconThis preview shows pages 1–6. Sign up to view the full content.

View Full Document Right Arrow Icon
© Eric Xing @ CMU, 2006-2008 1 Machine Learning Machine Learning 10 10 -701/15 701/15 -781, Fall 2008 781, Fall 2008 Neural Networks Neural Networks Eric Xing Eric Xing Lecture 6, September 24, 2008 Reading: Chap. 5 CB © Eric Xing @ CMU, 2006-2008 2 Learning highly non-linear functions f: X Æ Y z f might be non-linear function z X (vector of) continuous and/or discrete vars z Y (vector of) continuous and/or discrete vars The XOR gate Speech recognition
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
© Eric Xing @ CMU, 2006-2008 3 z From biological neuron to artificial neuron (perceptron) z Activation function z Artificial neuron networks z supervised learning z gradient descent Perceptron and Neural Nets Soma Soma Synapse Synapse Dendrites Axon Synapse Dendrites Axon Threshold Inputs x 1 x 2 Output Y Hard Limiter w 2 w 1 Linear Combiner θ = = n i i i w x X 1 ω < ω + = 0 0 X Y if , if , 1 1 Input Layer Output Layer Middle Layer I n p u t S i g n a l s O u t p u t © Eric Xing @ CMU, 2006-2008 4 Connectionist Models z Consider humans: z Neuron switching time ~ 0.001 second z Number of neurons ~ 10 10 z Connections per neuron ~ 10 4-5 z Scene recognition time ~ 0.1 second z 100 inference steps doesn't seem like enough Æ much parallel computation z Properties of artificial neural nets (ANN) z Many neuron-like threshold switching units z Many weighted interconnections among units z Highly parallel, distributed processes Synapses Axon Dendrites Synapses + + + - - (weights) Nodes
Background image of page 2
© Eric Xing @ CMU, 2006-2008 5 Abdominal Pain Perceptron Male Age Temp WBC Pain Intensity Pain Duration 37 10 1 12 0 1 adjustable weights 01 0 0 0 0 0 Appendicitis Diverticulitis Perforated Duodenal Ulcer Non-specific Pain Cholecystitis Small Bowel Obstruction Pancreatitis © Eric Xing @ CMU, 2006-2008 6 0.8 Myocardial Infarction “Probability” of MI 1 1 2 1 50 Male Age Smoker ECG:ST Elevation Pain Intensity 4 Pain Duration Myocardial Infarction Network
Background image of page 3

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
© Eric Xing @ CMU, 2006-2008 7 The "Driver" Network © Eric Xing @ CMU, 2006-2008 8 weights Output units No disease Pneumonia Flu Meningitis Input units Cough Headache what we got what we wanted - error Δ rule change weights to decrease the error Perceptrons
Background image of page 4
© Eric Xing @ CMU, 2006-2008 9 Input units Input to unit j : a j = Σ w ij x i j i Input to unit i : x i Measured value of variable i Output of unit j y j =f ( a j ) Output units Perceptrons ± a is known as an activation ± f () is a differentialbe, nonlinear activation function © Eric Xing @ CMU, 2006-2008 10
Background image of page 5

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Image of page 6
This is the end of the preview. Sign up to access the rest of the document.

This note was uploaded on 01/26/2010 for the course MACHINE LE 10701 taught by Professor Ericp.xing during the Fall '08 term at Carnegie Mellon.

Page1 / 20

lecture6-annotated - Machine Learning 10-701/15-781 Fall...

This preview shows document pages 1 - 6. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online