This preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Unformatted text preview: 12/17/2009 Data Mining: Principles and Algorithms 1 Data Mining: Principles and Algorithms Jianyong Wang Database Lab, Institute of Software Department of Computer Science and Technology Tsinghua University [email protected] 12/17/2009 Data Mining: Principles and Algorithms 2 Chapter 6. Classification and Prediction What is classification? What is prediction? Issues regarding classification and prediction Classification by decision tree induction Bayesian classification Rulebased classification Artificial Neural Networks Support Vector Machines (SVM) Associative classification Lazy learners (or learning from your neighbors) Other classification methods Ensemble methods Prediction Accuracy and error measures Summary 12/17/2009 Data Mining: Principles and Algorithms 3 Artificial Neural Networks Composed of basic units and weighted links between them The basic units (or nodes) are an idealization of neurons Responsible for basic computations The pattern of connections of the units determines the network architecture 12/17/2009 Data Mining: Principles and Algorithms 4 Artificial Neural Networks X 1 X 2 X 3 Y 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 X 1 X 2 X 3 Y Black box Output Input Output Y is 1 if at least two of the three inputs are equal to 1. 12/17/2009 Data Mining: Principles and Algorithms 5 Artificial Neural Networks X 1 X 2 X 3 Y 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 X 1 X 2 X 3 Y Black box 0.3 0.3 0.3 t=0.4 Output node Input nodes otherwise true is if 1 ) ( where ) 4 . 3 . 3 . 3 . ( 3 2 1 z z I X X X I Y 12/17/2009 Data Mining: Principles and Algorithms 6 Artificial Neural Networks Model is an assembly of interconnected nodes and weighted links Output node sums up each of its input value according to the weights of its links Compare output node against some threshold t X 1 X 2 X 3 Y Black box w 1 t Output node Input nodes w 2 w 3 ) ( t X w I Y i i i Perceptron Model ) ( t X w sign Y i i i or 12/17/2009 Data Mining: Principles and Algorithms 7 General Structure of ANN Activation function g(S i ) S i O i I 1 I 2 I 3 w i1 w i2 w i3 O i Neuron i Input Output threshold, t Input Layer Hidden Layer Output Layer x 1 x 2 x 3 x 4 x 5 y Training ANN means learning the weights of the neurons 12/17/2009 Data Mining: Principles and Algorithms 8 Common Activation Functions Step function:  g(x)=1, if x >= t ( t is a threshold) g(x) = 0, if x < t Sign function:  g(x)=1, if x >= t ( t is a threshold) g(x) = 1, if x < t Sigmoid function: g(x)= 1/(1+exp(x)) 12/17/2009 Data Mining: Principles and Algorithms 9 Network Structures Singlelayered network vs. multilayered network  Perceptron : contains only input and output nodes (but no hidden layer) Basically linear threshold functions (ltf): defined by weights W and threshold t , value is 1 iff W × x ≥ t , otherwise, 0 Applying model is straightforward, e.g., Y=step(0.3XApplying model is straightforward, e....
View
Full
Document
This note was uploaded on 06/02/2010 for the course COMPUTER DM2009F taught by Professor Wangwei during the Fall '09 term at Tsinghua University.
 Fall '09
 WangWei
 Algorithms, Data Mining

Click to edit the document details