{[ promptMessage ]}

Bookmark it

{[ promptMessage ]}

L11testing

# L11testing - Testing classifier accuracy(cse352 Professor...

This preview shows pages 1–9. Sign up to view the full content.

Testing classifier accuracy (cse352) Professor Anita Wasilewska Lecture Notes on Learning (6)

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
Overview • Introduction • Basic Concept on Training and Testing Resubstitution (N ; N) Holdout (2N/3 ; N/3) x-fold cross-validation (N-N/x ; N/x) Leave-one-out (N-1 ; 1)
Introduction Predictive Accuracy Evaluation The main methods of predictive accuracy evaluations are: Resubstitution (N ; N) Holdout (2N/3 ; N/3) x-fold cross-validation (N-N/x ; N/x) Leave-one-out (N-1 ; 1) where N is the number of records (instances) in the dataset

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
Training and Testing REMEMBER: we must know the classification (class attribute values) of all instances (records) used in the test procedure. Basic Concepts Success: instance (record) class is predicted correctly Error: instance class is predicted incorrectly Error rate : a percentage of errors made over the whole set of instances (records) used for testing Predictive Accuracy: a percentage of well classified data in the testing data set.
Training and Testing • Example: Testing Rules (testing record #1) = record #1.class - Succ Testing Rules (testing record #2) not= record #2.class - Error Testing Rules (testing record #3) = record #3.class - Succ Testing Rules (testing record #4) = instance #4.class - Succ Testing Rules (testing record #5) not= record #5.class - Error Error rate: 2 errors: #2 and #5 Error rate = 2/5=40% Predictive Accuracy: 3/5 = 60%

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
Resubstitution (N ; N)
Re-substitution Error Rate • Re-substitution error rate is obtained from training data • Training Data Error: uncertainty of the rules • The error rate is not always 0%, but usually (and hopefully) very low! • Resubstitution error rate indicates only how good (bad ) are our results (rules, patterns, NN) on the TRAINING data; expresses some knowledge about the algorithm used.

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
Re-substitution Error Rate Re-substitution Error Rate is usually used as the performance measure: The training error rate reflects imprecision of the training
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

### Page1 / 24

L11testing - Testing classifier accuracy(cse352 Professor...

This preview shows document pages 1 - 9. Sign up to view the full document.

View Full Document
Ask a homework question - tutors are online