The Hong Kong University of Science and Technology
Pattern Recognition
COMP 5217

Fall 2003
Topic #1
INTRODUCTION
COMP527
Pattern Recognition
Fall 2001
Mathematical Foundations
! Probability theory [DHS Section A.4]
! Linear algebra [DHS Section A.2]
1.2
Pattern Recognition
! Pattern recognition is the science that concerns the description and
c
The Hong Kong University of Science and Technology
Pattern Recognition
COMP 5217

Fall 2003
Topic #11
ESTIMATING AND COMPARING
CLASSIFIERS
COMP527
Pattern Recognition
Fall 2001
Motivations
! Why do we want to estimate the generalization error of a classifier
on a given classification problem?
! Reason 1:
"
We want to know if the classifier perfo
The Hong Kong University of Science and Technology
Pattern Recognition
COMP 5217

Fall 2003
Topic #9
MIXTURE DENSITY ESTIMATION
COMP527
Pattern Recognition
Fall 2001
Introduction
! Mixture density estimation methods are parametric density estimation
methods.
! They can be used for clustering applications parametric approach.
! Mixture densities:
The Hong Kong University of Science and Technology
Pattern Recognition
COMP 5217

Fall 2003
Topic #8
CLUSTERING
COMP527
Pattern Recognition
Fall 2001
Introduction
! While supervised procedures deal with labeled training examples (i.e.,
examples labeled by their category membership), unsupervised
procedures that we are going to study in the next
The Hong Kong University of Science and Technology
Pattern Recognition
COMP 5217

Fall 2003
Topic #7
FEATURE SELECTION AND EXTRACTION
COMP527
Pattern Recognition
Fall 2001
Introduction
! Pattern preprocessing may be necessary because:
"
"
Some features are irrelevant to the classification task.
Strong correlations exist between sets of features
The Hong Kong University of Science and Technology
Pattern Recognition
COMP 5217

Fall 2003
Topic #6
FEEDFORWARD NEURAL NETWORKS
COMP527
Pattern Recognition
Fall 2001
Layered Feedforward Networks
! Singlelayer feedforward networks:
"
One layer of processing units
d input units
"
Perceptron [Rosenblatt 1962]:
d cfw_0,1
c
"
c output units
or
d cf
The Hong Kong University of Science and Technology
Pattern Recognition
COMP 5217

Fall 2003
Topic #5
DISCRIMINANT FUNCTIONS
COMP527
Pattern Recognition
Fall 2001
Different Approaches to Pattern Classification
! Approach 1: Estimation of p(x  i)
"
"
From p(x  i) and P(i), use Bayes rule to compute P(i  x) and then
make decision.
Two alternativ
The Hong Kong University of Science and Technology
Pattern Recognition
COMP 5217

Fall 2003
Topic #4
NONPARAMETRIC DENSITY ESTIMATION
COMP527
Pattern Recognition
Fall 2001
Different Approaches to Pattern Classification
! Approach 1: Estimation of p(x  i)
"
"
From p(x  i) and P(i), use Bayes rule to compute P(i  x) and then
make decision.
Two
The Hong Kong University of Science and Technology
Pattern Recognition
COMP 5217

Fall 2003
Topic #3
PARAMETRIC DENSITY ESTIMATION
COMP527
Pattern Recognition
Fall 2001
Different Approaches to Pattern Classification
! Approach 1: Estimation of p(x  i)
"
"
From p(x  i) and P(i), use Bayes rule to compute P(i  x) and then
make decision.
Two alt
The Hong Kong University of Science and Technology
Pattern Recognition
COMP 5217

Fall 2003
Topic #2
BAYESIAN DECISION THEORY
COMP527
Pattern Recognition
Fall 2001
State of Nature
! Let be a random variable that denotes the class label for a pattern,
which is also said to be the state of nature of the pattern.
! A random variable has to be used
The Hong Kong University of Science and Technology
Pattern Recognition
COMP 5217

Fall 2003
Topic #12
COMBINING CLASSIFIERS
COMP527
Pattern Recognition
Fall 2001
Introduction
! For a given pattern classification problem, usually a number of different
methods can be used to solve it. These methods differ from each other in
at least the following