BayesianCurveFitting

BayesianCurveFitting - PATTERN
RECOGNITION

...

Info iconThis preview shows page 1. Sign up to view the full content.

View Full Document Right Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: PATTERN
RECOGNITION

 AND
MACHINE
LEARNING
 CHAPTER
1:
INTRODUCTION
 Example
 Handwri/en
Digit
Recogni6on
 Polynomial
Curve
Fi=ng 

 Sum‐of‐Squares
Error
Func6on
 0th
Order
Polynomial
 1st
Order
Polynomial
 3rd
Order
Polynomial
 9th
Order
Polynomial
 Over‐fi=ng
 Root‐Mean‐Square
(RMS)
Error:
 Polynomial
Coefficients


 Data
Set
Size:

 9th
Order
Polynomial
 Data
Set
Size:

 9th
Order
Polynomial
 Regulariza6on
 Penalize
large
coefficient
values
 Regulariza6on:

 Regulariza6on:

 Regulariza6on:










vs.

 Polynomial
Coefficients


 Probability
Theory
 Marginal
Probability
 Joint
Probability
 Condi6onal
Probability
 Probability
Theory
 Sum
Rule
 Product
Rule
 The
Rules
of
Probability
 Sum
Rule
 Product
Rule
 Bayes’
Theorem
 posterior
∝
likelihood
×
prior
 Probability
Theory
 Apples
and
Oranges
 4/10 6/10 Bayes’
Theorem
 p(Z|X,
Y)
=
p(Y|X,
Z)
p(Z|X)
/
p(Y|X)
 p(Y|X)
=
sum
over
Z
of

p(Y|X,
Z)
p(Z|X)

 Probability
Densi6es
 P’(x) = p(x) Transformed
Densi6es
 Suppose x = g(y) Expecta6ons
 Condi6onal
Expecta6on
 (discrete)
 Approximate
Expecta6on
 (discrete
and
con6nuous)
 Variances
and
Covariances
 The
Gaussian
Distribu6on
 Gaussian
Mean
and
Variance
 The
Mul6variate
Gaussian
 Gaussian
Parameter
Es6ma6on
 Likelihood
func6on
 Maximum
(Log)
Likelihood
 Proper6es
of









and

 Curve
Fi=ng
Re‐visited
 Maximum
Likelihood
 Determine











by
minimizing
sum‐of‐squares
error,












.
 Predic6ve
Distribu6on
 MAP:
A
Step
towards
Bayes
 Determine














by
minimizing
regularized
sum‐of‐squares
error,












.
 Bayesian
Curve
Fi=ng
 Bayesian
Predic6ve
Distribu6on
 ...
View Full Document

This note was uploaded on 02/27/2010 for the course ECE 544 taught by Professor Ray during the Spring '10 term at Rutgers.

Ask a homework question - tutors are online