prml-slides-2 - PATTERN RECOGNITION AND MACHINE LEARNING...

Info iconThis preview shows pages 1–35. Sign up to view the full content.

View Full Document Right Arrow Icon
PATTERN RECOGNITION AND MACHINE LEARNING CHAPTER 2: PROBABILITY DISTRIBUTIONS
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Parametric Distributions Basic building blocks: Need to determine given Representation: or ? Recall Curve Fitting
Background image of page 2
Binary Variables (1) Coin flipping: heads=1, tails=0 Bernoulli Distribution
Background image of page 3

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Binary Variables (2) N coin flips: Binomial Distribution
Background image of page 4
Binomial Distribution
Background image of page 5

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Parameter Estimation (1) ML for Bernoulli Given:
Background image of page 6
Parameter Estimation (2) Example: Prediction: all future tosses will land heads up Overfitting to D
Background image of page 7

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Beta Distribution Distribution over .
Background image of page 8
Bayesian Bernoulli The Beta distribution provides the conjugate prior for the Bernoulli distribution.
Background image of page 9

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Beta Distribution
Background image of page 10
Prior ∙ Likelihood = Posterior
Background image of page 11

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Properties of the Posterior As the size of the data set, N , increase
Background image of page 12
Prediction under the Posterior What is the probability that the next coin toss will land heads up?
Background image of page 13

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Multinomial Variables 1 -of- K coding scheme:
Background image of page 14
ML Parameter estimation Given: Ensure , use a Lagrange multiplier, ¸ .
Background image of page 15

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
The Multinomial Distribution
Background image of page 16
The Dirichlet Distribution Conjugate prior for the multinomial distribution.
Background image of page 17

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon