bayesian-learning

bayesian-learning - Bayesian Learning CS 464: Introduction...

Info iconThis preview shows pages 1–3. Sign up to view the full content.

View Full Document Right Arrow Icon
1 CS 464: Introduction to Machine Learning Bayesian Learning Slides adapted from Section 6.1, 6.2, 6.3, and 6.9 Machine Learning by Tom M. Mitchell http://www-2.cs.cmu.edu/afs/cs.cmu.edu/user/mitchell/ftp/mlbook.html 2 Bayesian Learning Bayes Theorem MAP, ML hypotheses MAP learners Naive Bayes learner 3 Roles for Bayesian Methods Provides practical learning algorithms: • Naive Bayes learning • Bayesian belief network learning (will be covered later) • Combine prior knowledge (prior probabilities) with observed data • Requires prior probabilities Provides useful conceptual framework • Provides “gold standard” for evaluating other learning algorithms • Provides insight into Occam's razor 4 Bayes Theorem In machine learning, we try to determine the best hypothesis from some hypothesis space H, given the observed training data D. In Bayesian learning, the best hypothesis means the most probable hypothesis, given the data D plus any initial knowledge about the prior probabilities of the various hypotheses in H. Bayes theorem provides a way to calculate the probability of a hypothesis based on its prior probability, the probabilities of observing various data given the hypothesis, and the observed data itself. 5 Bayes Theorem P( h | D ) = P( D | h ) P( h ) / P( D ) • P( h ) = prior prob. of hypothesis h • P( D ) = prior prob. of training data D • P( h | D ) = probability of h given D • P( D | h ) = probability of D given h 6 Bayes Theorem - Example Sample Space for events A and B P(A) = 4/7 P(B) = 3/ 7 P(B|A) = 2/4 P(A|B) = 2/3 Is Bayes Theorem correct? P(B|A) = P(A|B)P(B) / P(A) = ( 2/3 * 3/7 ) / 4/7 = 2/4 b CORRECT P(A|B) = P(B|A)P(A) / P(B) = ( 2/4 * 4/7 ) / 3/7 = 2/3 b CORRECT A holds T T F F T F T B holds T F T F T F F
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
7 Choosing Hypotheses Natural choice is most probable hypothesis
Background image of page 2
Image of page 3
This is the end of the preview. Sign up to access the rest of the document.

Page1 / 4

bayesian-learning - Bayesian Learning CS 464: Introduction...

This preview shows document pages 1 - 3. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online