13_naive_bayes

13_naive_bayes - Probability theory A crash course in...

Info iconThis preview shows pages 1–4. Sign up to view the full content.

View Full Document Right Arrow Icon
10/29/13 1 A crash course in probability and Naïve Bayes classification Chapter 9 1 Probability theory Random variable : a variable whose possible values are numerical outcomes of a random phenomenon. Examples: A person’s height, the outcome of a coin toss Distinguish between discrete and continuous variables. The distribution of a discrete random variable: The probabilities of each value it can take. Notation: P(X = x i ). These numbers satisfy: 2 i P ( X = x i )=1 Probability theory Marginal Probability Conditional Probability Joint Probability Probability theory A joint probability distribution for two variables is a table. If the two variables are binary, how many parameters does it have? What about joint probability of d variables P(X 1 ,…,X d )? How many parameters does it have if each variable is binary? 4
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
10/29/13 2 Probability theory Marginalization: Product Rule The Rules of Probability Marginalization Product Rule Independence: X and Y are independent if P(Y|X) = P(Y) This implies P(X,Y) = P(X) P(Y) Using probability in learning Interested in: X – features Y – labels For example, when classifying spam, we could estimate P(Y| Viagara, lottery) We would then classify an example if P(Y|X) > 0.5. However, it’s usually easier to model P(X | Y) 7 P ( Y | X ) 0 0.5 1 Maximum likelihood Fit a probabilistic model P(x | θ ) to data Estimate θ Given independent identically distributed (i.i.d.) data X = (x 1 , x 2 , …, x n ) Likelihood Log likelihood Maximum likelihood solution: parameters θ that maximize ln P(X | θ ) ln P ( X | )= n X i =1 ln P ( x i | ) P ( X | P ( x 1 | ) P ( x 2 | ) ,...,P ( x n | )
Background image of page 2
10/29/13 3 Example Example : coin toss Estimate the probability p that a coin lands “Heads” using the result of n coin tosses, h of which resulted in heads.
Background image of page 3

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Image of page 4
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

Page1 / 10

13_naive_bayes - Probability theory A crash course in...

This preview shows document pages 1 - 4. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online