{[ promptMessage ]}

Bookmark it

{[ promptMessage ]}

13_naive_bayes

# 13_naive_bayes - Probability theory A crash course in...

This preview shows pages 1–4. Sign up to view the full content.

10/29/13 1 A crash course in probability and Naïve Bayes classification Chapter 9 1 Probability theory Random variable : a variable whose possible values are numerical outcomes of a random phenomenon. Examples: A person’s height, the outcome of a coin toss Distinguish between discrete and continuous variables. The distribution of a discrete random variable: The probabilities of each value it can take. Notation: P(X = x i ). These numbers satisfy: 2 i P ( X = x i )=1 Probability theory Marginal Probability Conditional Probability Joint Probability Probability theory A joint probability distribution for two variables is a table. If the two variables are binary, how many parameters does it have? What about joint probability of d variables P(X 1 ,…,X d )? How many parameters does it have if each variable is binary? 4

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
10/29/13 2 Probability theory Marginalization: Product Rule The Rules of Probability Marginalization Product Rule Independence: X and Y are independent if P(Y|X) = P(Y) This implies P(X,Y) = P(X) P(Y) Using probability in learning Interested in: X – features Y – labels For example, when classifying spam, we could estimate P(Y| Viagara, lottery) We would then classify an example if P(Y|X) > 0.5. However, it’s usually easier to model P(X | Y) 7 P ( Y | X ) 0 0.5 1 Maximum likelihood Fit a probabilistic model P(x | θ ) to data Estimate θ Given independent identically distributed (i.i.d.) data X = (x 1 , x 2 , …, x n ) Likelihood Log likelihood Maximum likelihood solution: parameters θ that maximize ln P(X | θ ) ln P ( X | )= n X i =1 ln P ( x i | ) P ( X | P ( x 1 | ) P ( x 2 | ) ,...,P ( x n | )
10/29/13 3 Example Example : coin toss Estimate the probability p that a coin lands “Heads” using the result of n coin tosses, h of which resulted in heads.

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

### Page1 / 10

13_naive_bayes - Probability theory A crash course in...

This preview shows document pages 1 - 4. Sign up to view the full document.

View Full Document
Ask a homework question - tutors are online