best The way to approach a Bayes problem is
1) Determine what the data and hypotheses are. Try to be precise.
Data are things that you observe. Hypotheses are things you want to
know, but don't observe.
2) What are the prior and the likelihood?
P(H|D) = P(D|H) P(H)/P(D)
(i) the prior may come from counting previous occurances (frequentist) or
from someone's opinion (subjectivist) or
from the likelihood principle (see below
(ii) likelihoods usually come from a model of the world, but can also be estimated
by counting previous occurances.
3) Do you need to find the probability of the data? This is required
for finding P(H|D), but not for finding the most likely hypothesis.
If you do need to find P(D), is it given, or should it be found by
marginalization?
P(D) = P(D|h1)P(h1) + P(D|h2)P(h2) + P(D|h3)P(h3) + ...
4) Do you need to make the "Naive Bayes" assumption that data are conditionally
independent given the hypothesis?
P(d1,d2|H) = P(d1|H)P(d2|H)
5) you Once have estimated the probability of each hypothesis, do you need to make
predictions about future data? This generally requires using marginalization.
P(newD) = P(newD|h1)P(h1) + P(newD|h2)P(h2) + P(newD|h3)P(h3) + ...
where P(newD|H) is the likelihood used above, and P(H) is what was calculated above:
P(H | oldD)
_____________________________________________________________________________________________
* the likelihood principle states that, in some situations, there are classes of hypotheses, and
the prior on a hypothesis is
1/(the number of classes * the...