This preview shows pages 1–13. Sign up to view the full content.
This preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Unformatted text preview: Notes on Probability Nicolas Fillion April 1, 2008
021 Critical Thinking 1 Basic Notions Probability is a mathematical tool that we use in inductive inferences. As we
know, an inductive argument is one such that the truth of the premises does not
guarantee the truth of the conclusion. Rather, an inductive argument makes
the conclusion more or less likely. The science of quantifying likelihood is called
probability theory. A probability is a number between 0 and 1. When all the possibilities are
equally likely (this condition is important), a probability is just the ratio of the
number of desired outcomes to the number of possible outcomes. For an event
E, we denote the probability of the event by ‘P(E).’ Example 1 The probability of rolling a 6 with a fair die is: W = l 20.167 Pm)ng a 6) = number of faces 6 Example 2 The probability of rolling an even number on a fair die is: b  b 3
P(rolling a even number) = —num er of we" "mm W faces 2 — = 0.5
number of faces 6
Example 3 Suppose there is 5 choices of answer for a question in your exam.
What is the probability that you choose the right answer by chance?
number of right answer _ 1 P ' ht = — — — = 0.2
(mg answer) number of answers 5 Example 4 What is the probability of picking a jack in a normal 52card deck? , number of jack 4
P k : — : — : 0.077
(We ) number of cards 52
We can use this method to ﬁx the probability of simple events. Now, we
can ﬁnd the probability that one of these events happen twice in a row, or the
probability that two different events happen together. In this case, we ﬁnd the
number of possible outcomes by multiplying the number of possible outcomes of each of them. Example 5 How many different outcomes can we obtain if we roll two dice?
6 X 6 : 36 Example 6 How many diﬁerent outcomes can we obtain if we flip a coin three
times?
2 X 2 x 2 = 8 Example 7 How many diﬁerent outcomes can we obtain if we ﬂip a coin and
then roll a die?
2 x 6 = 12 We can use this counting method to ﬁnd the probability of complex events.
Consider these examples. Example 8 What is the probability of obtaining two sixes when rolling two fair
dice? P(hauing two sixes) = W = i = 0.027
number of outcomes 36 Example 9 What is the probability of ﬂipping two heads if we flip a coin twice? P(two heads) 2 number of pair of heads 2 l = 025 number of outcomes 4 Example 10 Now, we consider a more diﬁ‘icult problem. What is the proba
bility, when rolling a die twice, that the sum will equal 5‘? There are four ways
of obtaining numbers summing up to 5: 4 and 1, 3 and 2, 2 and 3, 1 and 4.
Note that “2 and 3” and “3 and 2” are diﬁ’erent, since the ﬁrst means that we
obtained 2 on the ﬁrst roll, while the second means that we obtained 3 on the
ﬁrst roll. We thus obtain: ' 5 4
P (sum of 5) = number of outcomes summing to = _ = 0111
number of outcomes 36 2 Permutations and Combinations We have seen how to count the number of possible outcomes for an event.
Sometimes, the order of the outcomes is important, but sometimes it is not.
When the order is important, we want to be able to ﬁnd the number of ways to
rearrange a certain number of items. For example, if we have the three letters
a, b and c, we can rearrange them in 6 ways: abc, acb, bac, bca, cba, cab The mathematical process that is involved here is sampling without replacement.
We start with three possible letters and we choose one. When we choose the
second letter, there is only two choices left. And when we choose the third letter, there is only one choice left. To ﬁnd the number of rearrangements of 3
objects, we thus multiply
3 X 2 X 1 = 6 Likewise, for four objects, we would have
4 x 3 x 2 x 1 = 24. Thus, there is 24 possible rearrangements of four objects. There is a general
formula that gives us the number of ways to rearrange n items. The function
that we use is called the factorial, denotes n!, and it is deﬁned as follows: nl=n><(n—1)><(n—2)><...><3><2><1 Note that, by convention,
0! = 1 Example 11 In how many ways can we rearrange the ten digits, using them
only once? The number of ways to rearrange them is: 10!:10x9x8x7x6x5x4x3x2x123,628,800 Example 12 How many signals can we send with ﬂags of 6 diﬂ'erent colors?
{ We assume that we use each color only once.) 6!:6x5x4x3x2x12720 The number of rearrangements of items is also called the number of permuta
tions. On the other hand, when order does not matter, we count the number of
combinations. Consider our permutations of the letters a, b, c: abc, acb, bac, bca, cba, cab When the order does not matter, all of them are equivalent. Thus, there is only
one possible combination of those three letters. Now, we would like to ﬁnd a general way to count the number of per
mutations and combinations. Suppose we have four ﬂags of different colors
(G, R, B, We may want to ask how many twoﬂag messages we can send
(we use each color only once). Obviously, the number will be different whether
the order of the colors matter or not. Let us make a list: Permutations: GR, RG, GB, BG, GVV, WG, RB, BR, RVV, WR, BVV, WB
Combinations: GR, GB, G'VV, RB, RVV, BW As we see, there is 12 permutations and 6 combinations. Note: since we ob
tain the combinations from the permutations by ignoring order, the number of
combinations is lower than the number of permutations. To ﬁnd the number of
permutations of 2 items out of 4, we simply multiply 4 x 3 = 12. Example 13 In how many ways can we pick 4 items out of 6'?
6 x 5 x 4 x 3 = 360 Note that this question is equivalent to: In how many ways can we permute 4
items out of 6? Note also that 6x5x4x3x2x1_6! 6 5 4 3 = _ _
X X X 2 X 1 2!
We can now understand the general formula for counting permutations: n!
(n — r)! nPr= Here, “Pr means that we pick r elements out of n. The “P” stands for permu
tations. Example 14 In how many ways can we rearrange 6 ﬂags out of 10 ﬂags of
diﬂ'erent color? (Assume we use each color just once) 10! i 10! 7 3,628,800 — _ — _ : 151 200
(10—6)! 4! 24 ’ As in the last examples, the numbers are often very big. This is why it is
important to understand how to use the general formula. Now, as we have seen, the number of combinations is lower than the number
of permutations, since we ignore the order. To get the number of combinations,
we simply have to divide the number of permutations by the possible number of
arrangements of the items we have chosen. For r items, this number corresponds
to r!. Example 15 In how many ways can we pick 4 items out of 6, if the order does notmatter‘?
6>< 5 x4 x 3 _ 4! 15 Note also that
6>< 5 X4 x 3 i 6! 4! ’ M
We can now understand the general formula for counting permutations: n! 1101' = —
r!(n — r)! Here, "Or means that we combine r elements out of n. The “C” stands for
combinations. With these formula, we are already able to compute the probability of more
complex events, such as winning the Lotto 6/49 or having four of a kind in a
poker hand. Example 16 What are the chances of winning Lotto 6/49? P(wm) _ # winning combination _ 1 _ 1 _ 1
# possible combinations 4906 $ 13,983, 816 It is approximately one chance over 14,000,000. Example 17 What are the chances of having four of a kind in a poker hand? # four of a kind _ 13 X 48 _ 624
# possible hands _ 5205 _ 2,598,960 P(4 of a = = 0.00024
Note that it is essential to mutiply by 48, since four queens and a 3 of spade is
not the same as four queens and a 4 of diamond. As you see, the probability is
very low! Example 18 Suppose we ﬂip a coin ﬁve times. What is the probability of having
4 heads? To ﬁnd the answer, we need the number of ways to have 4 heads in 5
ﬂips (which is 504) and the number of possible outcomes for ﬁve flips {which
is 25 = 32 ): # of ways to have 4 heads _ 504 5 = — = 0.156
0 0882 6 OH comes
# fp N t 25 32 P(4 heads) 2 3 Joint and Alternative Occurrences A joint occurrence is one in which many events happen together. If we want
to ﬁnd the probability of joint occurrences of events, we may simply multiply
their probability, given that they are independent. By independent, we mean
that they are not inﬂuencing one another. Example 19 What is the probability of obtaining two sides when rolling two
fair dice?
1 1 1 P(having two sixes) = P(rolling 6') x P(rolling 6) = E X E = % = 0.027 In general, if two events denoted by A and B are independent events, we may
ﬁnd their joint probability by means of this formula: P(A&B) = P(A) >< P(B) We are often interested in ﬁnding the probability of another sort of complex
event, namely the probability of alternative occurrences. It happens when we
want to know the probability that something, or another thing, happens. Example 20 What is the probability of rolling a 1 or a 2 on a fair die? P(l v 2) = P(1)+ P(2) = g = 0.333 We may simply add up the probability of 1 and 2, since they are independent. Let us explain independence in terms of mutual exclusion. We say that two
events are mutually exclusive when they cannot both occur at the same time.
In our last example, rolling a 1 and rolling a 2 are independent, because we
cannot roll a 1 and a 2 on the same roll. Generally speaking, if we have two
mutually exclusive events A and B, we can ﬁnd their alternative probability by
means of the formula P(A v B) = P(A) + P(B) Example 21 An urn contains 20 red balls, and 10 blue balls. What is the
probability that, when we pick two balls without replacement, we obtain exactly
one red and one blue? Obviously, there are two ways of obtaining this outcome, i.e., R&B and
B&R. The order is important, since we pick balls without replacement. Now,
these two ways of obtaining exactly one red and one blue balls are just alternative
events. Thus, 20 10 10 20
P(1 red 65 1 blue) — P(R&B) +P(B&R) — 30 X 29 + 30 x 29 — 0.230
Now, consider two events, such as drawing a diamond from a deck, and drawing
an ace. Clearly, we cannot ﬁnd their alternative occurrence by simply adding
them together, since we would count the probability of drawing the as of dia
mond twice. Accordingly, when two events A and B are not mutually exclusive,
we ﬁnd their alternative probability by means of the formula: P(A v B) = P(A) + P(B) — P(A&B) A special case of probability of alternative and joint occurrences is that in
which the complex event we are interested with contains phrases such as at least
or at most. In this case, we almost always want to use the formula P(E happens) 2 1 — P(E does not happen) Thus, instead of ﬁnd the probability that something happens, we ﬁnd the prob
ability that it does not happen; when subtracted from one, they are equal. Example 22 What is the probability of having at least one tail if we flip a fair
coin ﬁve times? 1
P(at least one tail) 2 1— P(5 heads) 2 1 — = 0.969 Example 23 What is the probability of rolling at least one six with two fair
dice? 5
P(at least one 6): 1 — P(zero 6) = 1 — = 0.306 4 Conditional Probability Conditional probability is very important for cases in which events are not inde
pendent. In such cases, the probability of an event depends on the occurrence of
some other events. This is why, in such cases, we talk about the probability of
an event, given the occurrence of some other event. The mathematical symbol
that we use to express “given” is “”. So, the probability of having rolled a 6
on a fair die, given that we have rolled an even number, is expressed as follows: P(rolling a 6  rolling an even) In general, for two events A and B, we say that the probability of A given B is
P(A  B). Note that if A and B are independent, asking what is the probability
of A and asking what is the probability of A given B will be the same (i.e., if
A and B are independent, P(A) : P(A  B). But in most cases, A and B are
not independent. In this case, we ﬁnd the conditional probability by means of the following formulas:
P(A&B) P(AB)= P(B) Example 24 An urn contain 100 balls. 20 of them are red. What is the prob
ability of picking two red balls, if we pick without replacement? P(2 reds) = P(red on ﬁrst pick) X P(red on second pick  red on ﬁrst pick)
_ n X a _ a
_ 100 99 _ 495 Example 25 An urn contain 100 balls. 20 of them are red. What is the prob
ability of picking two red balls, if we pick with replacement? n
100 = 0.038 P(2 reds) = P(red) X P(red) =( )2 = 0.04 Example 26 What is the probability of drawing two queens in a normal 52card
deck a) with replacement and b) without replacement?
With replacement: 4
P(two queens) = (a)2 = 0.006 Without replacement: P(two queens) = P(Q on Ist draw) X P(Q on 2nd draw  Q on Ist draw) 4 3 [I2 —p=ﬂﬁandn=20
D13— p=D.Tandn=20' —p=D.5 andn=40
[I1E — [I1I1 —
El.12 
D1
[IDS
[IDE
[ID4 [ID2 Figure 1: The Binomial Distribution. 5 Binomial Distribution The binomial distribution is a mathematical construction that allows us to ﬁnd
the probability that an event will occur several times. Let us try to see how
this construction works. As we have seen, the probability P(E) of an event E
is always between 0 and 1. In cases where we apply the binomial distribution,
different occurences of event E are always independent. Let us consider a few
examples. Example 27 There is 100 balls in an urn. 40 of them are blue. What is the
probability of drawing exactly 3 blue balls in 5 picks? We need to have 3 blue balls and 2 balls of a diﬁerent color. Also, we must
consider that there are many different ways to obtain 3 blue balls and 2 balls
of a diﬁ’erent color. Actually, we may verify that this number is the number
of combinations 503 = 10. The probability of picking exactly 3 blue balls in .5
picks is thus: P(3 blue in 5 picks: 5 0'3 X (%)3 X( 60 2 _
100) _0.2304 In general, when the probability of an event is p, the probability of failure is
1 — p and the probability of r successes in n trials is “CT )< pr X (1 —p)"_r. The graph of this function is in ﬁgure 1. 6 Sampling In the last example, we knew the probability of picking a blue ball. From our
knowledge of this number, we asked: What is the probability of drawing exactly
3 blue balls in 5 picks? With sampling problems, the situation is reversed. For
example, we would start with the fact (or data) that we have picked exactly
three blue balls out of 5 picks, and we would ask: What is the probability of
picking a blue ball in this urn? Clearly, our answer will contain some uncertainty. Let us call the items that we look at samples and let us call their number the
sample size. As a matter of fact, when our sample size is small, our uncertainty
is large. When our sample size is large, our uncertainty is small. Probability
theory is the science that allows us to understand our degree of uncertainty
precisely. To understand how we quantify our uncertainty, let us consider an example.
Suppose we have an urn with 1000 balls. We know that they are either red or
blue. We want to know what is the “real probability” of picking a red ball (or,
alternatively, what is the real proportion of red balls). In “sampling problems,”
we always start with a set of data. In this example, we decide to pick 100 balls
in order to obtain data. We happen to have 40 red balls, and 60 blue balls. On the basis of the data obtained, what is our best guess regarding the
number of red balls in the urn? Well, since 40% of the balls sampled are red,
our best guess is that 400 out of 1000 balls are red. But can we be sure, on the
basis of this sample, that there is exactly 400 red balls in the urn? No. It is an
inductive inference and, as such, the truth of the conclusion is not guaranteed.
Thus, even if 400 is our best guess, 399 would not be a bad guess either. Now,
the question is: how far from 400 can we go, and still be making a good guess?
The answer will depend on the conﬁdence interval. Before giving a deﬁnite answer, let us generalize our reasoning. In the last
paragraph, we have asked what number is our best guess. But more generally, we
may ask how conﬁdent we can be that the “real proportion” is a given number,
plus or minus another number? We could ask, for example, how conﬁdent we
are that the number of red balls in the urn is between 350 and 450; that can be
mathematically written as follows: P(350 < # red < 450) The interval going from 350 to 450 is called the margin of error; it can be
written 400 :l: 50. Given a margin of error, we ﬁnd the probability that the
“real proportion” is in the margin of error by simply adding up the alternative
probability. In our example, we would add the probability that there is 350
red balls in the urn, given that we sampled 40 reds out of a hundred, plus the
probability that there is 351 red balls in the urn, given that we sampled 40 reds
out of a hundred, plus the probability that there is 352 red balls in the urn,
given that we sampled 40 reds out of a hundred, and so on, until we reach 450.
Obviously, it is very long to calculate. To make our lives a little easier, we can
use charts that, basically, contain the answers that we are looking for. IR mean E lower upper limit limit Figure 2: Conﬁdence Interval. However, before using the chart, we must also understand the notion of
conﬁdence interval. Consider this expression: P(350 < # red < 450) = 0.95 The meaning of such an expression is as follows: We can afﬁrm with a certainty
of 95% that the number of red ball in the urn will be between 350 and 450, given
that we have sample 40 red balls out of a hundred.1 We can have an intuitive
idea of what it means by having a 110k at ﬁgure 2. The grey area contains 95%
of the total area under the curve. In our example, the lower limit would be 350,
and the upper limit 450. Thus, when we say that the number of red balls in
the urn is between 350 and 450, 19 times out of 20, we are really just saying
the ratio of they grey area under the curve to the white area under the curve is
19:1.
The chart we will use for our exercises is the following: Conﬁdence Level
0 . 67 0 .95 0.99 10
100
1000 Figure 3: Conﬁdence Interval Chart Sample Size In relation to sampling, the following notions are important: 1Note that 0.95 is not the correct number here. I use it only for the sake of familiarity. 10 o Gambler’s fallacy
o Simpson’s paradox
0 Regression fallacy They are both explained in details in the notes and in Kenyon. Be sure to
understand them carefully. 7 Solutions to the Probability Problems of the
2005 Final Exam 12. How must sample size change to increase the level of conﬁdence
for a ﬁxed margin of error? (Use Fig. 3) The best way to see the answer from the chart is to ﬁnd two entries with the
same number. We have 0.15 that appears twice. If our margin of error is 0.15,
and our sample size is 10, then our level of conﬁdence is 0.67. Moreover, if our
margin of error if 0.15, and our sample size is 100, then our level of conﬁdence
is 0.99. Thus, to increase our level of conﬁdence, the sample size must increase.
13. What is the probability of randomly drawing a king or a queen
or a diamond on a single draw from a standard deck of 52 playing
cards? Since the word “or” is use, we know that we have to ﬁnd the probability of
alternative events. The probability of the alternative three events will thus be
the sum of their individual probability. However, we know that the three events
are not mutually exclusive, since there is a king and a queen of diamond. We
will thus have to subtract the probability of the king of diamond and the queen
of diamond, since their probability will have been included twice in the sum.
Thus: P(K V Q V O) = P(K) + P(Q) + P(O) — P(K&<)) — P(Q&(>)
4 4 13 1 1
=E+E+E—E—E=O.365
14. A jar contains 80 red marbles and 120 blue marbles. What are
the chances of drawing no less than 2 red marbles on 10 random draws
with replacement? Since the marbles are drawn with replacement, we know that the many
drawing are independent from one another. We can thus use the binomial
formula. Also, since the problem contains the phrase “no less than”, we know
that it will be simply to solve if we ﬁnd the probability that the desired event does not happen, and subtract it from 1. That will yield the same answer, but 11 with simpler calculations. Thus, P(no less than 2 reds in 10 draws) is equal to = 1 — P(less than 2 reds in 10 draws) = 1 — P(O or 1 red in 10 draws) = 1 — (P(O red in 10 draws) + P( 1 reds in 10 draws)) 80 80 80 80
W00 — m 1° — 1001 x (—r x (1 — —>9 = 1 — 10 00 X ( 120
1 1 2 1 9
—1 (3) 0 10 3(3) —0.9996
15 a). Using the chart (Fig 3), what sample size is required to be 67%
certain that the observed frequency is within 5% (plus or minus) of
the actual frequency in the population?
The answer is 100. To ﬁnd it, we simply identify the row that corresponds
to 0.05 in the column corresponding to a conﬁdence interval of 0.67.
15 b). Use the chart (Fig 3). Assuming there are 30,000 students at
UWO, and 1,000 are surveyed at random, how conﬁdent can we be
that at least 20,100 are from Ontario if 70% of those surveyed said
they were from Ontario?
We have a sample size equal to 1,000. Now, note that 38:338 : 0.67. Since
.70 — 0.67 = 0.03, our margin of error is 0.03. By using the chart, we thus know that our conﬁdence interval is 0.95. 8 Additional Problems — From Tutorial Example 28 An urn contains 100 balls, 75 of which are red, 15 blue, 5 white,
and 5 green. What is the probability of randomly drawing at least one red ball
if two balls are chosen without replacement. 25 24 31
>=_ ==__._=_z_=_
P(#Red _ 1) 1 P(R 0) 1 100 99 33 0 939 93 9%
Example 29 Suppose that a chessplayer has a 90% chance of winning any
game in a competition.
a} What is the probability that (s)he wins exactly 3 games out of a total of 5 games played?
I P(#Win = 3) = nCrXpT(l—p)"_T = mmsoﬂalo)? = 0.0729 = 7.29% b) Now, suppose that their 5game match end as soon as a player win three
games. What is the probability that {s)he wins the match (i.e. 3 games) in
exactly 5 ga'ues? P(win in 5) = P(win 2 out of4 Ed win the ﬁfth)
4! : m(0.9)2(0.10)2 x 0.9 : 0.04374 z 4.4% 12 c) What is the probability that {s)he wins in exactly three games?
P(3 out of 3) = 0.93 = 0.729 = 72.9% Example 30 How many dices do you have to roll to have at least a 45% chance
of getting one or more 6? For 1 die: 1
P(#6 21): P(#6 : 1) : 6 : 16.7%
For 2 dices:
P(#6 21): 1 — P(#6 : 0): 1 — 2% z 0.3056 m 30.6%
For 3 dices: P(#6 2 1) = 1 — P(#6 = 0) = 1 — (g)3 m 0.421 x 42.1%
For 4 dices:
P(#6 2 1) = 1 — P(#6 = 0) = 1 — (2)4 m 0.518 z 51.8% Thus, the answer is 4. Example 31 Suppose that in our tutorial section there is 40 students. 20 are
fans of the Maple Leafs, 8 are fans of the Canadians, 10 don’t care at all, and
2 is fan of the Nordiques. All of us, of course, are decent enough not to be fans
of the Senators. However, 1 person is fan both of the Nordiques and the Maple
Leafs (anything but the Canadians!). What is the probability that a randomly
selected persons our section) is fan of a team with blue jerseys? The teams with blue jerseys (that have some fans in our section) are the Maple Leafs and the Nordiques). So,
20 2 1 21
40  40—40 40 0.525 52.5% Example 32 Consider the following chart for margin of error: Suppose a ran P(MVN) = P(M)+P(N) —P(M&N) — Conﬁdence Level
0.67 0.95 0.99 Sample Size dom sample of 100 students is taken and 60 prefer to drink beer over water.
a) What is the expected percent of student who prefer to drink beer? 60%
b) How conﬁdent can we be that more than 50% of the students prefer to drink beer over water?
0.95 *Appendix: Bayes7 Theorem and Applications P B B A
P(AB) = p(A)P(BA)+P(—A)P(Bl—A) 13 ...
View
Full
Document
This note was uploaded on 01/13/2012 for the course DCSI 3710 taught by Professor Pavur during the Fall '11 term at North Texas.
 Fall '11
 Pavur

Click to edit the document details