This preview shows page 1. Sign up to view the full content.
Unformatted text preview: tial velocity and angular
momentum of the toss, etc.. We also omit the rare possibility that the coin comes to rest
on its edge. Sometimes, conversely, we add detail in the interest of structural simplicity,
such as the use of a shuﬄed deck of 52 cards above.
The choice of the sample space in a probability model is similar to the choice of a mathematical model in any branch of science. That is, one simpliﬁes the physical situation by
eliminating essentially irrelevant detail. One often does this in an iterative way, using a
very simple model to acquire initial understanding, and then successively choosing more
detailed models based on the understanding from earlier models.
The mathematical theory of probability views the sample space simply as an abstract set of
elements, and from a strictly mathematical point of view, the idea of doing an experiment
and getting an outcome is a distraction. For visualizing the correspondence between the
theory and applications, however, it is better to view the abstract set of elements as the set
of possible outcomes of an idealized experiment in which, when the idealized experiment is
performed, one and only one of those outcomes occurs. The two views are mathematically
identical, but it will be helpful to refer to the ﬁrst view as a probability model and the 4 CHAPTER 1. INTRODUCTION AND REVIEW OF PROBABILITY second as an idealized experiment. In applied probability texts and technical articles, these
idealized experiments, rather than the real-world situations with randomness, often become
the primary topic of discussion.3 1.1.2 Assigning probabilities for ﬁnite sample spaces The word probability is widely used in everyday language, and most of us attach various
intuitive meanings4 to the word. For example, everyone would agree that something virtually impossible should be assigned a probability close to 0 and something virtually certain
should be assigned a probability close to 1. For these special cases, this provides a good
rationale for choosing probabilities. The relationship between virtual ly and close to are unclear at the moment, but if there is some implied limiting process, we would all agree that,
in the limit, certainty and impossibility correspond to probabilities 1 and 0 respectively.
Between virtual impossibility and certainty, if one outcome appears to be closer to certainty
than another, its probability should be correspondingly greater. This intuitive notion is imprecise and highly sub jective; it provides little rationale for choosing numerical probabilities
for diﬀerent outcomes, and, even worse, little rationale justifying that probability models
bear any precise relation to real-world situations.
Symmetry can often provide a better rationale for choosing probabilities. For example, the
symmetry between H and T for a coin, or the symmetry between the the six faces of a die,
motivates assigning equal probabilities, 1/2 each for H and T and 1/6 each for the six faces
of a die. This is reasonable and extremely useful, but there is no completely convincing
reason for choosing probabilities based on symmetry.
Another approach is to perform the experiment many times and choose the probability of
each outcome as the relative frequency of that outcome (i.e., the number of occurences
of that outcome (or event) divided by the total number of trials). Experience shows that
the relative frequency of an outcome often approaches a limiting value with an increasing
number of trials. Associating the probability of an outcome with that limiting relative
frequency is certainly close to our intuition and also appears to provide a testable criterion
between model and real world. This criterion is discussed in Sections 1.5.1 and 1.5.2 to
follow. This provides a very concrete way to use probabilities, since it suggests that the
randomness in a single trial tends to disappear in the aggregate of many trials. Other
approaches to choosing probability models will be discussed later. 3 This is not intended as criticism, since we will see that there are good reasons to concentrate initially
on such idealized experiments. However, readers should always be aware that modeling errors are the ma jor
cause of misleading results in applications of probability, and thus modeling must be seriously considered
before using the results.
It is popular to try to deﬁne probability by likelihood, but this is unhelpful since the words are synonyms
(except for a technical meaning that likelihood takes on in detection theory). 1.2. THE AXIOMS OF PROBABILITY THEORY 1.2 5 The axioms of probability theory As the applications of probability theory became increasingly varied and complex during
the 20th century, it became increasingly necessary to put the theory on a ﬁrm mathematical
footing, and this was accomplished by an axiomatization of the theory largely due to the
great Russian mathematician A. N. Kolmogorov  in 1932. Before stating and explaining
these axioms of probability theory, the following two examples explain why the simple
approach of the last section, assigning a probability...
View Full Document
This note was uploaded on 09/27/2010 for the course EE 229 taught by Professor R.srikant during the Spring '09 term at University of Illinois, Urbana Champaign.
- Spring '09