Chapter 2 Foundation of Probability Theory
Probability theory is the foundation of statistical science, providing a mathematical
means of modeling random experiments or uncertainty. Through these mathematical models,
researchers are able to draw inferences about the random experiments using observed data.
The aim of this chapter is to outline the basic ideas of probability theory that are fundamental
to the study of statistics. The entire structure of probability, and therefore of statistics, can be
built on the relatively straightforward foundation given in this chapter.
Random experiment, Sample space, Event, Borel ﬁeld, Sigma algebra, Probability
function, Probability space, Sets, Union, Intersection, Complement, Permutation, Combination,
Bayes’ theorem, Conditional probability, Independence, Multiplication rule.
2.1 Random Experiments
Many kinds of scientiﬁc research may be characterized in part by the fact that repeated
experiments, under more or less the same conditions, are standard practice. For instance, an
economist may be concerned with the prices of three speciﬁed commodities at various time
periods. The only way in which a researcher can elicit information about such a phenomenon
is to perform repeated experiments. Each experiment terminates with an outcome. But it is
characteristic of these experiments that the outcome cannot be predicted with certainty prior to
the performance of the experiment, although the experiment is of such a nature that a collection
of every possible outcome can be described prior to its performance.
. [Random Experiment]:
A random experiment is a mechanism which has
at least two possible outcomes, and which outcome to occur is unknown in advance. In other
words, a random experiment is a mechanism for which the outcome cannot be predicted with
The word “experiment” here means a process of observation or measurement in a
broad sense. It is not necessarily a real experiment as encountered in (e.g.) Physics.
There are two essential elements of a random experiment:
the set of all possible outcomes;
the likelihood with which each outcome will occur.
As is well-known, modern economics is a study on resource allocation in an uncertain envi-
ronment. When an economic agent makes a decision, he or she usually does not know precisely
the outcome of his or her action, which usually will arise in an uncertain manner. This, to some
extent, is similar to a random experiment.
Modern econometrics is built upon the following two fundamental axioms: