Discrete-time stochastic processes

# we also omit the rare possibility that the coin

This preview shows page 1. Sign up to view the full content.

This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: tial velocity and angular momentum of the toss, etc.. We also omit the rare possibility that the coin comes to rest on its edge. Sometimes, conversely, we add detail in the interest of structural simplicity, such as the use of a shuﬄed deck of 52 cards above. The choice of the sample space in a probability model is similar to the choice of a mathematical model in any branch of science. That is, one simpliﬁes the physical situation by eliminating essentially irrelevant detail. One often does this in an iterative way, using a very simple model to acquire initial understanding, and then successively choosing more detailed models based on the understanding from earlier models. The mathematical theory of probability views the sample space simply as an abstract set of elements, and from a strictly mathematical point of view, the idea of doing an experiment and getting an outcome is a distraction. For visualizing the correspondence between the theory and applications, however, it is better to view the abstract set of elements as the set of possible outcomes of an idealized experiment in which, when the idealized experiment is performed, one and only one of those outcomes occurs. The two views are mathematically identical, but it will be helpful to refer to the ﬁrst view as a probability model and the 4 CHAPTER 1. INTRODUCTION AND REVIEW OF PROBABILITY second as an idealized experiment. In applied probability texts and technical articles, these idealized experiments, rather than the real-world situations with randomness, often become the primary topic of discussion.3 1.1.2 Assigning probabilities for ﬁnite sample spaces The word probability is widely used in everyday language, and most of us attach various intuitive meanings4 to the word. For example, everyone would agree that something virtually impossible should be assigned a probability close to 0 and something virtually certain should be assigned a probability close to 1. For these special cases, this provides a good rationale for choosing probabilities. The relationship between virtual ly and close to are unclear at the moment, but if there is some implied limiting process, we would all agree that, in the limit, certainty and impossibility correspond to probabilities 1 and 0 respectively. Between virtual impossibility and certainty, if one outcome appears to be closer to certainty than another, its probability should be correspondingly greater. This intuitive notion is imprecise and highly sub jective; it provides little rationale for choosing numerical probabilities for diﬀerent outcomes, and, even worse, little rationale justifying that probability models bear any precise relation to real-world situations. Symmetry can often provide a better rationale for choosing probabilities. For example, the symmetry between H and T for a coin, or the symmetry between the the six faces of a die, motivates assigning equal probabilities, 1/2 each for H and T and 1/6 each for the six faces of a die. This is reasonable and extremely useful, but there is no completely convincing reason for choosing probabilities based on symmetry. Another approach is to perform the experiment many times and choose the probability of each outcome as the relative frequency of that outcome (i.e., the number of occurences of that outcome (or event) divided by the total number of trials). Experience shows that the relative frequency of an outcome often approaches a limiting value with an increasing number of trials. Associating the probability of an outcome with that limiting relative frequency is certainly close to our intuition and also appears to provide a testable criterion between model and real world. This criterion is discussed in Sections 1.5.1 and 1.5.2 to follow. This provides a very concrete way to use probabilities, since it suggests that the randomness in a single trial tends to disappear in the aggregate of many trials. Other approaches to choosing probability models will be discussed later. 3 This is not intended as criticism, since we will see that there are good reasons to concentrate initially on such idealized experiments. However, readers should always be aware that modeling errors are the ma jor cause of misleading results in applications of probability, and thus modeling must be seriously considered before using the results. 4 It is popular to try to deﬁne probability by likelihood, but this is unhelpful since the words are synonyms (except for a technical meaning that likelihood takes on in detection theory). 1.2. THE AXIOMS OF PROBABILITY THEORY 1.2 5 The axioms of probability theory As the applications of probability theory became increasingly varied and complex during the 20th century, it became increasingly necessary to put the theory on a ﬁrm mathematical footing, and this was accomplished by an axiomatization of the theory largely due to the great Russian mathematician A. N. Kolmogorov [14] in 1932. Before stating and explaining these axioms of probability theory, the following two examples explain why the simple approach of the last section, assigning a probability...
View Full Document

## This note was uploaded on 09/27/2010 for the course EE 229 taught by Professor R.srikant during the Spring '09 term at University of Illinois, Urbana Champaign.

Ask a homework question - tutors are online