Discrete-time stochastic processes

Third by using demorgans law h ic an ac n n n it is

Info iconThis preview shows page 1. Sign up to view the full content.

View Full Document Right Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: d if it were omitted. Second, by making all but a finite number of events in a union be the empty event, it is clear that all finite unions of events are events. Third, by using deMorgan’s law, h[ ic \ An = Ac . n n n it is clear that finite and countable intersections of events are events. Although we will not make a big fuss about these axioms in the rest of the text, we will be careful to use only complements and countable unions and intersections in our analysis. Thus subsets that are not events will not arise and we need not concern ourselves with them. Note that the axioms do not say that all subsets of ≠ are events. In fact, there are many rather silly ways to define classes of events that obey the axioms. For example, the axioms 7 The mathematical name for a class of elements satisfying these axioms is a σ -algebra, sometimes called a σ -field. 1.2. THE AXIOMS OF PROBABILITY THEORY 7 are satisfied by choosing only the universal set ≠ and the empty set φ to be events. We shall avoid such trivialities by assuming that for each sample point ω , the singleton subset {ω } is an event. For finite sample spaces, this assumption, plus the axioms above, imply that all subsets are events for finite sample spaces. For uncountably infinite sample spaces, such as the sinusoidal phase above, this assumption, plus the axioms above, still leaves considerable freedom in choosing a class of events. As an example, the class of all subsets of ≠ satisfies the axioms but surprisingly does not allow the probability axioms to be satisfied in any sensible way. How to choose an appropriate class of events requires an understanding of measure theory which would take us too far afield for our purposes. Thus we neither assume nor develop measure theory here.8 From a pragmatic standpoint, we start with the class of events of interest, such as those required to define the random variables needed in the problem. That class is then extended so as to be closed under complementation and countable unions. Measure theory shows that this extension can always be done, but here we simply accept that extension as a known result. 1.2.2 Axioms of probability Given any sample space ≠ and any class of events satisfying the axioms of events, the following three probability axioms9 hold: 1. Pr {≠} = 1. 2. For every event A, Pr {A} ≥ 0. 3. For every sequence A1 , A2 , . . . of disjoint events, the probability of their union is n [1 o X1 Pr An = Pr {An } , (1.1) n=1 P1 where n=1 Pr {An } n=1 is shorthand for limm→1 Pm The axioms imply the following useful corollaries: n=1 Pr {An }. Pr {φ} = 0 o Xm An = Pr {An } for disjoint events A1 , . . . , Am (1.3) for all A (1.4) Pr {A} ≤ Pr {B } Pr n[m for all A, B such that A ⊆ B (1.5) for finite or countable disjoint events. (1.7) n=1 n=1 c Pr {A } = 1 − Pr {A} X n 8 Pr {A} ≤ 1 Pr {An } ≤ 1 (1.2) for all A (1.6) There is no doubt that measure theory is useful in probability theory, and serious students of probability should certainly learn measure theory at some point. For application-oriented people, however, it seems advisable to acquire more insight and understanding of probability, at a graduate level, before concentrating on the abstractions and subtleties of measure theory. 9 Sometimes finite additivity, (1.3) is added as an additional axiom. This might aid intuition and also avoids the very technical proofs given for (1.2) and (1.3). 8 CHAPTER 1. INTRODUCTION AND REVIEW OF PROBABILITY To verify (1.2), note that events are disjoint if they have no outcomes in common, and thus the empty event φ is disjoint from itself and every other event. Thus, φ, φ, . . . , is a sequence of disjoint events, and it follows from Axiom 3 (see Exercise 1.1) that Pr {φ} = 0. To verify (1.3), apply Axiom 3 to the disjoint sequence A1 , . . . , Am , φ, φ, . . . . One might reasonably guess that (1.3), along with Axioms 1 and 2 implies Axiom 3. Exercise 1.3 shows why this guess is incorrect. S To verify (1.4), note that ≠ = A Ac . Then apply (1.3) to the disjoint sets A and Ac . S To verify (1.5), note that if A ⊆ F , then F = A (F Ac ) where10 A and F Ac are disjoint. S Apply (1.3) to A (F Ac ), (1.5) and note that Pr {F Ac } ≥ 0. S To verify (1.6) and (1.7), first substitute ≠ for F in (1.5) and then substitute n An for A. The axioms specify the probability of any disjoint union of events in terms of the individual event probabilities, but what about a finite or countable union of arbitrary events A1 , A2 , . . . ? Exercise 1.4 shows that in this case, (1.1) or (1.3) can be generalized to n[ oX n [n−1 o Pr An = Pr An − Ai . (1.8) n n i=1 In order to use this, one must know not only the event probabilities for A1 , A2 . . . , but also the probabilities of their intersections. The union bound, which is also derived in Exercise 1.4 depends only on the individual event probabilities, but gives only the following upper bound on the union probability. n[ oX Pr An ≤ Pr...
View Full Document

This note was uploaded on 09/27/2010 for the course EE 229 taught by Professor R.srikant during the Spring '09 term at University of Illinois, Urbana Champaign.

Ask a homework question - tutors are online