This preview shows page 1. Sign up to view the full content.
Unformatted text preview: OUNTABLE STATE SPACES c) Verify that the above hypothesis is correct.
d) Find an expression for π0 .
e) Find an expression for the steady state probability that an arriving customer is discarded. Chapter 7 RANDOM WALKS, LARGE
DEVIATIONS, AND
MARTINGALES
7.1 Introduction Deﬁnition 7.1. Let {Xi ; i ≥ 1} be a sequence of IID random variables, and let Sn =
X1 + X2 + · · · + Xn . The integertime stochastic process {Sn ; n ≥ 1} is cal led a random
walk, or, more precisely, the random walk based on {Xi ; i ≥ 1}.
For any given n, Sn is simply a sum of IID random variables, but here the behavior of
the entire random walk process, {Sn ; n ≥ 1}, is of interest. Thus, for a given real number
α > 0, we might try to ﬁnd the probabiity that Sn ≥ α for any n, or given that Sn ≥ α
for one or more values of n, we might want to ﬁnd the distribution of the smallest n such
that Sn ≥ α. typical questions about random walks are ﬁnding the smallest n such that
Sn reaches or exceeds a threshold, and ﬁnding the probability that the threshold is ever
reached or crossed.
Since Sn tends to drift downward with increasing n if E [X ] = X < 0, and tends to drift
upward if X > 0, the results to be obtained depend critically on whether X < 0, X > 0,
or X = 0. Since results for X < 0 can be easily translated into results for X > 0 by
considering {−Sn ; n ≥ 0}, we will focus on the case X < 0. As one might expect, both the
results and the techniques have a very diﬀerent ﬂavor when X = 0, since here the random
walk does not drift but typically wanders around in a rather aimless fashion.
The following three subsections discuss three special cases of random walks. The ﬁrst two,
simple random walks and integer random walks, will be useful throughout as examples,
since they can be easily visualized and analyzed. The third special case is that of renewal
processes, which we have already studied and which will provide additional insight into the
general study of random walks.
After this, Sections 3.2 and 3.3 show how two ma jor application areas, G/G/1 queues and
279 280 CHAPTER 7. RANDOM WALKS, LARGE DEVIATIONS, AND MARTINGALES hypothesis testing, can be treated in terms of random walks. These sections also show why
questions related to threshold crossings are so important in random walks.
Section 3.4 then develops the theory of threshold crossing for general random walks and
Section 3.5 extends and in many ways simpliﬁes these results through the use of stopping
rules and a powerful generalization of Wald’s equality known as Wald’s identity.
The remainder of the chapter is devoted to a rather general type of stochastic process called
martingales. The topic of martingales is both a sub ject of interest in its own right and also
a tool that provides additional insight into random walks, laws of large numbers, and other
basic topics in probability and stochastic processes. 7.1.1 Simple random walks Suppose X1 , X2 , . . . are IID binary random variables, each taking on the value 1 with
probability p and −1 with probability q = 1 − p. Letting Sn = X1 + · · · + Xn , the sequence
of sums {Sn ; n ≥ 1}, is called a simple random walk. Sn is the diﬀerence between positive
and negative occurrences in the ﬁrst n trials. Thus, if there are j positive occurrences for
0 ≤ j ≤ n, then Sn = 2j − n, and
Pr {Sn = 2j − n} = n!
pj (1 − p)n−j .
j !(n − j )! (7.1) This distribution allows us to answer questions about Sn for any given n, but it is not very
helpful in answering such questions as the following: for any given integer k > 0, what is
the probability that the sequence S1 , S2 , . . . ever reaches or exceeds k? This probability can
S
be expressed as1 Pr { 1 {Sn ≥ k}} and is referred to as the probability that the random
n=1
walk crosses a threshold at k. Exercise 7.1 demonstrates the surprisingly simple result that
for a simple random walk with p < 1/2, this threshold crossing probability is
(1
)µ
∂k
[
p
Pr
{Sn ≥ k} =
.
(7.2)
1−p
n=1
Sections 7.4 and 7.5 treat this same question for general random walks. They also treat
questions such as the overshoot given a threshold crossing, the time at which the threshold
is crossed given that it is crossed, and the probability of crossing such a positive threshold
before crossing a given negative threshold. 7.1.2 Integervalued random walks Suppose next that X1 , X2 , . . . are arbitrary IID integervalued random variables. We can
again ask for the probability that such an integer valued random walk crosses a threshold
1 This same probability is often expressed as as Pr {sup1 Sn ≥ k}. For a general random walk, the
n=1
S
event n≥1 {Sn ≥ k} is slightly diﬀerent from supn≥1 Sn ≥ k. The latter event can include sample sequences
s1 , s2 , . . . in which a subsequence of values sn approach k as a limit but never quite reach k. This is
impossible for a simple random walk since all sk must be integers. It is possible, but can be shown to have
probability zero, for general random walks. We will avoid this sillin...
View
Full
Document
This note was uploaded on 09/27/2010 for the course EE 229 taught by Professor R.srikant during the Spring '09 term at University of Illinois, Urbana Champaign.
 Spring '09
 R.Srikant

Click to edit the document details