This preview shows page 1. Sign up to view the full content.
Unformatted text preview: (t) have the density fY (t) (y ). 3.9. EXERCISES 137 a) Show that these densities are related by the integral equation
Zy
µ(t + y ) = fY (t) (y ) +
µ(t + u)fX (y − u)du.
R u=0 µ(t + y )e−ry dy b) Let Lµ,t (r) = y≥0
and let LY (t) (r) and LX (r) be the Laplace transforms
of fY (t) (y ) and fX (x) respectively. Find LY (t) (r) as a function of Lµ,t and LX .
c) Consider the interrenewal density fX (x) = (1/2)e−x + e−2x for x ≥ 0 (as in Example
3.3.1). Find Lµ,t (r) and LY (t) (r) for this example.
d) Find fY (t) (y ). Show that your answer reduces to that of (3.37) in the limit as t → 1.
e) Explain how to go about ﬁnding fY (t) (y ) in general, assuming that fX has a rational
Laplace transform.
Exercise 3.31. Show that for a G/G/1 queue, the timeaverage wait in the system is the
same as limn→1 E [Wn ]. Hint: Consider an integer renewal counting process {M (n); n ≥ 0}
where M (n) is the number of renewals in the G/G/1 process of Section 3.6 that have
occurred by the nth arrival. Show that this renewal process has a span of 1. Then consider
{Wn ; n ≥ 1} as a reward within this renewal process.
Exercise 3.32. If one extends the deﬁnition of renewal processes to include interrenewal
intervals of duration 0, with Pr {X =0} = α, show that the expected number of simultaneous renewals at a renewal epoch is 1/(1 − α), and that, for a nonarithmetic process, the
probability of 1 or more renewals in the interval (t, t + δ ] tends to (1 − α)δ /E [X ] + o(δ ) as
t → 1.
Exercise 3.33. The purpose of this exercise is to show why the interchange of expectation
and sum in the proof of Wald’s equality is justiﬁed when E [J ] < 1 but not otherwise.
Let X1 , X2 , . . . , be a sequence of IID rv’s, each with the distribution FX . Assume that
E [X ] < 1.
a) Show that Sn = X1 + · · · + Xn is a rv for each integer n > 0. Note: Sn is obviously
a mapping from the sample space to the real numbers, but you must show that it is ﬁnite
with probability 1. Hint: Recall the additivity axiom for the real numbers.
b) Let J be a stopping time for X1 , X2 , . . . . Show that SJ = X1 + · · · XJ is a rv. Hint:
P
Represent Pr {SJ } as 1 1 Pr {J = n} Sn .
n= c) For the stopping time J above, let J (k) = min(J, k) be the stopping time J truncated
to integer k. Explain why the interchange of sum and§ expectation in the proof of Wald’s
£
equality is justiﬁed in this case, so E [SJ (k) ] = X E J (k) .
d) In parts d), e), and f ), assume, in addition to the assumptions above, that FX (0) = 0,
i.e., that the Xi are positive rv’s. Show that limk→1 E [SJ (k) ] < 1 if E [J ] < 1 and
limk→1 E [SJ (k) ] = 1 if E [J ] = 1.
e) Show that
Pr {SJ (k) > x} ≤ Pr {SJ > x} 138 CHAPTER 3. RENEWAL PROCESSES for all k, x.
f ) Show that E [SJ ] = X E [J ] if E [J ] < 1 and E [SJ ] = 1 if E [J ] = 1.
g) Now assume that X has both negative and positive values with nonzero probability and
P
+
−
+
let X + = max(0, X ) and X − = min(X, 0). Express SJ as SJ + SJ where SJ = J=1 Xi+
i
P
−
and SJ = J=1 Xi− . Show that E [SJ ] = X E [J ] if E [J ] < 1 and that E [Sj ] is undeﬁned
i
otherwise. Chapter 4 FINITESTATE MARKOV
CHAINS
4.1 Introduction The counting processes {N (t), t ≥ 0} of Chapterss 2 and 3 have the property that N (t)
changes at discrete instants of time, but is deﬁned for all real t ≥ 0. Such stochastic
processes are generally called continuous time processes. The Markov chains to be discussed
in this and the next chapter are stochastic processes deﬁned only at integer values of time,
n = 0, 1, . . . . At each integer time n ≥ 0, there is an integer valued random variable
(rv) Xn , called the state at time n, and the process is the family of rv’s {Xn , n ≥ 0}.
These processes are often called discrete time processes, but we prefer the more speciﬁc
term integer time processes. An integer time process {Xn ; n ≥ 0} can also be viewed as a
continuous time process {X (t); t ≥ 0} by taking X (t) = Xn for n ≤ t < n + 1, but since
changes only occur at integer times, it is usually simpler to view the process only at integer
times.
In general, for Markov chains, the set of possible values for each rv Xn is a countable set
usually taken to be {0, 1, 2, . . . }. In this chapter (except for Theorems 4.2 and 4.3), we
restrict attention to a ﬁnite set of possible values, say {1, . . . , M}. Thus we are looking at
processes whose sample functions are sequences of integers, each between 1 and M. There is
no special signiﬁcance to using integer labels for states, and no compelling reason to include
0 as a state for the countably inﬁnite case and not to include 0 for the ﬁnite case. For
the countably inﬁnite case, the most common applications come from queueing theory, and
the state often represents the number of waiting customers, which can be zero. For the
ﬁnite case, we often use vectors and matrices, and it is more conventional to use positive
integer labels. In some exampl...
View
Full
Document
This note was uploaded on 09/27/2010 for the course EE 229 taught by Professor R.srikant during the Spring '09 term at University of Illinois, Urbana Champaign.
 Spring '09
 R.Srikant

Click to edit the document details