Discrete-time stochastic processes

46 for each given j in the class we rst show that

Info iconThis preview shows page 1. Sign up to view the full content.

View Full Document Right Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: (t) have the density fY (t) (y ). 3.9. EXERCISES 137 a) Show that these densities are related by the integral equation Zy µ(t + y ) = fY (t) (y ) + µ(t + u)fX (y − u)du. R u=0 µ(t + y )e−ry dy b) Let Lµ,t (r) = y≥0 and let LY (t) (r) and LX (r) be the Laplace transforms of fY (t) (y ) and fX (x) respectively. Find LY (t) (r) as a function of Lµ,t and LX . c) Consider the inter-renewal density fX (x) = (1/2)e−x + e−2x for x ≥ 0 (as in Example 3.3.1). Find Lµ,t (r) and LY (t) (r) for this example. d) Find fY (t) (y ). Show that your answer reduces to that of (3.37) in the limit as t → 1. e) Explain how to go about finding fY (t) (y ) in general, assuming that fX has a rational Laplace transform. Exercise 3.31. Show that for a G/G/1 queue, the time-average wait in the system is the same as limn→1 E [Wn ]. Hint: Consider an integer renewal counting process {M (n); n ≥ 0} where M (n) is the number of renewals in the G/G/1 process of Section 3.6 that have occurred by the nth arrival. Show that this renewal process has a span of 1. Then consider {Wn ; n ≥ 1} as a reward within this renewal process. Exercise 3.32. If one extends the definition of renewal processes to include inter-renewal intervals of duration 0, with Pr {X =0} = α, show that the expected number of simultaneous renewals at a renewal epoch is 1/(1 − α), and that, for a non-arithmetic process, the probability of 1 or more renewals in the interval (t, t + δ ] tends to (1 − α)δ /E [X ] + o(δ ) as t → 1. Exercise 3.33. The purpose of this exercise is to show why the interchange of expectation and sum in the proof of Wald’s equality is justified when E [J ] < 1 but not otherwise. Let X1 , X2 , . . . , be a sequence of IID rv’s, each with the distribution FX . Assume that E [|X |] < 1. a) Show that Sn = X1 + · · · + Xn is a rv for each integer n > 0. Note: Sn is obviously a mapping from the sample space to the real numbers, but you must show that it is finite with probability 1. Hint: Recall the additivity axiom for the real numbers. b) Let J be a stopping time for X1 , X2 , . . . . Show that SJ = X1 + · · · XJ is a rv. Hint: P Represent Pr {SJ } as 1 1 Pr {J = n} Sn . n= c) For the stopping time J above, let J (k) = min(J, k) be the stopping time J truncated to integer k. Explain why the interchange of sum and§ expectation in the proof of Wald’s £ equality is justified in this case, so E [SJ (k) ] = X E J (k) . d) In parts d), e), and f ), assume, in addition to the assumptions above, that FX (0) = 0, i.e., that the Xi are positive rv’s. Show that limk→1 E [SJ (k) ] < 1 if E [J ] < 1 and limk→1 E [SJ (k) ] = 1 if E [J ] = 1. e) Show that Pr {SJ (k) > x} ≤ Pr {SJ > x} 138 CHAPTER 3. RENEWAL PROCESSES for all k, x. f ) Show that E [SJ ] = X E [J ] if E [J ] < 1 and E [SJ ] = 1 if E [J ] = 1. g) Now assume that X has both negative and positive values with nonzero probability and P + − + let X + = max(0, X ) and X − = min(X, 0). Express SJ as SJ + SJ where SJ = J=1 Xi+ i P − and SJ = J=1 Xi− . Show that E [SJ ] = X E [J ] if E [J ] < 1 and that E [Sj ] is undefined i otherwise. Chapter 4 FINITE-STATE MARKOV CHAINS 4.1 Introduction The counting processes {N (t), t ≥ 0} of Chapterss 2 and 3 have the property that N (t) changes at discrete instants of time, but is defined for all real t ≥ 0. Such stochastic processes are generally called continuous time processes. The Markov chains to be discussed in this and the next chapter are stochastic processes defined only at integer values of time, n = 0, 1, . . . . At each integer time n ≥ 0, there is an integer valued random variable (rv) Xn , called the state at time n, and the process is the family of rv’s {Xn , n ≥ 0}. These processes are often called discrete time processes, but we prefer the more specific term integer time processes. An integer time process {Xn ; n ≥ 0} can also be viewed as a continuous time process {X (t); t ≥ 0} by taking X (t) = Xn for n ≤ t < n + 1, but since changes only occur at integer times, it is usually simpler to view the process only at integer times. In general, for Markov chains, the set of possible values for each rv Xn is a countable set usually taken to be {0, 1, 2, . . . }. In this chapter (except for Theorems 4.2 and 4.3), we restrict attention to a finite set of possible values, say {1, . . . , M}. Thus we are looking at processes whose sample functions are sequences of integers, each between 1 and M. There is no special significance to using integer labels for states, and no compelling reason to include 0 as a state for the countably infinite case and not to include 0 for the finite case. For the countably infinite case, the most common applications come from queueing theory, and the state often represents the number of waiting customers, which can be zero. For the finite case, we often use vectors and matrices, and it is more conventional to use positive integer labels. In some exampl...
View Full Document

This note was uploaded on 09/27/2010 for the course EE 229 taught by Professor R.srikant during the Spring '09 term at University of Illinois, Urbana Champaign.

Ask a homework question - tutors are online