31
A.2. SOLUTIONS FOR CHAPTER 2
d) Find the density of the interarrival time for cfw_NA (t); t
0. Note: This part is quite messy and is done
most easily via Laplace transforms.
Solution: The process c
30
APPENDIX A. SOLUTIONS TO EXERCISES
where
Z
m(t) =
t
[1
G( )] d =
0
Z
t
F(a/ ) d.
0
Finally, note that the answer is the same if V is discrete or mixed.
Exercise 2.23: Let cfw_N1 (t); t > 0 be a Poi
29
A.2. SOLUTIONS FOR CHAPTER 2
1
X
n=i
f(si |n)p(n) =
=
=
1
si 1 X (t si )n
i
(i i)!
(n i)!
i si 1 e
i
(i
n=i
si
i)!
i si 1 e
i
(i
1
X
i
n
n i (t
i)!
t
si )n i e
(n i)!
n=i
si
e
(t si )
.
This is the
28
APPENDIX A. SOLUTIONS TO EXERCISES
PMF of the number waiting at time t and take the limit of this PMF as t ! 1. For very
large t, the number M of combined arrivals before t is large with high proba
A.2. SOLUTIONS FOR CHAPTER 2
27
in (A.12). In other words, Nm is independent of the arrival time of bus m 1. From the
formula in (A.12), the PMF of the number entering a bus is also independent of m.
26
APPENDIX A. SOLUTIONS TO EXERCISES
Solution: Let the sample value of N1 + N2 be n12 . By the same argument in (c),
n
pN1 +N2 |N (n12 |n) =
(p1 + p2 )n12 (1 p1 p2 )n n12 .
n12
e) Find the conditio
23
A.2. SOLUTIONS FOR CHAPTER 2
Solution: There are 2t binary n-tuples and each has a probability, so the joint PMF can
be viewed as a vector of 2t numbers. The binomial probability pN ( ) (k) = q k (
24
APPENDIX A. SOLUTIONS TO EXERCISES
Taking the limit as m ! 1, each of the n terms in the product approaches 1, so the limit
is 1/n!, verifying the rst equality in (c). For the second,
1
t
m
m
n
t
=
25
A.2. SOLUTIONS FOR CHAPTER 2
e
Solution: Again expressing N (t+s) = N (t) + N (t, t+s),
h
i
e
E [N (t) N (t+s)] = E N 2 (t) + E N (t)N (t, t+s)
= E N 2 (t) + E [N (t)] E [N (s)]
=
t+
t + t s.
2 2
I
22
APPENDIX A. SOLUTIONS TO EXERCISES
Your solution should contain four 3-tuples with probability 1/8 each, two 3-tuples with probability 1/4 each,
and two 3-tuples with probability 0. Note that by ma
18
APPENDIX A. SOLUTIONS TO EXERCISES
Gaussian with mean 0 and standard deviation 103 . Since one or more occurrences of 1012
occur only with probability 10 4 , this can be neglected in the sketch, so
19
A.1. SOLUTIONS FOR CHAPTER 1
Exercise 1.48: Let cfw_Yn ; n
cfw_Yn ; n
1 be a sequence of rvs and assume that limn!1 E [|Yn |] = 0. Show that
1 converges to 0 in probability. Hint 1: Look for the ea
17
A.1. SOLUTIONS FOR CHAPTER 1
Exercise 1.44: Let X1 , X2 . . . be a sequence of IID rvs each with mean 0 and variance
p
Sn = X1 + + Xn for all n and consider the random variable Sn / n
2
. Let
p
S2n
15
A.1. SOLUTIONS FOR CHAPTER 1
Solution:
n
=
k+1
n!
n!
=
(k + 1)!(n k 1)!
k!(k + 1)(n k)!/(n
n n k
n n k
=
.
k k+1
k
k
b) Extend (a) to show that, for all ` n
k,
!
n
k+`
!
n
n k
k
k
Solution: Usi
16
APPENDIX A. SOLUTIONS TO EXERCISES
follows. Replacing the upper bound in (A.8) with the asymptotic inequality in (A.9), we
get Prcfw_Sn k ppqp pSn (k).
Exercise 1.39: Let cfw_Xi ; i
1 be IID binary
14
APPENDIX A. SOLUTIONS TO EXERCISES
Solution: The mean value of Ij (x) is FX (x) and the variance (after a short calculation) is
FX (x)Fc (x). This is nite (and in fact at most 1/4), so Theorem 1.7.
12
0 as z
nite.
APPENDIX A. SOLUTIONS TO EXERCISES
2
as z ! 1, so the integral is nite. Thus r+ belongs to the region where gZ (r) is
The whole point of this is that the random variables for which r+
9
A.1. SOLUTIONS FOR CHAPTER 1
Only one or two lines of explanation are needed.
Solution: We have shown that X1 + X2 is a rv, so (X1 + X2 ) + X3 is a rv, etc.
Exercise 1.15: (Stieltjes integration) a)
2
A.1
APPENDIX A. SOLUTIONS TO EXERCISES
Solutions for Chapter 1
Exercise 1.2: This exercise derives the probability of an arbitrary (non-disjoint) union of events, derives
the union bound, and derive
59
A.4. SOLUTIONS FOR CHAPTER 4
Solution: The ergodicity of k and k 0 assures the inherently reachable assumption of Theorem 4.6.8, and thus we know from the fact that k satises the termination condit
57
A.4. SOLUTIONS FOR CHAPTER 4
of nal cost going from 1 to 0. If George is in the drive state (2) and uses the try policy,
r2 = 0 and with probability 1/2, he parks and has one unit nal cost getting
60
A.5
APPENDIX A. SOLUTIONS TO EXERCISES
Solutions for Chapter 5
Exercise 5.1: The purpose of this exercise is to show that for an arbitrary renewal process, N (t), the
number of renewals in (0, t] i
58
APPENDIX A. SOLUTIONS TO EXERCISES
(k0 )
(k )
Solution: For all i 2 R0 , Pij i = Pij i . The steady state vector is determined solely by
0
0
the transition probabilities in the recurrent class, so
55
A.4. SOLUTIONS FOR CHAPTER 4
Solution:
p1
: n
0 H
Y
p0
0
01
010
p1
p0
z
X n
z
X n
z
X n 1
1
2
3 X
y
p1
p0
The solution for v0 and v1 in terms of v2 is the same as (a). The basic equation for v2 i
54
APPENDIX A. SOLUTIONS TO EXERCISES
Finally, state 2 is the trapping state, so v2 = 0 and v0 = 1/p0 p1 .
b) For parts b) to d), let (a1 , a2 , a3 , . . . , ak ) = (0, 1, 1, . . . , 1), i.e., zero fo
56
APPENDIX A. SOLUTIONS TO EXERCISES
c) Let r be an arbitrary reward vector and consider the equation
w + g (1) (1) + g (2) (2) = r + [P ]w .
(A.19)
Determine what values g (1) and g (2) must have in
52
APPENDIX A. SOLUTIONS TO EXERCISES
Solution: With arbitrary P12 , the steady state probabilities become
1 =
0.01
;
P12 + 0.01
2 =
P12
.
P12 + 0.01
The steady state gain, g = r , then becomes g = P1
51
A.4. SOLUTIONS FOR CHAPTER 4
b) Suppose one has found the expected rst-passage-times vj for states j = 2 to 4 (or in general from 2 to
M). Find an expression for v1 , the expected rst recurrence ti
47
A.4. SOLUTIONS FOR CHAPTER 4
From the rst equation, 2 = 0 and from the third 3 = 0, so 2 = 1 is the right eigenvector,
unique within a scale factor.
d) Use (c) to show that there is no diagonal mat
53
A.4. SOLUTIONS FOR CHAPTER 4
Solution: We have already seen that v1 (n) = 0 for all n, and thus, since P23 = 0, we have
v2 (n) = 1 + (1/2)v2 (n 1). Since v2 (0) = 0, this iterates to
v2 (n) = 1 +
1