Solution to Mid-term Examination
1. (a) The Markov matrix P is an (N + 1) (N + 1) matrix as follows:
01
0
0
1 0 N 1 0
N
N
.
. .
.
. . . .
.
.
.
. ,
.
N i
i
.
0
.
.N
N
. .
.
. . . .
.
.
.
1
0 N 1
0
N
N
0
0
1
0
where P0,1 = 1, PN,N 1 = 1 and when 0
Solution to Quiz 2
1. (a) Since all the ospring distributions have the same distributions ,
we have
E1 (Xn |Xn1 = k ) = E (k ) = kE ( ) = k.
(b) Let us prove Ex (Xn ) = n x. When n = 0, obviously. Suppose the
argument holds for n 1, then
Ex (Xn |Xn1 = k )
Solution to Quiz 3
1. (a)
P (X (t) = n|X (r) = m)
P (X (t) = n, X (r) = m)
=
P (X (r) = m)
P (X (t) X (r) = n m, X (r) = m)
=
P (X (r) = m)
=P (X (t) X (r) = n m)
=P (X (t r) = n m)
=
(t r)nm e(tr)
.
(n m)!
(b)
P (X (r) = m|X (t) = n)
P (X (t) = n, X (r)
Solution to Quiz 1
1. (a) Let the airport A be state 0, hotel B be state 1 and hotel C be
state 2. By what is stated in question, P (0, 0) = 0, P (0, 1) =
4
1
4
P (0, 2) = 1 ; P (1, 0) = 5 , P (1, 1) = 0, P (1, 2) = 5 ; P (2, 0) = 5 ,
2
1
P (2, 1) = 5 , P
Solution to Assignment 5
1 (p.107). D =
. The eigenvalue of D are 0 and ( + ) with its cor1
1
responding eigenvector are
and
respectively. Write Q =
1
1
.
It is easy to see that
Q1 DQ =
0
0
0 ( + )
.
Hence,
P (t) = eDt =
n=0
(Dt)n
=Q
n!
=
1
0
(+)t
0e
Solution to Assignment 2
20a (p.44). By a rearrangement of states, we have the following
0
1
2
4
3
5
012435
1
1
0000
2
2
1
2
0000
3
3
1
008700
8
3
004100
4
1
1
0101
4
4
4
4
1
010152
5
5
5
We see that cfw_0, 1 and cfw_2, 4 are nite irreducible closed sets,
Solution to Assignment 1
2 (p.41). Note that the state space is cfw_0, 1, , d. Denote Xn the number of black
balls in box 1. Recall that we want to nd
P (x, y ) = P(Xn+1 = y |Xn = x).
If x = 0, then y = 1. In this case, it is easy to see that P (x, y ) =
Remark on tutorial 4
Im sorry that I made a mistake in Exercise 3 in the tutorial, some students point out
that we should tackle the boundary term more carefully. It is right. The following is a
complete solution.
Ex3 Consider now the Gambler Ruin Problem
The Chinese
University o
Hong Kong
places very h
importance o
honesty in
academic wo
submitted by
students, and
adopts a pol
zero toleran
cheating in
examination
plagiarism. A
related offen
will lead to
disciplinary
action includ
termination
studies at t