STA3007
Applied probability (0809)
Solution to Assignment 5
Chapter 4
Problem 1.2
Let
X
n
be the number of balls in urn
A
, the state space is
{
0
,
1
,
2
,
3
,
4
,
5
}
.
The
transition matrix is as follows
P
=
0
1
0
0
0
0
1
5
0
4
5
0
0
0
0
2
5
0
3
5
0
0
0
0
3
5
0
2
5
0
0
0
0
4
5
0
1
5
0
0
0
0
1
0
.
By solving the following linear system equations
P
T
π
=
π,
5
i
=0
π
i
= 1
,
we have
π
0
=
π
5
=
1
32
,
π
1
=
π
4
=
5
32
,
π
2
=
π
3
=
10
32
.
Therefore, in the long run, the fraction of time that the urn
A
is empty is
π
0
= 1
/
32 =
0
.
0313.
Problem 1.3
The stationary distribution
π
satisfies the following system equations
π
0
=
α
1
π
0
+
π
1
π
1
=
α
2
π
0
+
π
2
π
2
=
α
3
π
0
+
π
3
π
3
=
α
4
π
0
+
π
4
π
4
=
α
5
π
0
+
π
5
π
5
=
α
6
π
0
5
i
=0
π
i
=
1
,
Representing
π
i
(
i
= 0
,
1
,
2
,
3
,
4
,
5) by
α
and
π
0
from the first six equations, and substi
tuting them into the last equation, we get the limiting probability of being in state 0
1
This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
is
π
0
=
1
∑
6
k
=1
kα
k
.
Problem 1.4
Let
Z
n
= (
X
n
, X
n
+1
), then it is easily to show that
{
Z
n
}
n
≥
0
is a Markov chain. The
question is to find the fraction of the transitions which are from a prescribed state
k
to a
prescribed state
m
(note that they are adjacent). It is the limiting distribution of
This is the end of the preview.
Sign up
to
access the rest of the document.
 Spring '11
 afdsa
 Probability theory, Markov chain, Πi

Click to edit the document details