ORIE 3510 – Homework 3 Solutions
Instructor: Mark E. Lewis
due 2PM, Wednesday February 15, 2012 (ORIE Hallway drop box)
1. (a)
{
X
n
}
is not a Markov chain. To see this, it sufces to check that
P
(
X
4
= 1

X
3
= 0
, X
2
= 1)
n
=
P
(
X
4
= 1

X
3
= 0
, X
2
=
−
1)
.
Indeed,
P
(
X
4
= 1

X
3
= 0
, X
2
= 1) =
P
(min(0
,
1) +
ǫ
4
= 1
, X
3
= 0
, X
2
= 1)
P
(
X
3
= 0
, X
2
= 1)
=
P
(
ǫ
4
= 1) =
1
2
,
but
P
(
X
4
= 1

X
3
= 0
, X
2
=
−
1) =
P
(min(0
,
−
1) +
ǫ
4
= 1
, X
3
= 0
, X
2
=
−
1)
P
(
X
3
= 0
, X
2
=
−
1)
=
P
(
ǫ
4
= 2) = 0
.
(b) De±ne
Y
n
= (
X
n
, X
n

1
),
n
= 1
,
2
, . . .
, then the sequence
Y
n
is a Markov chain. To prove
this, we see that
P
(
Y
n
= (
i
n
, i
n

1
)

Y
n

1
= (
i
n

1
, i
n

2
)
, . . . , Y
1
= (0
,
0))
=
P
(
X
n
=
i
n
, X
n

1
=
i
n

1

X
n

1
=
i
n

1
, X
n

2
=
i
n

2
, . . . , X
1
= 0
, X
0
= 0)
=
P
(min(
i
n

1
, i
n

2
) +
ǫ
n
=
i
n
, X
n

1
=
i
n

1
, . . . , X
0
= 0)
P
(
X
n

1
=
i
n

1
, . . . , X
0
= 0)
=
P
(
ǫ
n
=
i
n
−
min(
i
n

1
, i
n

2
))
=
P
(min(
i
n

1
, i
n

2
) +
ǫ
n
=
i
n
, X
n

1
=
i
n

1
, X
n

2
=
i
n

2
)
P
(
X
n

1
=
i
n

1
, X
n

2
=
i
n

2
)
=
P
(
Y
n
= (
i
n
, i
n

1
)

Y
n

1
= (
i
n

1
, i
n

2
))
.
2. We will show that
P
((
X
n
+1
, Y
n
+1
) = (
i
n
+1
, j
n
+1
)

(
X
n
, Y
n
) = (
i
n
, j
n
)
, . . . ,
(
X
0
, Y
0
) = (
i
0
, j
0
))
=
P
((
X
n
+1
This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
This is the end of the preview.
Sign up
to
access the rest of the document.
 Spring '09
 RESNIK
 Markov chain, Xn, Mark E. Lewis

Click to edit the document details