This preview shows pages 1–2. Sign up to view the full content.
ORIE 3510 – Homework 3
Instructor: Mark E. Lewis
due 2PM, Wednesday February 15, 2012 (ORIE Hallway drop box)
1. Consider the stochastic process
X
n
= min(
X
n

1
, X
n

2
) +
ǫ
n
with
X
0
= 0,
X
1
= 0 and
ǫ
n
is a random variable taking values

1 and 1 with probability
1
/
2, and
{
ǫ
n
, n
≥
0
}
are i.i.d.
(a) Is
X
n
a Markov chain?
(b) Based on
X
n
, construct a Markov chain.
2. Let
{
X
n
, n
≥
0
}
and
{
Y
n
, n
≥
0
}
be two independent Markov chains each with the same
discrete space
X
and same transition matrix. Defne the process
{
Z
n
, n
≥
0
}
=
{
(
X
n
, Y
n
)
, n
≥
0
}
with the state space
X
×
X
. Show that
{
Z
n
, n
≥
0
}
is a Markov chain and give the transition
matrix.
3. Suppose that
{
X
n
, n
≥
0
}
is a Markov chain. Show that
{
Y
n
, n
≥
1
}
=
{
(
X
n
, X
n

1
)
, n
≥
1
}
is a Markov chain and give its transition matrix.
4. Suppose that
{
Z
n
, n
≥
1
}
are i.i.d. random variables representing outcomes o± successive
throws o± a ±air die. Defne
X
n
= max
{
Z
1
, Z
This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
This is the end of the preview. Sign up
to
access the rest of the document.
 Spring '10
 LEWIS

Click to edit the document details