632 Introduction to Stochastic Processes Fall 2003
Midterm Exam I
Instructions:
Show calculations and justify nonobvious statements for
full credit. When you quote a result proved in class, state the hypotheses
and conclusion clearly, and justify why the hypotheses are met. The points
add up 100.
General notation:
P
x
(
A
) is the probability of the event
A
when the
chain starts in state
x
,
P
μ
(
A
) the probability when the initial state is random
with distribution
μ
.
T
y
= min
{
n
≥
1 :
X
n
=
y
}
is the ﬁrst time after 0 that
the chain visits
y
, or
∞
if no visit to
y
ever happens.
ρ
x,y
=
P
x
(
T
y
<
∞
) is
the probability that the chain visits
y
some time after time 0, given that it
started at
x
.
N
(
y
) is the number of visits to state
y
, not counting a possible
visit at time 0.
1.
For this problem, let
P
be deﬁned by
P
=
0
1
0
0
0
1
1
/
3 2
/
3 0
.
(a) (15 pts) Explain why the power
P
n
of the matrix converges as
n
→ ∞
and ﬁnd the limit matrix.
(b) (10 pts) Let
This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
This is the end of the preview.
Sign up
to
access the rest of the document.
 Spring '07
 Seppalainen
 Probability, Stochastic process, pts, Markov chain, justify nonobvious statements, Stochastic Processes Fall

Click to edit the document details