MS&E 221
Midterm Examination Solutions
Stochastic Modeling
21st February 2003
Prof. Peter W. Glynn
Page 1 of 4
Midterm Examination Solutions
1. (30 points)
Consider a driver for whom the probability of having an accident in any given year depends on
whether she had an accident the year before. Specifically, the probability of her having an accident
in a particular year given that she did not have an accident the year before is 0
.
1, while the
probability of her having an accident given that she had an accident the year before is 0
.
6.
(a) (4 points)
Assuming our driver did not have an accident in 2002, what is the expected
number of years till her next accident?
The time until her next accident is a geometric r.v.
with success probability 0.1, so its
expected value is 10.
(b) (5 points)
Suppose that our driver is currently 20 years old. What is the probability that
our driver will have more than 14 accidents by the time she is 70?
Consider the Markov Chain
Y
= (
Y
n
:
n
≥
0)
on state space
{
A, N
}
where
Y
n
=
A
if the
driver has an accident in year
n
and
Y
n
=
N
otherwise. Its graph representation is
A
B
0.6
0.9
0.1
0.4
We use a CLTbased approximation for
K
(
n
)
4
=
∑
n

1
j
=0
I
(
Y
j
=
A
)
, (i.e.
for the number of
accidents she has had by year
n
). Let
f
=
(
1
0
)
be the corresponding “reward function” and
π
be the stationary probabilities.
Solving
π
=
πP
,
πe
= 1
,
π
≥
0
we get
π
(
A
) = 1
/
5
,
π
(
N
) = 4
/
5
. Hence, the long term average reward per year (mean number of accidents per
year) is
α
4
=
πf
= 1
/
5
.
Let
ˆ
f
be the centered reward,
ˆ
f
=
‡
4
/
5

1
/
5
·
.
From the CLT from DTMC we know that
K
(
n
)

nα
σ
√
n
=
⇒
N
(0
,
1)
as
n
→ ∞
,
where
σ
2
=
π
(
A
)
E
A
•
‡
∑
T
A

1
j
=0
ˆ
f
(
X
j
)
·
2
‚
and
T
A
4
= inf
{
j
≥
1 :
Y
j
=
A
}
.
This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
This is the end of the preview.
Sign up
to
access the rest of the document.
 Winter '11
 Ramesh
 Probability theory, Markov chain, Midterm Examination, DTMC

Click to edit the document details