±
2008 by Karl Sigman
1
Regenerative Processes
Continuoustime case
Given a stochastic process
X
=
{
X
(
t
) :
t
≥
0
}
, suppose that there exists a (proper) random
time
τ
=
τ
1
such that
{
X
(
τ
+
t
) :
t
≥
0
}
has the same distribution as
X
and is independent
of the past
C
1
=
{{
X
(
t
) : 0
≤
t < τ
}
, τ
}
. Then we say that
X
regenerated
at time
τ
,
meaning that it has “started over again” probabilistically, as if it was time
t
= 0 again, and
its future is independent of its past. In particular,
X
(
τ
) has the same distribution as
X
(0),
and is independent of the regeneration time
τ
.
C
1
is called the ﬁrst
cycle
and
X
1
=
τ
1
is
the ﬁrst
cycle length
. But if such a
τ
1
exists, then there must be a second such time
τ
2
> τ
1
yielding an identically distributed second cycle
C
2
=
{{
X
(
τ
1
+
t
) : 0
≤
t < τ
2

τ
1
}
, τ
2

τ
1
}
.
Continuing in this fashion we conclude that there is a renewal process of such times
{
τ
k
:
k
≥
1
}
(with
τ
0
def
= 0) with iid cycle lengths
X
k
=
τ
k

τ
k

1
, k
≥
1
,
and such that the entire cycles
C
k
=
{{
X
(
τ
k

1
+
t
) : 0
≤
t < X
k
}
, X
k
}
are iid.
Note how in particular,
X
(
τ
k
) is independent of the time
τ
k
: Upon regeneration the value
of the process is independent of what time it is.
The above deﬁnes what is called a
regenerative process
. It is said to be
positive recurrent
if
the renewal process is so, that is, if 0
< E
(
X
1
)
<
∞
. Otherwise it is said to be
null recurrent
if
E
(
X
1
) =
∞
.
As an example:
Any recurrent continuoustime Markov chain is regenerative.
Fix any state
i
and let
X
(0) =
i
. Then let
τ
=
τ
i,i
denote the ﬁrst time that the chain returns back to state
i
(after ﬁrst leaving it). By the (strong) Markov property, the chain indeed starts all over again
in state
i
at time
τ
independent of its past. Since
i
is recurrent, we know that the chain must in
fact return over and over again, and we can let
τ
k
denote the
k
th
such time. We also know that
positive recurrence of the chain, by deﬁnition, means that
E
(
τ
i,i
)
<
∞
, which is thus equivalent
to being a positive recurrent regenerative process.
Because
{
τ
k
}
is a renewal process and the cycles
{
C
k
}
are iid, we can apply the renewal
reward theorem in a variety of ways to a regenerative process so as to compute various time
average quantities of interest. The general result can be stated in words as
the time average is equal to the expected value over a cycle divided by the expected
cycle length.
For example, we can obtain the timeaverage of the process itself as
lim
t
→∞
1
t
Z
t
0
X
(
s
)
ds
=
E
(
R
)
E
(
X
)
, wp
1
,
where
X
=
X
1
and
R
=
R
1
=
R
X
0
X
(
s
)
ds
. The proof is derived by letting
N
(
t
) denote the
number of renewals (regenerations) by time
t
, and observing that
Z
t
0
X
(
s
)
ds
≈
N
(
t
)
X
j
=1
R
j
,
where
R
j
=
Z
τ
j
τ
j

1
X
(
s
)
ds.
1