J. Appl. Prob.
46,
1209–1212 (2009)
Printed in England
©
Applied Probability Trust
2009
AN INEQUALITY FOR VARIANCES
OF THE DISCOUNTED REWARDS
EUGENE A. FEINBERG
∗ ∗∗
and
JUN FEI,
∗ ∗∗∗
Stony Brook University
Abstract
We consider the following two definitions of discounting: (i) multiplicative coefficient
in front of the rewards, and (ii) probability that the process has not been stopped if the
stopping time has an exponential distribution independent of the process. It is well known
that the expected total discounted rewards corresponding to these definitions are the same.
In this note we show that, the variance of the total discounted rewards is smaller for the
first definition than for the second definition.
Keywords:
Total discounted reward; variance; stopping time
2000 Mathematics Subject Classification: Primary 60G40
Secondary 90C40
1. Introduction
In this note we study two definitions of discounting: (i) multiplicative coefficient in front
of the rewards, and (ii) probability that the process has not been stopped if the stopping time
has an exponential distribution independent of the process.
It is well known that the total
discounted rewards corresponding to these definitions have equal expectations. However, as
we will show, the second moment and variance are smaller for the first definition than for the
second definition.
Since its introduction by Markowitz in his Nobel Prize winning paper [5], variance has
played an important role in stochastic optimization. In particular, there is a significant amount
of literature on various optimizations of Markov decision processes (MDPs); see the pioneering
work by Jaquette [4] and Sobel [7]–[9], a survey by White [11], and recent references by Van
Dijk and Sladký [10] and BaykalGürsoy and Gürsoy [1].
This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
This is the end of the preview.
Sign up
to
access the rest of the document.
 Fall '08
 Feinberg,E
 Variance, Probability theory, Tn, J2, total discounted rewards

Click to edit the document details