This preview shows pages 1–3. Sign up to view the full content.
This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Unformatted text preview: Applied Probability Trust (5 October 2009) INEQUALITY FOR VARIANCES OF THE DISCOUNTED RE WARDS EUGENE A. FEINBERG, * Stony Brook University JUN FEI, ** Stony Brook University Abstract We consider the following two definitions of discounting: (i) multiplicative coefficient in front of the rewards, and (ii) probability that the process has not been stopped if the stopping time has an exponential distribution independent of the process. It is wellknown that the expected total discounted rewards corresponding to these definitions are the same. In this note we show that for the first definition the variance of the total discounted rewards is smaller than for the second one. Keywords: total discounted reward; variance; stopping time 2000 Mathematics Subject Classification: Primary 60G40 Secondary 90C40 1. Introduction In this note we study two definitions of discounting: (i) multiplicative coefficient in front of the rewards, and (ii) probability that the process has not been stopped if the stopping time has an exponential distribution independent of the process. It is wellknown that the total discounted rewards corresponding to these definitions have equal expectations. However, as we will show, the second moment and variance are smaller for the first definition than for the second definition. Since introduction by Markowitz in his NobelPrize winning paper [5], variance has played an important role in stochastic optimization. In particular there is a significant * Postal address: Department of Applied Mathematics & Statistics, Stony Brook University, Stony Brook, NY 11794. Email: [email protected] ** Postal address: Department of Applied Mathematics & Statistics, Stony Brook University, Stony Brook, NY 11794. Email: [email protected] 1 2 E.A. Feinberg, Jun Fei amount of literature on various optimization of Markov Decision Processes (MDPs), see the pioneering work by Jaquette [4] and Sobel [7, 8, 9], a survey by White [11], and recent references by Van Dijk and Sladk´y [10] and BaykalG¨ursoy and G¨ursoy [1].and recent references by Van Dijk and Sladk´y [10] and BaykalG¨ursoy and G¨ursoy [1]....
View
Full
Document
This note was uploaded on 12/06/2011 for the course MATH 101 taught by Professor Eugenea.feinberg during the Fall '11 term at State University of New York.
 Fall '11
 EugeneA.Feinberg
 Counting, Probability, Variance

Click to edit the document details