Feinberg_Fei_Rev - Applied Probability Trust (5 October...

Info iconThis preview shows pages 1–3. Sign up to view the full content.

View Full Document Right Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: Applied Probability Trust (5 October 2009) INEQUALITY FOR VARIANCES OF THE DISCOUNTED RE- WARDS EUGENE A. FEINBERG, * Stony Brook University JUN FEI, ** Stony Brook University Abstract We consider the following two definitions of discounting: (i) multiplicative coefficient in front of the rewards, and (ii) probability that the process has not been stopped if the stopping time has an exponential distribution independent of the process. It is well-known that the expected total discounted rewards corresponding to these definitions are the same. In this note we show that for the first definition the variance of the total discounted rewards is smaller than for the second one. Keywords: total discounted reward; variance; stopping time 2000 Mathematics Subject Classification: Primary 60G40 Secondary 90C40 1. Introduction In this note we study two definitions of discounting: (i) multiplicative coefficient in front of the rewards, and (ii) probability that the process has not been stopped if the stopping time has an exponential distribution independent of the process. It is well-known that the total discounted rewards corresponding to these definitions have equal expectations. However, as we will show, the second moment and variance are smaller for the first definition than for the second definition. Since introduction by Markowitz in his Nobel-Prize winning paper [5], variance has played an important role in stochastic optimization. In particular there is a significant * Postal address: Department of Applied Mathematics & Statistics, Stony Brook University, Stony Brook, NY 11794. Email: [email protected] ** Postal address: Department of Applied Mathematics & Statistics, Stony Brook University, Stony Brook, NY 11794. Email: [email protected] 1 2 E.A. Feinberg, Jun Fei amount of literature on various optimization of Markov Decision Processes (MDPs), see the pioneering work by Jaquette [4] and Sobel [7, 8, 9], a survey by White [11], and recent references by Van Dijk and Sladk´y [10] and Baykal-G¨ursoy and G¨ursoy [1].and recent references by Van Dijk and Sladk´y [10] and Baykal-G¨ursoy and G¨ursoy [1]....
View Full Document

This note was uploaded on 12/06/2011 for the course MATH 101 taught by Professor Eugenea.feinberg during the Fall '11 term at State University of New York.

Page1 / 5

Feinberg_Fei_Rev - Applied Probability Trust (5 October...

This preview shows document pages 1 - 3. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online