HW3_ES250_sol_a

HW3_ES250_sol_a - Harvard SEAS ES250 – Information Theory...

Info iconThis preview shows pages 1–3. Sign up to view the full content.

View Full Document Right Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: Harvard SEAS ES250 – Information Theory Homework 3 Solutions 1. Let X ∼ p ( x ), x = 1 , 2 , ··· ,m , denote the winner of a horse race. Suppose the odds o ( x ) are fair with respect to p ( x ), i.e., o ( x ) = 1 p ( x ) . Let b ( x ) be the amount bet on horse x , b ( x ) ≥ 0, ∑ m 1 b ( x ) = 1. Then the resulting wealth factor is S ( x ) = b ( x ) o ( x ), with probability p ( x ). (a) Find the expected wealth ES ( X ). (b) Find W ∗ , the optimal growth rate of wealth. (c) Suppose Y = braceleftbigg 1 , X = 1 or 2 , otherwise If this side information is available before the bet, how much does it increase the growth rate W ∗ ? (d) Find I ( X ; Y ) Solution : (a) The expected wealth ES ( X ) is ES ( X ) = m summationdisplay x =1 S ( x ) p ( x ) = m summationdisplay x =1 b ( x ) o ( x ) p ( x ) = m summationdisplay x =1 b ( x ) (since o ( x ) = 1 /p ( x )) = 1 (b) The optimal growth rate of wealth, W ∗ , is achieved when b ( x ) = p ( x ) for all x , in which case, W ∗ = E [log S ( X )] = m summationdisplay x =1 p ( x ) log( b ( x ) o ( x )) = m summationdisplay x =1 p ( x ) log( p ( x ) /p ( x )) = m summationdisplay x =1 p ( x ) log(1) = 0 , so we maintain our current wealth. 1 Harvard SEAS ES250 – Information Theory (c) The increase in our growth rate due to the side information is given by I ( X ; Y ). Let q = Pr( Y = 1) = p (1) + p (2). I ( X ; Y ) = H ( Y )- H ( Y | X ) = H ( Y ) (since Y is a deterministic function of X ) = H ( q ) (d) Already computed above. 2. Many years ago in ancient St. Petersburg the following gambling proposition caused great conster- nation. For an entry fee of c units, a gambler receives a payoff of 2 k units with probability 2 − k , k = 1 , 2 , ··· . (a) Show that the expected payoff for this game is infinite. For this reason, it was argued that c = ∞ was a ”fair” price to pay to play this game. Most people find this answer absurd. (b) Suppose that the gambler can buy a share of the game. For example, if he invests c/ 2 units in the game, he receives 1 2 a share and a return X/ 2, where Pr( X = 2 k ) = 2 − k , k = 1 , 2 , ··· . Suppose that X 1 ,X 2 , ··· are i.i.d. according to this distribution and that the gambler reinvests all his wealth each time. Thus, his wealth S n at time n is given by S n = n productdisplay i =1 X i c Show that this limit is ∞ or 0, with probability 1, accordingly as c < c ∗ or c > c ∗ . Identify the ”fair” entry fee c ∗ . More realistically, the gambler should be allowed to keep a proportion ¯ b = 1- b of his money in his pocket and invest the rest in the St. Petersburg game. His wealth at time n is then S n = n productdisplay i =1 parenleftbigg ¯ b + bX i c parenrightbigg Let W ( b,c ) = ∞ summationdisplay k =1 2 − k log parenleftbigg 1- b + b 2 k c parenrightbigg We have S n ....
View Full Document

This note was uploaded on 12/01/2010 for the course ADLAC 1023 at Stanford.

Page1 / 9

HW3_ES250_sol_a - Harvard SEAS ES250 – Information Theory...

This preview shows document pages 1 - 3. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online