This
** preview**
has intentionally

**sections.**

*blurred***to view the full version.**

*Sign up*
**Unformatted text preview: **Harvard SEAS ES250 – Information Theory Homework 3 Solutions 1. Let X ∼ p ( x ), x = 1 , 2 , ··· ,m , denote the winner of a horse race. Suppose the odds o ( x ) are fair with respect to p ( x ), i.e., o ( x ) = 1 p ( x ) . Let b ( x ) be the amount bet on horse x , b ( x ) ≥ 0, ∑ m 1 b ( x ) = 1. Then the resulting wealth factor is S ( x ) = b ( x ) o ( x ), with probability p ( x ). (a) Find the expected wealth ES ( X ). (b) Find W ∗ , the optimal growth rate of wealth. (c) Suppose Y = braceleftbigg 1 , X = 1 or 2 , otherwise If this side information is available before the bet, how much does it increase the growth rate W ∗ ? (d) Find I ( X ; Y ) Solution : (a) The expected wealth ES ( X ) is ES ( X ) = m summationdisplay x =1 S ( x ) p ( x ) = m summationdisplay x =1 b ( x ) o ( x ) p ( x ) = m summationdisplay x =1 b ( x ) (since o ( x ) = 1 /p ( x )) = 1 (b) The optimal growth rate of wealth, W ∗ , is achieved when b ( x ) = p ( x ) for all x , in which case, W ∗ = E [log S ( X )] = m summationdisplay x =1 p ( x ) log( b ( x ) o ( x )) = m summationdisplay x =1 p ( x ) log( p ( x ) /p ( x )) = m summationdisplay x =1 p ( x ) log(1) = 0 , so we maintain our current wealth. 1 Harvard SEAS ES250 – Information Theory (c) The increase in our growth rate due to the side information is given by I ( X ; Y ). Let q = Pr( Y = 1) = p (1) + p (2). I ( X ; Y ) = H ( Y )- H ( Y | X ) = H ( Y ) (since Y is a deterministic function of X ) = H ( q ) (d) Already computed above. 2. Many years ago in ancient St. Petersburg the following gambling proposition caused great conster- nation. For an entry fee of c units, a gambler receives a payoff of 2 k units with probability 2 − k , k = 1 , 2 , ··· . (a) Show that the expected payoff for this game is infinite. For this reason, it was argued that c = ∞ was a ”fair” price to pay to play this game. Most people find this answer absurd. (b) Suppose that the gambler can buy a share of the game. For example, if he invests c/ 2 units in the game, he receives 1 2 a share and a return X/ 2, where Pr( X = 2 k ) = 2 − k , k = 1 , 2 , ··· . Suppose that X 1 ,X 2 , ··· are i.i.d. according to this distribution and that the gambler reinvests all his wealth each time. Thus, his wealth S n at time n is given by S n = n productdisplay i =1 X i c Show that this limit is ∞ or 0, with probability 1, accordingly as c < c ∗ or c > c ∗ . Identify the ”fair” entry fee c ∗ . More realistically, the gambler should be allowed to keep a proportion ¯ b = 1- b of his money in his pocket and invest the rest in the St. Petersburg game. His wealth at time n is then S n = n productdisplay i =1 parenleftbigg ¯ b + bX i c parenrightbigg Let W ( b,c ) = ∞ summationdisplay k =1 2 − k log parenleftbigg 1- b + b 2 k c parenrightbigg We have S n ....

View
Full
Document