This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Unformatted text preview: ECE 563 Information Theory Homework 2 Solutions Instructor: R. Srikant TA: Akshay Kashyap 3.3 Let type a cuts be those that divide the piece in ratio (2 / 3 , 1 / 3) and type b cuts be those that divide the piece in ratio (3 / 5 , 2 / 5). Consider a sequence of n cuts. Define N ( a  n ) and N ( b  n ) to be the number of type a and type b cuts in this sequence. Clearly, N ( a  n ) + N ( b  n ) = n . By the law of large numbers, N ( a  n ) n = p ( a ) + a n , where lim n a n = 0. and p ( a ) is the probability that a cut is of type a . Similarly, N ( b  n ) n = p ( b ) + b n , where lim n b n = 0. Now, at the end of n cuts, the resulting piece is of size parenleftbigg 2 3 parenrightbigg N ( a  n ) parenleftbigg 3 5 parenrightbigg N ( b  n ) , which, by the preceding arguments, is exp braceleftbigg n bracketleftbigg N ( a  n ) n log 2 3 + N ( b  n ) n log 3 5 bracketrightbiggbracerightbigg = exp braceleftbigg n bracketleftbigg p ( a ) log 2 3 + p ( b ) log 3 5 bracketrightbiggbracerightbigg = exp braceleftbigg n bracketleftbigg 3 4 log 2 3 + 1 4 log 3 5 bracketrightbiggbracerightbigg , to the first order in exponent. Note: Some of you chose to find the expected size of the cake. That is not the same as the size of the cake to the first order of the exponent (which is what was asked in the question). Consider a sequence of i.i.d. random variables X i , i = 1 , 2 , . . . , where X 1 takes the value r with probability p and the value s with probability q = 1 p . Then, E [ n productdisplay i =1 X i ] = ( E [ X 1 ]) n = ( pr + (1 p ) s ) n , which is trivially exp { n log( pr + (1 p ) s ) } to the first order of exponent. However, n productdisplay i =1 X i = r N ( r  n ) s N ( s  n ) = exp braceleftbigg n bracketleftbigg N ( r  n ) n log r + N ( s  n ) n log s bracketrightbiggbracerightbigg , and since N ( r  n ) n p and N ( s  n ) n (1 p ) almost surely as n , producttext n i =1 X i to the first order in exponent is exp { n ( p log r + (1 p ) log s ) } . Since in general, log( pr + (1 p ) s ) negationslash = p log r + (1 p ) log s, 1 E [ producttext n i =1 X i ] need not be the same as producttext n i =1 X i to first order of exponent. In fact, since log is strictly concave over the positive real line, Jensens inequality gives us log( pr + (1 p ) s ) p log r + (1 p ) log s, with strict inequality when...
View
Full Document
 Spring '10
 323
 Derivative, Probability theory, Markov chain, Xn, Xn X

Click to edit the document details