hw2_sol - ECE 563 Information Theory Homework 2 Solutions...

Info iconThis preview shows pages 1–3. Sign up to view the full content.

View Full Document Right Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: ECE 563 Information Theory Homework 2 Solutions Instructor: R. Srikant TA: Akshay Kashyap 3.3 Let type a cuts be those that divide the piece in ratio (2 / 3 , 1 / 3) and type b cuts be those that divide the piece in ratio (3 / 5 , 2 / 5). Consider a sequence of n cuts. Define N ( a | n ) and N ( b | n ) to be the number of type a and type b cuts in this sequence. Clearly, N ( a | n ) + N ( b | n ) = n . By the law of large numbers, N ( a | n ) n = p ( a ) + a n , where lim n a n = 0. and p ( a ) is the probability that a cut is of type a . Similarly, N ( b | n ) n = p ( b ) + b n , where lim n b n = 0. Now, at the end of n cuts, the resulting piece is of size parenleftbigg 2 3 parenrightbigg N ( a | n ) parenleftbigg 3 5 parenrightbigg N ( b | n ) , which, by the preceding arguments, is exp braceleftbigg n bracketleftbigg N ( a | n ) n log 2 3 + N ( b | n ) n log 3 5 bracketrightbiggbracerightbigg = exp braceleftbigg n bracketleftbigg p ( a ) log 2 3 + p ( b ) log 3 5 bracketrightbiggbracerightbigg = exp braceleftbigg n bracketleftbigg 3 4 log 2 3 + 1 4 log 3 5 bracketrightbiggbracerightbigg , to the first order in exponent. Note: Some of you chose to find the expected size of the cake. That is not the same as the size of the cake to the first order of the exponent (which is what was asked in the question). Consider a sequence of i.i.d. random variables X i , i = 1 , 2 , . . . , where X 1 takes the value r with probability p and the value s with probability q = 1- p . Then, E [ n productdisplay i =1 X i ] = ( E [ X 1 ]) n = ( pr + (1- p ) s ) n , which is trivially exp { n log( pr + (1- p ) s ) } to the first order of exponent. However, n productdisplay i =1 X i = r N ( r | n ) s N ( s | n ) = exp braceleftbigg n bracketleftbigg N ( r | n ) n log r + N ( s | n ) n log s bracketrightbiggbracerightbigg , and since N ( r | n ) n p and N ( s | n ) n (1- p ) almost surely as n , producttext n i =1 X i to the first order in exponent is exp { n ( p log r + (1- p ) log s ) } . Since in general, log( pr + (1- p ) s ) negationslash = p log r + (1- p ) log s, 1 E [ producttext n i =1 X i ] need not be the same as producttext n i =1 X i to first order of exponent. In fact, since log is strictly concave over the positive real line, Jensens inequality gives us log( pr + (1- p ) s ) p log r + (1- p ) log s, with strict inequality when...
View Full Document

This note was uploaded on 11/19/2010 for the course ANTH 122 taught by Professor 323 during the Spring '10 term at Centennial College.

Page1 / 5

hw2_sol - ECE 563 Information Theory Homework 2 Solutions...

This preview shows document pages 1 - 3. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online