tut2_sol - EE 4212 Information and Coding Solution to...

Info iconThis preview shows pages 1–2. Sign up to view the full content.

View Full Document Right Arrow Icon
1 EE 4212 Information and Coding Solution to Tutorial 2 1. Binary Memoryless Source (BMS) (i) Binary Entropy Function ) 1 log( ) 1 ( log ) 1 ( 1 log ) 1 ( ) 0 ( 1 log ) 0 ( 1 log ) ( 1 0 1 0 p p p p P P P P P P I P H ii i i i i = + = = = ∑∑ == u (ii) Substitute p =1- q into H ( u ) p q p H H q q q q q q q q H = = = = 1 ) ( ) ( log ) 1 log( ) 1 ( )] 1 ( 1 log[ )] 1 ( 1 [ ) 1 log( ) 1 ( ) ( u u u Mid-point: 1- p = p => 1-2 p =0 => p =1/2 (iii) By first-order and second-order differentiation on H ( u ) with respect to p ] ln ) 1 ln( [ 2 ln 1 )] 1 ln( ) 1 1 )( 1 ( ln ) 1 ( [ 2 ln 1 p p p p p p p p dp dH + = + + = ) 1 1 1 ( 2 ln 1 2 2 p p dp H d + = 2 1 1 ln ) 1 ln( 0 ln ) 1 ln( 0 = = = = + => = p p p p p p p dp dH Put p =1/2 into the second-order derivative, 0 2 ln 4 ) 1 1 1 ( 2 ln 1 2 1 2 1 2 1 2 2 < = + = = p dp H d Ö the turning point p =1/2 is a maximum, not a minimum Conclusion : H ( u ) is at its maximum when p =1/2 and it follows the theory of “maximum entropy for equiprobable source output”. In this case, the entropy is bit H p 1 2 log ) 2 1 1 log( ) 2 1 1 ( 2 1
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Image of page 2
This is the end of the preview. Sign up to access the rest of the document.

This note was uploaded on 02/26/2011 for the course EE 4011 taught by Professor Wadewe during the Spring '11 term at City University of Hong Kong.

Page1 / 3

tut2_sol - EE 4212 Information and Coding Solution to...

This preview shows document pages 1 - 2. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online