This preview shows pages 1–2. Sign up to view the full content.
This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Unformatted text preview: ECE 534: Elements of Information Theory, Fall 2010 Homework 4 Solutions Problem 4.8 (Davide Basilio Bartolini) Since the sequence X comes from a discrete memoryless source, the samples X i are independent random variables and p 2 = 1 p 1 . First, we need to compute the estimated duration for a sample, which is (due to the fact that the value of a symbol is equal to its duration): E [ T ] = E [ X i ] = p 1 + 2 p 2 = 2 p 1 Then, we compute the entropy for a sample of the process: H ( X i ) = ( p 1 log 2 p 1 + p 2 log 2 p 2 ) = ( p 1 log 2 p 1 (1 p 1 )log 2 (1 p 1 )) So, the expression for the entropy per unit time is: H ( X ) = H ( X i ) E [ T ] = ( p 1 log 2 p 1 (1 p 1 )log 2 (1 p 1 )) 2 p 1 To find the value for p 1 that maximizes H ( X ), we can compute its partial derivative w.r.t. p 1 and find the value of p 1 that makes it zero (i.e. the value for which its numerator is zero): δ H ( X ) δp 1 = T ( δH ( X )) δp 1 H ( X )( δT δp 1 ) T 2 = log e (1 p 1 ) 2log e p 1 T 2 (using base e for the logs in entropies) log e (1 p 1 ) 2log e p 1 = 0 ⇔ 1 p 1 = p 2 1 ⇔ p 2 1 + p 1 1 = 0 The last equation which gives the results for p 1 : 1 2 ( √ 5 + 1) and 1 2 ( √ 5 1). Of the two found values, the second is a maximum for H ( X ) (moreover, the first solution is less than zero, so it cannot be a probability value) and the...
View
Full
Document
This note was uploaded on 01/19/2012 for the course ECE 534 taught by Professor Natashadevroye during the Fall '10 term at Ill. Chicago.
 Fall '10
 NatashaDevroye

Click to edit the document details