HW4s[1] - ECE 534: Elements of Information Theory, Fall...

Info iconThis preview shows pages 1–2. Sign up to view the full content.

View Full Document Right Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: ECE 534: Elements of Information Theory, Fall 2010 Homework 4 Solutions Problem 4.8 (Davide Basilio Bartolini) Since the sequence X comes from a discrete memoryless source, the samples X i are independent random variables and p 2 = 1- p 1 . First, we need to compute the estimated duration for a sample, which is (due to the fact that the value of a symbol is equal to its duration): E [ T ] = E [ X i ] = p 1 + 2 p 2 = 2- p 1 Then, we compute the entropy for a sample of the process: H ( X i ) =- ( p 1 log 2 p 1 + p 2 log 2 p 2 ) =- ( p 1 log 2 p 1- (1- p 1 )log 2 (1- p 1 )) So, the expression for the entropy per unit time is: H ( X ) = H ( X i ) E [ T ] =- ( p 1 log 2 p 1- (1- p 1 )log 2 (1- p 1 )) 2- p 1 To find the value for p 1 that maximizes H ( X ), we can compute its partial derivative w.r.t. p 1 and find the value of p 1 that makes it zero (i.e. the value for which its numerator is zero): δ H ( X ) δp 1 = T ( δH ( X )) δp 1- H ( X )( δT δp 1 ) T 2 = log e (1- p 1 )- 2log e p 1 T 2 (using base e for the logs in entropies) log e (1- p 1 )- 2log e p 1 = 0 ⇔ 1- p 1 = p 2 1 ⇔ p 2 1 + p 1- 1 = 0 The last equation which gives the results for p 1 :- 1 2 ( √ 5 + 1) and 1 2 ( √ 5- 1). Of the two found values, the second is a maximum for H ( X ) (moreover, the first solution is less than zero, so it cannot be a probability value) and the...
View Full Document

This note was uploaded on 01/19/2012 for the course ECE 534 taught by Professor Natashadevroye during the Fall '10 term at Ill. Chicago.

Page1 / 3

HW4s[1] - ECE 534: Elements of Information Theory, Fall...

This preview shows document pages 1 - 2. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online