7.25 - EE 376A Information Theory Prof. T. Weissman Friday,...

Info iconThis preview shows pages 1–3. Sign up to view the full content.

View Full Document Right Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: EE 376A Information Theory Prof. T. Weissman Friday, Feb 19, 2010 Homework Set #6 (Due: 5pm Friday, Feb. 26, 2010) 1. Channel capacity with cost constraint There are applications in which some chan- nel input symbols are more costly than others. Letting : |X| 7 R denote the cost function. The per symbol cost of the transmitted sequence X n is ( X n ) = 1 n n i =1 ( X i ). The expected per symbol cost of the code is then given by E( X n ) = 2- nR 2 n R X k =1 ( x n ( k )) , where x n ( k ) is the k th codeword in the code book. Definition: A rate R is achievable at cost , if > 0 there exists n and a channel code of block-length n, rate R- , E ( X n ) + and probability of error P ( n ) e . Definition: The capacity-cost function of the channel is defined as C () = sup { R : R is achievable at cost } . In words, C () is the maximum rate of reliable communication when restricting the transmission to an expected cost that does not exceed per channel use. (a) Prove that for a given P Y | X ( y | x ), I ( X ; Y ) is a concave function of P X ( x ). Hint: First show that if f : R n R , is a convex function, then for any n m real-valued matrix A , g ( X ) , f ( AX ), g : R m R , is also a convex function, i.e. linear transformation preserves convexity. (b) Consider the following expression C ( I ) () = max E( X ) I ( X ; Y ) where the maximum on the right hand side is over all random variables X at the channel input that satisfy the indicated cost constraint. Prove that C ( I ) () is a non-decreasing concave function in . You may use the result from Part (a). (c) The converse: C () C ( I ) (). For any sequence of schemes of rate R , E( X n ) , and vanishing probability of error: 1 nR = H ( M ) = I ( M ; Y n ) + H ( M | Y n ) ( a ) I ( M ; Y n ) + n n ( b ) I ( X n ; Y n ) + n n ( c ) H ( Y n )- n X i =1 H ( Y i | X i ) + n n ( d ) n X i =1 I ( X i ; Y i ) + n n ( e ) n X i =1 C ( I ) (E( X i )) + n n ( f ) nC ( I ) (E( X n )) + n n ( g ) nC ( I ) () + n n , where n 0 as n . Provide explanations for inequalities (a) (g). Solutions: (a) First we show that if f : R n R , is a convex function, then for any n m real-valued matrix A , g ( X ) , f ( AX ), g : R m R , is also a convex function....
View Full Document

Page1 / 8

7.25 - EE 376A Information Theory Prof. T. Weissman Friday,...

This preview shows document pages 1 - 3. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online