This preview shows pages 1–3. Sign up to view the full content.
This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Unformatted text preview: EE 376A Information Theory Prof. T. Weissman Friday, Feb 19, 2010 Homework Set #6 (Due: 5pm Friday, Feb. 26, 2010) 1. Channel capacity with cost constraint There are applications in which some chan nel input symbols are more costly than others. Letting : X 7 R denote the cost function. The per symbol cost of the transmitted sequence X n is ( X n ) = 1 n n i =1 ( X i ). The expected per symbol cost of the code is then given by E( X n ) = 2 nR 2 n R X k =1 ( x n ( k )) , where x n ( k ) is the k th codeword in the code book. Definition: A rate R is achievable at cost , if > 0 there exists n and a channel code of blocklength n, rate R , E ( X n ) + and probability of error P ( n ) e . Definition: The capacitycost function of the channel is defined as C () = sup { R : R is achievable at cost } . In words, C () is the maximum rate of reliable communication when restricting the transmission to an expected cost that does not exceed per channel use. (a) Prove that for a given P Y  X ( y  x ), I ( X ; Y ) is a concave function of P X ( x ). Hint: First show that if f : R n R , is a convex function, then for any n m realvalued matrix A , g ( X ) , f ( AX ), g : R m R , is also a convex function, i.e. linear transformation preserves convexity. (b) Consider the following expression C ( I ) () = max E( X ) I ( X ; Y ) where the maximum on the right hand side is over all random variables X at the channel input that satisfy the indicated cost constraint. Prove that C ( I ) () is a nondecreasing concave function in . You may use the result from Part (a). (c) The converse: C () C ( I ) (). For any sequence of schemes of rate R , E( X n ) , and vanishing probability of error: 1 nR = H ( M ) = I ( M ; Y n ) + H ( M  Y n ) ( a ) I ( M ; Y n ) + n n ( b ) I ( X n ; Y n ) + n n ( c ) H ( Y n ) n X i =1 H ( Y i  X i ) + n n ( d ) n X i =1 I ( X i ; Y i ) + n n ( e ) n X i =1 C ( I ) (E( X i )) + n n ( f ) nC ( I ) (E( X n )) + n n ( g ) nC ( I ) () + n n , where n 0 as n . Provide explanations for inequalities (a) (g). Solutions: (a) First we show that if f : R n R , is a convex function, then for any n m realvalued matrix A , g ( X ) , f ( AX ), g : R m R , is also a convex function....
View
Full
Document
 Spring '10
 sd

Click to edit the document details