hw4solniis

# hw4solniis - Problem Set 4 MATH 778C Spring 2009 Cooper...

This preview shows pages 1–4. Sign up to view the full content.

Problem Set 4 MATH 778C, Spring 2009, Cooper Expiration: Thursday April 30 You are awarded up to 25 points per problem, 5 points for submitting solutions in L A T E X, and 5 points per solution that is used for the answer key. All answers must be fully rigorous – do not assume anything that you are not sure everyone else in the class knew prior to Day 1 of this class. However, you may cite without proof any result proven in class. 1. Two n -sided dice with sides labeled 1 through n are rolled, resulting in the i.i.d. random variables X and Y . Let Z = X + Y . Compute H ( X ), H ( X, Y ), H ( Z ), H ( X | Z ), I ( X ; Y ) and I ( X ; Y | Z ). (Note that this last quantity is not I ( X ; ( Y | Z )).) Solution (Ser-Wei Fu): Let f ( k ) = k i =1 i log i . Observe that X and Y are uniform random variables. H ( X ) = log n. H ( X, Y ) = 2 log n. Using the fact that the entropy of an uniform random variable is the logarithm of the size of the support. H ( Z ) = - X z p ( z ) log p ( z ) = - n X z =2 p ( z ) log p ( z ) - 2 n X z = n +1 p ( z ) log p ( z ) . Observe that the number of pairs with the same sum is increasing for small sums and decreasing after z = n + 1. H ( Z ) = - n X z =2 z - 1 n 2 log z - 1 n 2 - 2 n X z = n +1 2 n - z + 1 n 2 log 2 n - z + 1 n 2 = - n - 1 X i =1 i n 2 log i n 2 - n X i =1 i n 2 log i n 2 = 2 n - 1 X i =1 i n 2 log n + 2 n X i =1 i n 2 log n - 1 n 2 ( f ( n - 1) + f ( n )) = 2 log n - 1 n 2 ( f ( n - 1) + f ( n )) .

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
H ( X | Z ) = X z p ( z ) H ( X | Z = z ) = n X z =2 p ( z ) H ( X | Z = z ) + 2 n X z = n +1 p ( z ) H ( X | Z = z ) . Observe that ( X | Z = z ) is a uniform random variable. H ( X | Z ) = 1 n 2 n - 1 X i =1 i log i + n X i =1 i log i ! = 1 n 2 ( f ( n - 1) + f ( n )) . I ( X ; Y ) = 0 . Since X and Y are independent. I ( X ; Y | Z ) = H ( X | Z ) - H ( X | Y, Z ) = 1 n 2 ( f ( n - 1) + f ( n )) . The part of H ( X | Y, Z ) is equal to zero because given y, z the value of x is fixed. The original way we looked at this. I ( X ; Y | Z ) = I ( X ; Y, Z ) - I ( X ; Z ) = H ( X ) + H ( Y, Z ) - H ( X, Y, Z ) - H ( X ) - H ( X | Z ) = H ( Y, Z ) - H ( X, Y, Z ) + H ( X | Z ) = H ( Z ) + H ( Y | Z ) + H ( X | Z ) - H ( X, Y, Z ) = 1 n 2 ( f ( n - 1) + f ( n )) . Notice that H ( X, Y, Z ) = 2 log n since it is another uniform random vari- able. 2. Consider a channel with binary inputs that has both erasures and errors. Let the probability of error be and the probability of erasure be α . (Hence, the probability of correct transmission is 1 - α - .) What is the capacity of this channel? Solution (Aaron Dutle): Suppose P (0) = p and P (1) = 1 - p is a maximizing probability distribu- tion for I ( X ; Y ) . Since the channel is symmetric with respect to the labels 0 , 1 , we get that the distribution P (0) = 1 - p and P (1) = p is also a maximizing distribution. Since I ( X ; Y ) is always concave in p ( x ), we see that the average of these two distributions (which is p (0) = p (1) = 1 / 2) is also a maximizing distribution. Using the formula I ( X ; Y ) = x y P ( x, y ) log P ( x,y ) P ( x ) P ( y ) , the symmetry of the channel, and just computing the needed probabilities, we get
C = X x X y P ( x, y ) log P ( x, y ) P ( x ) P ( y ) = 2 X y P (0 , y ) log 2 P (0 , y ) P ( y ) = 2 1 2 (1 - α - ) log 2(1 - α - ) (1 - α ) 1 2 ( α ) log (1) 1 2 ( ) log 2 (1 - α ) = (1 - α ) - (1 - α ) log(1 - α ) + (1 - α - ) log(1 - α - ) + log .

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

### What students are saying

• As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

Kiran Temple University Fox School of Business ‘17, Course Hero Intern

• I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

Dana University of Pennsylvania ‘17, Course Hero Intern

• The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

Jill Tulane University ‘16, Course Hero Intern