This preview shows pages 1–2. Sign up to view the full content.
This preview has intentionally blurred sections. Sign up to view the full version.View Full Document
Unformatted text preview: Sample Midterm Solutions The examination is for 120 minutes. The maximum score is 70 points. Your answers should be unambiguous. Please show all work to allow for the possibility of partial credit. 1. ( 8 points ) I have a coin in my pocket. It is either fair (it comes up either heads or tails equiprobably when tossed) or it is biased with the probability of it coming up heads being 3 4 . A priori, it is equally likely to be fair or biased. Let X denote the identity of the coin. Thus P ( X = u ) = P ( X = b ) = 1 2 . The coin is tossed ten times. The outcomes of the tosses are independent and identically distributed. The coin comes up heads 3 times. What is the conditional entropy of X given this event ? You need not reduce your answer to a fraction. Solution : Let Y denote the number of heads among the ten tosses of the coin. Hence P ( Y = k | X = u ) = 10 k ! ( 1 2 ) 10 , k 10 P ( Y = k | X = b ) = 10 k ! ( 3 4 ) k ( 1 4 ) 10- k , k 10 . We are asked to find H ( X | Y = 3). To this end we find P ( X = b | Y = 3) = 10 k ( 3 4 ) 3 ( 1 4 ) 7 10 k ( 3 4 ) 3 ( 1 4 ) 7 + 10 k ( 1 2 ) 10 = 3 3 3 3 + 2 10 = 27 1051 , and P ( X = u | Y = 3) = 1024 1051 . Hence H ( X | Y = 3) = 27 1051 log 1051 27 + 1024 1051 log 1051 1024 . 2. ( 7 points ) A Stanford student claims that if I ( X ; Y | Z ) = I ( X ; Z | Y ) = I ( Y ; Z | X ) then X , Y , and Z are independent. Either prove or disprove this statement. Solution : Suppose X = Y = Z . Then I ( X ; Y | Z ) = H ( X | Z )- H ( X | Y,Z ) = 0, and similarly for I ( X ; Z | Y ) and I ( Y ; Z | X ). However in this case X , Y , and Z are not mutually independent unless X is a constant. Thus the claim is false....
View Full Document
This note was uploaded on 01/19/2012 for the course EE 229A taught by Professor Venkatanantharam during the Spring '07 term at University of California, Berkeley.
- Spring '07