{[ promptMessage ]}

Bookmark it

{[ promptMessage ]}

# spring2006 - 1 ECE 534 Elements of Information Theory...

This preview shows pages 1–3. Sign up to view the full content.

1 ECE 534: Elements of Information Theory Solutions to Midterm Exam (Spring 2006) Problem 1 [20 pts.] A discrete memoryless source has an alphabet of three letters, x i , i = 1 , 2 , 3 , with probabilities 0 . 4 , 0 . 4 , and 0 . 2 , respectively. (a) Find the binary Huffman code for this source and determine the average number of bits needed for each source letter. (b) Suppose two letters at a time are encoded into a binary sequence. Find the Huffman code and the average number of bits needed per source letter. Solution: (a) One possible Huffman code is: C (1) = 0 , C (2) = 10 , and C (3) = 11 . The average number of bits per source letter is 0 . 4 + (0 . 4 + 0 . 2) × 2 = 1 . 6 . (b) One possible code construction is: C (11) = 000 , C (12) = 001 , C (13) = 100 , C (21) = 010 , C (22) = 011 , C (23) = 110 , C (31) = 111 , C (32) = 1010 , and C (33) = 1011 . The average number of bits per source letter is [3 + (0 . 08 + 0 . 04)] / 2 = 1 . 56 . Problem 2 [20 pts.] A source X produces letters from a three-symbol alphabet with the probability assignment P X (0) = 1 / 4 , P X (1) = 1 / 4 , and P X (2) = 1 / 2 . Each source letter x is transmitted through two channels simultaneously with outputs y and z and the transition probabilities indicated below: 2 1 0 1 0 x y P ( y | x ) 1 1 1 2 1 2 a0 a0 a0 a0 a0 a0 a0 a0 a0 a0 a0 a0 a0 a8 a8 a8 a8 a8 a8 a8 a8 a8 a8 a8 a8 a8 2 1 0 1 0 x z P ( z | x ) 1 1 1 a8 a8 a8 a8 a8 a8 a8 a8 a8 a8 a8 a8 a8 a8 a8 a8 a8 a8 a8 a8 a8 a8 a8 a8 a8 a8 Calculate H ( X ) , H ( Y ) , H ( Z ) , H ( Y, Z ) , I ( X ; Y ) , I ( X ; Z ) , I ( X ; Y | Z ) , and I ( X ; Y, Z ) . Solution: H ( X ) = 1 4 log 2 4 + 1 4 log 2 4 + 1 2 log 2 2 = 1 . 5

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
2 The probability distribution of Y is P Y (0) = 1 4 + 1 2 1 2 = 1 2 . Therefore, H ( Y ) = 1 Similarly, P Z (0) = P Z (1) = 1 2 and H ( Z ) = 1 If Z = 1 , then
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

### Page1 / 5

spring2006 - 1 ECE 534 Elements of Information Theory...

This preview shows document pages 1 - 3. Sign up to view the full document.

View Full Document
Ask a homework question - tutors are online