ece6605-hwk1 - to see a sequence of independent, equally...

Info iconThis preview shows pages 1–2. Sign up to view the full content.

View Full Document Right Arrow Icon
GEORGIA INSTITUTE OF TECHNOLOGY School of Electrical and Computer Engineering ECE 6605 Information Theory Assigned: Tuesday, Sep. 7, 2010 Due: Atlanta and Savannah students: Friday, Sep. 17, 2010 Due: Video students: Wed., September 22, 2010 Problem Set #1 Problem 1-5: Solve the questions 2.1 (from page 43), 2.4, 2.7 (part a only), 2.17 and 2.18 from Chapter 2 of the textbook (the second edition). Problem 6: In class, we considered the discrete memoryless source ( A,B,C,D ) with prob- abilities (1 / 2 , 1 / 4 , 1 / 8 , 1 / 8), with the binary source code: A 00 B 01 C 10 D 11 (a) With this code, calculate the probability of the channel seeing a “0” and a “1”. (b) See if you can rearrange the four codewords { 00 , 01 , 10 , 11 } diFerently, so that these probabilities are both 1/2. (c) Explain why the rearranged code you found in part (b) is still not much good, despite Shannon’s prescription, which says that the code should cause the channel
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Background image of page 2
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: to see a sequence of independent, equally likely 0s and 1s. Problem 7: Let X , Y and Z be three discrete random variables. Using Jensens inequality, or otherwise (e.g., the chain rule), show that I ( X,Y ; Z ) I ( Y ; Z ). When does the equality hold? Problem 8: Let X 1 , X 2 be discrete random variables drawn according to probability mass functions p 1 ( ) and p 2 ( ) over the respective alphabets X 1 = { 1 , 2 ,... ,m } and X 2 = { m + 1 , 2 ,... ,n } . Let X = b X 1 with probability X 2 with probability 1- 1 Find H ( X ) in terms of H ( X 1 ) and H ( X 2 ) and . Hint: dene a function of X as : = f ( X ) = b 1 when X = X 1 2 when X = X 2 and use H ( X, ). 2...
View Full Document

This note was uploaded on 10/10/2010 for the course ECE 6605 taught by Professor Staff during the Fall '08 term at Georgia Institute of Technology.

Page1 / 2

ece6605-hwk1 - to see a sequence of independent, equally...

This preview shows document pages 1 - 2. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online