# 7.14 - ECE 1502 — Information Theory 7.9 Suboptimal codes...

This preview shows pages 1–3. Sign up to view the full content.

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: ECE 1502 — Information Theory November 16, 2007 7.9 Suboptimal codes . From the proof of the channel coding theorem, it follows that using a random code with codewords generated according to probability p ( x ), we can send information at a rate I ( X ; Y ) corresponding to that p ( x ) with an arbitrarily low probability of error. For the Z channel described in the previous problem, we can calculate I ( X ; Y ) for a uniform distribution on the input. The distribution on Y is (3/4, 1/4), and therefore I ( X ; Y ) = H ( Y )- H ( Y | X ) = H ( 3 4 , 1 4 )- 1 2 H ( 1 2 , 1 2 ) = 3 2- 3 4 log 3 . (1) 7.14 Channels with dependence between the letters . (a) First, I ( X 1 , X 2 ; Y 1 , Y 2 ) = H ( X 1 , X 2 )- H ( X 1 , X 2 | Y 1 , Y 2 ) = H ( X 1 , X 2 ) , where the last equality follows from the fact that ( Y 1 , Y 2 ) uniquely identifies ( X 1 , X 2 ). (b) C = max p ( x 1 ,x 2 ) H ( X 1 , X 2 ) = log 2 (4) = 2 , when the four input pairs are used equiprobably. (c) When the input pairs are equiprobable, it is easy to show that p ( x 1 | y 1 ) = p ( x 1 ) . Therefore, I ( X 1 ; Y 1 ) = H ( X 1 )- H ( X 1 | Y 1 ) = H ( X 1 )- H ( X 1 ) = . 7.19 Capacity of the carrier pigeon channel . (a) Since a pigeon arrives at the destination every 5 minutes, there are 12 pigeons per hour that arrive safely. Each pigeon carries an 8 bit message, thus the capacity of the link is 96 bits/hour. Effectively, we get to use an errorless channel (with 256 inputs and 256 outputs, and thus capacity C 1 = log 2 (256) = 8 bits/channel-use) 12 times per hour. 1 (b) We can model this problem using an erasure channel with 256 inputs and 257 outputs (256 output symbols that match the inputs, and one erasure); the input is received correctly with probability 1- α , and an erasure occurs with probability α . Just an in the binary-input erasure channel, it is easy to show that the capacity is C 2 = (1- α ) log 2 (256) = 8(1- α ) bits/channel-use, which is equivalent to 96(1- α ) bits/hour. (c) We can model this problem using a 256-ary symmetric channel (just like the BSC, but with 256 inputs and 256 outputs). A pigeon arrives safely with probability 1- α , and if the pigeon is shot down, the probability that the dummy carries the intended message is 1 256 . Therefore, the message is received correctly with probability....
View Full Document

{[ snackBarMessage ]}

### Page1 / 6

7.14 - ECE 1502 — Information Theory 7.9 Suboptimal codes...

This preview shows document pages 1 - 3. Sign up to view the full document.

View Full Document
Ask a homework question - tutors are online