This preview has intentionally blurred sections. Sign up to view the full version.View Full Document
Unformatted text preview: ECE 1502 — Information Theory November 16, 2007 7.9 Suboptimal codes . From the proof of the channel coding theorem, it follows that using a random code with codewords generated according to probability p ( x ), we can send information at a rate I ( X ; Y ) corresponding to that p ( x ) with an arbitrarily low probability of error. For the Z channel described in the previous problem, we can calculate I ( X ; Y ) for a uniform distribution on the input. The distribution on Y is (3/4, 1/4), and therefore I ( X ; Y ) = H ( Y )- H ( Y | X ) = H ( 3 4 , 1 4 )- 1 2 H ( 1 2 , 1 2 ) = 3 2- 3 4 log 3 . (1) 7.14 Channels with dependence between the letters . (a) First, I ( X 1 , X 2 ; Y 1 , Y 2 ) = H ( X 1 , X 2 )- H ( X 1 , X 2 | Y 1 , Y 2 ) = H ( X 1 , X 2 ) , where the last equality follows from the fact that ( Y 1 , Y 2 ) uniquely identifies ( X 1 , X 2 ). (b) C = max p ( x 1 ,x 2 ) H ( X 1 , X 2 ) = log 2 (4) = 2 , when the four input pairs are used equiprobably. (c) When the input pairs are equiprobable, it is easy to show that p ( x 1 | y 1 ) = p ( x 1 ) . Therefore, I ( X 1 ; Y 1 ) = H ( X 1 )- H ( X 1 | Y 1 ) = H ( X 1 )- H ( X 1 ) = . 7.19 Capacity of the carrier pigeon channel . (a) Since a pigeon arrives at the destination every 5 minutes, there are 12 pigeons per hour that arrive safely. Each pigeon carries an 8 bit message, thus the capacity of the link is 96 bits/hour. Effectively, we get to use an errorless channel (with 256 inputs and 256 outputs, and thus capacity C 1 = log 2 (256) = 8 bits/channel-use) 12 times per hour. 1 (b) We can model this problem using an erasure channel with 256 inputs and 257 outputs (256 output symbols that match the inputs, and one erasure); the input is received correctly with probability 1- α , and an erasure occurs with probability α . Just an in the binary-input erasure channel, it is easy to show that the capacity is C 2 = (1- α ) log 2 (256) = 8(1- α ) bits/channel-use, which is equivalent to 96(1- α ) bits/hour. (c) We can model this problem using a 256-ary symmetric channel (just like the BSC, but with 256 inputs and 256 outputs). A pigeon arrives safely with probability 1- α , and if the pigeon is shot down, the probability that the dummy carries the intended message is 1 256 . Therefore, the message is received correctly with probability....
View Full Document
- Spring '10
- Normal Distribution, Probability theory, Yi, max H