This preview shows pages 1–3. Sign up to view the full content.
This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Unformatted text preview: Error Correcting Codes: Combinatorics, Algorithms and Applications (Fall 2007) Lecture 11: Shannon vs. Hamming September 21,2007 Lecturer: Atri Rudra Scribe: Kanke Gao & Atri Rudra In the last lecture, we proved the positive part of Shannon’s capacity theorem for the BSC. We showed that by the probabilistic method, there exists an encoding function E and a decoding function D such that E m Pr noise e of BSC p [ D ( E ( m ) + e ) negationslash = m ] ≤ 2 − δ ′ n . (1) In other words, the average decoding error probability is small. However, we need to show that the maximum decoding error probability over all messages is small. In the last lecture, we quickly went over how (1) implies the latter. We will start today’s lecture by going over this argument again. 1 Shannon’s Capacity Theorem ( Cont. ) As was mentioned in the last lecture, the trick is to throw away all the messages that have high error probability. In particular, we only keep the messages with probability error at most the median error probability. Claim 1.1. Let the messages be ordered by m 1 , m 2 ,..., m 2 k and define P i = Pr noise e ofBSC p [ D ( E ( m i ) + e ) negationslash = m i ] . Assume that P 1 ≤ P 2 ≤ ... ≤ P 2 k and (1) holds, then P 2 k − 1 ≤ 2 · 2 − δ ′ n Proof. By the definition of P i , 1 2 k 2 k summationdisplay i =1 P i = E m Pr noise e ofBSC p [ D ( E ( m ) + e ) negationslash = m ] ≤ 2 − δ ′ n , (2) where (2) follows from (1). For the sake of contradiction assume that P 2 k − 1 > 2 · 2 − δ ′ n . (3) So, 1 2 k 2 k summationdisplay i =1 P i ≥ 1 2 k 2 k summationdisplay i =2 k − 1 +1 P i (4) 1 > 2 · 2 − δ ′ n · 2 k − 1 2 k (5) > 2 − δ ′ n , (6) where (4) follows by dropping half the summands from the sum. (5) follows (3) and the assumpwhere (4) follows by dropping half the summands from the sum....
View
Full
Document
 Spring '10
 Rejaei
 Electromagnet

Click to edit the document details