lect11 - Error Correcting Codes: Combinatorics, Algorithms...

Info iconThis preview shows pages 1–3. Sign up to view the full content.

View Full Document Right Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: Error Correcting Codes: Combinatorics, Algorithms and Applications (Fall 2007) Lecture 11: Shannon vs. Hamming September 21,2007 Lecturer: Atri Rudra Scribe: Kanke Gao & Atri Rudra In the last lecture, we proved the positive part of Shannon’s capacity theorem for the BSC. We showed that by the probabilistic method, there exists an encoding function E and a decoding function D such that E m Pr noise e of BSC p [ D ( E ( m ) + e ) negationslash = m ] ≤ 2 − δ ′ n . (1) In other words, the average decoding error probability is small. However, we need to show that the maximum decoding error probability over all messages is small. In the last lecture, we quickly went over how (1) implies the latter. We will start today’s lecture by going over this argument again. 1 Shannon’s Capacity Theorem ( Cont. ) As was mentioned in the last lecture, the trick is to throw away all the messages that have high error probability. In particular, we only keep the messages with probability error at most the median error probability. Claim 1.1. Let the messages be ordered by m 1 , m 2 ,..., m 2 k and define P i = Pr noise e ofBSC p [ D ( E ( m i ) + e ) negationslash = m i ] . Assume that P 1 ≤ P 2 ≤ ... ≤ P 2 k and (1) holds, then P 2 k − 1 ≤ 2 · 2 − δ ′ n Proof. By the definition of P i , 1 2 k 2 k summationdisplay i =1 P i = E m Pr noise e ofBSC p [ D ( E ( m ) + e ) negationslash = m ] ≤ 2 − δ ′ n , (2) where (2) follows from (1). For the sake of contradiction assume that P 2 k − 1 > 2 · 2 − δ ′ n . (3) So, 1 2 k 2 k summationdisplay i =1 P i ≥ 1 2 k 2 k summationdisplay i =2 k − 1 +1 P i (4) 1 > 2 · 2 − δ ′ n · 2 k − 1 2 k (5) > 2 − δ ′ n , (6) where (4) follows by dropping half the summands from the sum. (5) follows (3) and the assump-where (4) follows by dropping half the summands from the sum....
View Full Document

Page1 / 4

lect11 - Error Correcting Codes: Combinatorics, Algorithms...

This preview shows document pages 1 - 3. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online