This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Unformatted text preview: Error Correcting Codes: Combinatorics, Algorithms and Applications (Fall 2007) Lecture 10: Shannon’s Theorem September 19, 2007 Lecturer: Atri Rudra Scribe: Atri Rudra & Michael Pfetsch In the last lecture, we proved part (2) of Shannon’s capacity theorem for the binary symmetric channel (BSC), which we restate here (throughout these notes we will use “ e ∼ BSC p ” as a shorthand for “noise e from BSC p ”): Theorem 0.1. Let ≤ p < 1 2 be a real number. For every < ε ≤ 1 2 p , the following statements are true for large enough integer n: 1. There exists a real δ > , and encoding function E : { , 1 } k → { , 1 } n , and a decoding function D : { , 1 } n → { , 1 } k , where k ≤ b (1 H ( p + ε )) n c such that the following holds for every m ∈ { , 1 } k : Pr e ∼ BSC p [ D ( E ( m ) + e ) 6 = m ] ≤ 2 δn 2. If k ≥ d (1 H ( p ) + ε ) n e then for every encoding function E : { , 1 } → { , 1 } n and decoding function D : { , 1 } n → { , 1 } k the following is true for some m ∈ { , 1 } k : Pr e ∼ BSC p [ D ( E ( m ) + e ) 6 = m ] ≥ 1 2 In today’s lecture, we will prove part (1) of Theorem 0.1 1 Proof overview The proof of part (1) of Theorem 0.1 will be accomplished by randomly selecting an encoding function E : { , 1 } k → { , 1 } n . That is, for every m ∈ { , 1 } k pick E ( m ) uniformly and inde pendently at random from { , 1 } n . D will be the maximum likelihood decoding (MLD) function. The proof will have the following two steps: 1. Step 1 : For any arbitrary m ∈ { , 1 } k , we will show that for a random choice of E, the probability of failure, over BSC p noise, is small. This implies the existence of a good encoding function for any arbitrary message. 2. Step 2 : We will show a similar result for all m . This involves dropping half of the code words....
View
Full Document
 Spring '11
 RUDRA
 Algorithms, Probability, error probability, Chernoff

Click to edit the document details