lect10 - Error Correcting Codes Combinatorics Algorithms...

Info iconThis preview shows pages 1–2. Sign up to view the full content.

View Full Document Right Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: Error Correcting Codes: Combinatorics, Algorithms and Applications (Fall 2007) Lecture 10: Shannon’s Theorem September 19, 2007 Lecturer: Atri Rudra Scribe: Atri Rudra & Michael Pfetsch In the last lecture, we proved part (2) of Shannon’s capacity theorem for the binary symmetric channel (BSC), which we restate here (throughout these notes we will use “ e ∼ BSC p ” as a shorthand for “noise e from BSC p ”): Theorem 0.1. Let ≤ p < 1 2 be a real number. For every < ε ≤ 1 2- p , the following statements are true for large enough integer n: 1. There exists a real δ > , and encoding function E : { , 1 } k → { , 1 } n , and a decoding function D : { , 1 } n → { , 1 } k , where k ≤ b (1- H ( p + ε )) n c such that the following holds for every m ∈ { , 1 } k : Pr e ∼ BSC p [ D ( E ( m ) + e ) 6 = m ] ≤ 2- δn 2. If k ≥ d (1- H ( p ) + ε ) n e then for every encoding function E : { , 1 } → { , 1 } n and decoding function D : { , 1 } n → { , 1 } k the following is true for some m ∈ { , 1 } k : Pr e ∼ BSC p [ D ( E ( m ) + e ) 6 = m ] ≥ 1 2 In today’s lecture, we will prove part (1) of Theorem 0.1 1 Proof overview The proof of part (1) of Theorem 0.1 will be accomplished by randomly selecting an encoding function E : { , 1 } k → { , 1 } n . That is, for every m ∈ { , 1 } k pick E ( m ) uniformly and inde- pendently at random from { , 1 } n . D will be the maximum likelihood decoding (MLD) function. The proof will have the following two steps: 1. Step 1 : For any arbitrary m ∈ { , 1 } k , we will show that for a random choice of E, the probability of failure, over BSC p noise, is small. This implies the existence of a good encoding function for any arbitrary message. 2. Step 2 : We will show a similar result for all m . This involves dropping half of the code words....
View Full Document

{[ snackBarMessage ]}

Page1 / 5

lect10 - Error Correcting Codes Combinatorics Algorithms...

This preview shows document pages 1 - 2. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online