This preview shows pages 1–2. Sign up to view the full content.
This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Unformatted text preview: Error Correcting Codes: Combinatorics, Algorithms and Applications (Fall 2007) Lecture 10: Shannons Theorem September 19, 2007 Lecturer: Atri Rudra Scribe: Atri Rudra & Michael Pfetsch In the last lecture, we proved part (2) of Shannons capacity theorem for the binary symmetric channel (BSC), which we restate here (throughout these notes we will use e BSC p as a shorthand for noise e from BSC p ): Theorem 0.1. Let p < 1 2 be a real number. For every < 1 2 p , the following statements are true for large enough integer n: 1. There exists a real > , and encoding function E : { , 1 } k { , 1 } n , and a decoding function D : { , 1 } n { , 1 } k , where k b (1 H ( p + )) n c such that the following holds for every m { , 1 } k : Pr e BSC p [ D ( E ( m ) + e ) 6 = m ] 2 n 2. If k d (1 H ( p ) + ) n e then for every encoding function E : { , 1 } { , 1 } n and decoding function D : { , 1 } n { , 1 } k the following is true for some m { , 1 } k : Pr e BSC p [ D ( E ( m ) + e ) 6 = m ] 1 2 In todays lecture, we will prove part (1) of Theorem 0.1 1 Proof overview The proof of part (1) of Theorem 0.1 will be accomplished by randomly selecting an encoding function E : { , 1 } k { , 1 } n . That is, for every m { , 1 } k pick E ( m ) uniformly and inde pendently at random from { , 1 } n . D will be the maximum likelihood decoding (MLD) function. The proof will have the following two steps: 1. Step 1 : For any arbitrary m { , 1 } k , we will show that for a random choice of E, the probability of failure, over BSC p noise, is small. This implies the existence of a good encoding function for any arbitrary message. 2. Step 2 : We will show a similar result for all m . This involves dropping half of the code words....
View
Full
Document
This note was uploaded on 01/02/2012 for the course CSE 545 taught by Professor Rudra during the Spring '11 term at SUNY Buffalo.
 Spring '11
 RUDRA
 Algorithms

Click to edit the document details