This preview shows pages 1–2. Sign up to view the full content.
This preview has intentionally blurred sections. Sign up to view the full version.View Full Document
Unformatted text preview: 6.895 Essential Coding Theory September 13, 2004 Lecture 2 Lecturer: Madhu Sudan Scribe: Joungkeun Lim 1 Overview We consider the problem of communication in which a source wish to transmit information to a receiver. The transmission is conducted through a channel, which may generate errors in the information de- pending on the options. In this model, we will introduce Shannon’s coding theorem, which shows that depending on the properties of the source and the channel, the probability of the receiver’s restoring the original data varies with a threshold. 2 Shannon’s theory of information In this section we will discuss the main result from Shannon’s paper which was introduced in 1948 and founded the theory of information. There are three entities in Shannon’s model: • Source : The party which produces information by a probabilistic process. • Channel : The means of passing information from source to receiver. It may generate errors while transporting the information. • Receiver : The party which receives the information and tries to figure out information at source’s end. There are two options for channel, “Noisy” and “Noiseless” • Noisy channel : A channel that ﬂips some bits of information sent across them. The bits that ﬂips are determined by a probabilistic process. • Noiseless channel : A channel that perfectly transmits the information from source to receiver without any error. The source will generate and encode its message, and send it to receiver through the channel. When the message arrives, the receiver will decode the message. We want to find the encoding-decoding scheme which makes it possible for a receiver to restore the exact massage which a source sent. Shannon’s theorem states the conditions with which a restoration can be conducted with high probability....
View Full Document
- Spring '10