lect02 - 6.895 Essential Coding Theory September 13, 2004...

Info iconThis preview shows pages 1–2. Sign up to view the full content.

View Full Document Right Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: 6.895 Essential Coding Theory September 13, 2004 Lecture 2 Lecturer: Madhu Sudan Scribe: Joungkeun Lim 1 Overview We consider the problem of communication in which a source wish to transmit information to a receiver. The transmission is conducted through a channel, which may generate errors in the information de- pending on the options. In this model, we will introduce Shannon’s coding theorem, which shows that depending on the properties of the source and the channel, the probability of the receiver’s restoring the original data varies with a threshold. 2 Shannon’s theory of information In this section we will discuss the main result from Shannon’s paper which was introduced in 1948 and founded the theory of information. There are three entities in Shannon’s model: • Source : The party which produces information by a probabilistic process. • Channel : The means of passing information from source to receiver. It may generate errors while transporting the information. • Receiver : The party which receives the information and tries to figure out information at source’s end. There are two options for channel, “Noisy” and “Noiseless” • Noisy channel : A channel that flips some bits of information sent across them. The bits that flips are determined by a probabilistic process. • Noiseless channel : A channel that perfectly transmits the information from source to receiver without any error. The source will generate and encode its message, and send it to receiver through the channel. When the message arrives, the receiver will decode the message. We want to find the encoding-decoding scheme which makes it possible for a receiver to restore the exact massage which a source sent. Shannon’s theorem states the conditions with which a restoration can be conducted with high probability....
View Full Document

Page1 / 3

lect02 - 6.895 Essential Coding Theory September 13, 2004...

This preview shows document pages 1 - 2. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online