# lect02 - 6.895 Essential Coding Theory Lecture 2 Lecturer...

This preview shows pages 1–2. Sign up to view the full content.

6.895 Essential Coding Theory September 13, 2004 Lecture 2 Lecturer: Madhu Sudan Scribe: Joungkeun Lim 1 Overview We consider the problem of communication in which a source wish to transmit information to a receiver. The transmission is conducted through a channel, which may generate errors in the information de- pending on the options. In this model, we will introduce Shannon’s coding theorem, which shows that depending on the properties of the source and the channel, the probability of the receiver’s restoring the original data varies with a threshold. 2 Shannon’s theory of information In this section we will discuss the main result from Shannon’s paper which was introduced in 1948 and founded the theory of information. There are three entities in Shannon’s model: Source : The party which produces information by a probabilistic process. Channel : The means of passing information from source to receiver. It may generate errors while transporting the information. Receiver : The party which receives the information and tries to figure out information at source’s end. There are two options for channel, “Noisy” and “Noiseless” Noisy channel : A channel that ﬂips some bits of information sent across them. The bits that ﬂips are determined by a probabilistic process. Noiseless channel : A channel that perfectly transmits the information from source to receiver without any error. The source will generate and encode its message, and send it to receiver through the channel. When the message arrives, the receiver will decode the message. We want to find the encoding-decoding scheme which makes it possible for a receiver to restore the exact massage which a source sent. Shannon’s theorem states the conditions with which a restoration can be conducted with high probability.

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

### What students are saying

• As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

Kiran Temple University Fox School of Business ‘17, Course Hero Intern

• I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

Dana University of Pennsylvania ‘17, Course Hero Intern

• The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

Jill Tulane University ‘16, Course Hero Intern