This preview shows pages 1–3. Sign up to view the full content.
This preview has intentionally blurred sections. Sign up to view the full version.View Full Document
Unformatted text preview: Classical Information Theory (Notes for Course on Quantum Computation and Information Theory. Sec. 11) Robert B. Griffiths Version of 17 Feb. 2003 References: CT = Cover and Thomas, Elements of Information Theory (Wiley, 1991): chapters 2, 3, and 8. QCQI = Quantum Computation and Quantum Information by Nielsen and Chuang (Cambridge, 2000), Secs. 11.1, 11.2 11 Classical Information Theory 11.1 Introduction F In order to discuss quantum computing and quantum cryptography in the presence of noise, one needs to go beyond qualitative statements and introduce quantitative measures of quantum information. At the present time this task is far from complete, and it isn’t clear that we have the best approach. • The most useful quantitative quantum measures developed thus far are based on analo- gies with classical information theory, a well-developed subject with many applications to real communication problems. So learning something about it is worthwhile even if eventu- ally we have to use something quite different to handle quantum information. ◦ The material below is a rapid survey of certain ideas in classical information which look like they are of use in the quantum domain. • CT is a standard reference for (classical) information theory. QCQI provides a very abbreviated introduction to the subject with a stress on mathematical properties. The notes below are intended to be a more intuitive introduction to the subject, with proofs and lots of details left to QCQI and CT. 11.2 Shannon Entropy F Suppose we have a certain message we want to transmit from location A to location B . What resources are needed to get it from here to there? How long will it take if we have a channel with a capacity of c bits per second? If transmission introduces errors, what do we 1 do about them? These are the sorts of questions which are addressed by information theory as developed by Shannon and his successors. • There are, clearly, N = 2 n messages which can be represented by a string of n bits; e.g., 8 messages 000, 001,. . . 111 for n = 3. Hence if we are trying to transmit one of N distinct messages it seems sensible to define the amount of information carried by a single message to be log 2 N bits, which we hereafter abbreviate to log N (in contrast to ln N for the natural logarithm). ◦ Substituting some other base for the logarithm merely means multiplying log 2 N by a constant factor, which is, in effect, changing the units in which we measure information. E.g., ln N nits in place of log N bits. F A key observation is that if we are in the business of repeatedly transmitting messages from a collection of N messages, and if the messages can be assigned a non-uniform probability distribution , then it is, on average, possible to use fewer than log N bits per message in order to transmit them, or to store them....
View Full Document
This note was uploaded on 02/05/2012 for the course EE EE308 taught by Professor B.k.dey during the Spring '09 term at IIT Bombay.
- Spring '09