Lecture1 - Colorado State University, Ft. Collins ECE 516:...

Info iconThis preview shows pages 1–3. Sign up to view the full content.

View Full Document Right Arrow Icon
1 Colorado State University, Ft. Collins Fall 2008 ECE 516: Information Theory Lecture 1 August 26, 2008 Syllabus Instructor: J. Rockey Luo, B118, Engr Bld., (970) 491-7411, rockey@engr.colostate.edu Office Hours: Tuesday Thursday afternoon, or by appointment. Prerequisites: ECE/STAT303, ECE421/Instructor’s permission Textbook: Thomas M. Cover and Joy A Thomas, “Elements of Information Theory”, 2 nd Edition, John Wiley & Sons, New Jersey, 2005. (ISBN 0-471-24195-4) Webpage: http://www.engr.colostate.edu/ECE516/ Course Outline: 1. Entropy, relative entropy and mutual information (Chapter 2) 2. The asymptotic equipartition property (Chapter 3) 3. Entropy rate of stochastic processes (Chapter 4) 4. Data compression (Chapter 5) 5. Channel capacity (Chapter 7) 6. Differential entropy (Chapter 8) 7. The Gaussian channel (Chapter 9) 8. Network information theory (Chapter 15) 9. Rate distortion theory (Chapter 10) Grading: Homeworks (60%), Final Presentation (40%) Homework will be assigned weekly, due in two weeks before the class. Collaborations are encouraged. Late submission gets 70%. Copying gets 30%. No submission gets 0. You have to write clearly whether you collaborated with others or you copied others’ results. Final requires a 20-minute presentation of a full-length journal paper of your choice. You should justify why it is suitable for the information theory course. No class on Sep. 4 and Oct. 16.
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
2 0 Non-technical Remarks about Information Theory Information Theory is a study of fundamental limits on the performance of communication systems. Started with C. E. Shannon’s “The Mathematical Theory of Communication”, 1948, reprint: 1963 Claude E. Shannon, 04/30/1916 – 02/24/2001 Information Theory is a major breakthrough that transformed conventional communication to modern communication. 0.1 What to learn from Information Theory A practical system always has many interacting parameters. Information Theory showed us an example of abstracting highly complex systems, and obtaining fundamental understandings about system design. Key objective: learning about how to formulate a research problem. Note that Information Theory only provides an example. We need to follow Shannon’s soul, but not necessarily his method. 0.2 Some properties of this Information Theory course Highly mathematical. But you do have the necessary background. Example: A number game. Give you four numbers, you use basic arithmetic operations to connect the four numbers to get the result 24. The four numbers: 2 5 6 7. Answer: 2x6+5+7=24 Easy The numbers: 1 5 5 5. Not that easy. But, you do have the necessary background! Theoretical understandings may seem to be useless.
Background image of page 2
Image of page 3
This is the end of the preview. Sign up to access the rest of the document.

This note was uploaded on 03/17/2010 for the course ECE 516 taught by Professor Rocky during the Spring '08 term at Colorado State.

Page1 / 9

Lecture1 - Colorado State University, Ft. Collins ECE 516:...

This preview shows document pages 1 - 3. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online