1
Colorado State University, Ft. Collins
Fall 2008
ECE 516: Information Theory
Lecture 1 August 26, 2008
Syllabus
Instructor:
J. Rockey Luo, B118, Engr Bld., (970) 4917411,
[email protected]
Office Hours:
Tuesday Thursday afternoon, or by appointment.
Prerequisites:
ECE/STAT303, ECE421/Instructor’s permission
Textbook:
Thomas M. Cover and Joy A Thomas, “Elements of Information Theory”,
2
nd
Edition, John Wiley & Sons, New Jersey, 2005. (ISBN 0471241954)
Webpage:
http://www.engr.colostate.edu/ECE516/
Course Outline:
1.
Entropy, relative entropy and mutual information (Chapter 2)
2.
The asymptotic equipartition property (Chapter 3)
3.
Entropy rate of stochastic processes (Chapter 4)
4.
Data compression (Chapter 5)
5.
Channel capacity (Chapter 7)
6.
Differential entropy (Chapter 8)
7.
The Gaussian channel (Chapter 9)
8.
Network information theory (Chapter 15)
9.
Rate distortion theory (Chapter 10)
Grading:
Homeworks (60%), Final Presentation (40%)
Homework will be assigned weekly, due in two weeks before the class. Collaborations are
encouraged. Late submission gets 70%. Copying gets 30%. No submission gets 0. You have
to write clearly whether you collaborated with others or you copied others’ results.
Final requires a 20minute presentation of a fulllength journal paper of your choice. You
should justify why it is suitable for the information theory course.
No class on Sep. 4 and Oct. 16.
This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
2
0
Nontechnical Remarks about Information Theory
Information Theory is a study of fundamental limits on the performance of
communication systems.
Started with C. E. Shannon’s “The Mathematical Theory of Communication”, 1948,
reprint: 1963
Claude E. Shannon, 04/30/1916 – 02/24/2001
Information Theory is a major breakthrough that transformed conventional
communication to modern communication.
0.1
What to learn from Information Theory
A practical system always has many interacting parameters. Information Theory
showed us an example of abstracting highly complex systems, and obtaining
fundamental understandings about system design.
Key objective: learning about how to formulate a research problem.
Note that Information Theory only provides an example. We need to follow
Shannon’s soul, but not necessarily his method.
0.2
Some properties of this Information Theory course
Highly mathematical. But you do have the necessary background.
Example: A number game. Give you four numbers, you use basic arithmetic
operations to connect the four numbers to get the result 24.
This is the end of the preview.
Sign up
to
access the rest of the document.
 Spring '08
 Rocky
 Information Theory

Click to edit the document details