The Hebrew University of Jerusalem
Introduction to Information Theory Course 67548, winter 2007-2008. Final examination, moed B.
Lecturer: Prof. Michael Werman June 15, 2008
Instructions: There are 4 questions (some of which are subdivided into parts). Yo
Proof of McMillans Theorem: One Direction
Ohad Shamir November 16, 2008
The following is a proof of one direction in McMillans Theorem - we saw the other direction in class. Before we begin, a reminder about notation: we assume there is a source (makor) X
The Hebrew University of Jerusalem
Introduction to Information Theory Course 67548, winter 2007-2008. Final examination, moed A.
Lecturer: Prof. Michael Werman May 1, 2008
Instructions: There are 4 questions (some of which are subdivided into parts). You
Introduction to Information Theory (67548)
January 27, 2009
Assignment 5- Solution
Lecturer: Prof. Michael Werman Due:
Note: Unless specied otherwise, all entropies and logarithms should be taken with base 2.
Problem 1
Another Error Correcting Codes Bound
Introduction to Information Theory (67548)
January 12, 2009
Assignment 4: Gaussian Channel and Dierential Entropy
Lecturer: Prof. Michael Werman Due: Sunday, January 25, 2009
Note: Unless specied otherwise, all entropies and logarithms should be taken wit
Introduction to Information Theory (67548)
December 25, 2008
Assignment 3
Lecturer: Prof. Michael Werman Due: Sunday, January 11, 2009
Note: Unless specied otherwise, all entropies and logarithms should be taken with base 2.
Problem 1
Binary Channel
1. We
Introduction to Information Theory (67548)
December 25, 2008
Assignment 3: Channels and Coding
Lecturer: Prof. Michael Werman Due: Sunday, January 11, 2009
Note: Unless specied otherwise, all entropies and logarithms should be taken with base 2.
Problem 1
Introduction to Information Theory (67548)
December 30, 2008
Assignment 2 - Solution
Lecturer: Prof. Michael Werman Due:
Note: Unless specied otherwise, all entropies and logarithms should be taken with base 2.
Problem 1
AEP and Source Coding
100 100 + .
Introduction to Information Theory (67548)
December 5, 2008
Assignment 2
Lecturer: Prof. Michael Werman Due: Sunday, Dec. 21, 2008
Note: Unless specied otherwise, all entropies and logarithms should be taken with base 2.
Problem 1
AEP and Source Coding
A
Introduction to Information Theory (67548)
December 21, 2008
Assignment 1: Solution
Lecturer: Prof. Michael Werman Important note: All logarithms are in base 2 unless specied otherwise. Due:
Problem 1
1.
Joint Distribution and Entropy
p(x, y ) log(p(x, y
Introduction to Information Theory (67548)
November 27, 2008
Assignment 0: Solution
Lecturer: Prof. Michael Werman Due:
Problem 1
Part (a)
Probability and Conditional Probability
Leting A be the event that there is a girl in the family, and B be the event
Introduction to Information Theory (67548)
January 12, 2009
Assignment 4: Gaussian Channel and Dierential Entropy
Lecturer: Prof. Michael Werman Due: Sunday, January 25, 2009
Note: Unless specied otherwise, all entropies and logarithms should be taken wit
Introduction to Information Theory (67548)
November 10, 2008
Assignment 0: Probability Review
Lecturer: Prof. Michael Werman Due: Sunday, Nov. 16, 2008
Problem 1
Probability and Conditional Probability
Suppose that in all families with two children, all f
Conditional Mutual Information
Ohad Shamir December 2, 2008
We have seen in class the standard denition of mutual information I (X ; Y ) between two random variables: I (X ; Y ) =
x,y
p(x, y ) log
p(x, y ) . p(x)p(y )
We have also seen that mutual informa