Information Theory, Spring 2012
Communications Engineering Department
School of Electrical & Computer Engineering
University of Tehran
Assignment Set 3
Variable Length Codes, Arithmetic codes
t
Esfand 28, 1390
Due date: Thursday, Farvardin 17, 1391, 12:00
EE376B/Stat 376B
Information Theory
Prof. T. Cover
Handout #12
Thursday, May 17, 2011
Due Thursday, May 24, 2011
Homework Set #6
1. One bit quantization of a single Gaussian random variable
Let X N (0, 2 ) and let the distortion measure be squared error.
EE376B/Stat 376B
Information Theory
Prof. T. Cover
Handout #11
Thursday, May 12, 2011
Due Thursday, May 19, 2011
Homework Set #5
1. Random program. Will the sun rise tomorrow?
Suppose that a random program (symbols i.i.d. uniform over the symbol set) is f
EE376B
Information Theory
Prof. T. Cover
Handout #10
Tuesday, May 2, 2011
Due Tuesday, May 9, 2011
Homework Set #4
1. Maximum Entropy and Counting
Let X = cfw_1, 2, . . . , m. Show that the number of sequences xn X n satisfying
1 n
nH
, to rst order in t
EE376B
Information Theory
Prof. T. Cover
Handout #6
Tuesday, April 19, 2011
Due Tuesday, May 03, 2011
Homework Set #3
1. The cooperative capacity of a multiple access channel.
B
X1
(W1 , W2 ) rr
rr
j
r
X2
E
p(y |x1 , x2 )
EY
E (W , W )
1 2
E
Figure 1: Mu
EE376B
Information Theory
Prof. T. Cover
Handout #4
Tuesday, April 12, 2011
Due Tuesday, April 19, 2011
Homework Set #2
1. Multiple layer waterlling
Let C (x) = 1 log(1 + x) denote the channel capacity of a Gaussian channel with signal
2
to noise ratio x.
ELG 5170 Information Theory Midterm Exam
30 points, 2 Hours, Close-Book, Calculator Allowed
Oct 23, 2004
Warning: Questions are not necessarily arranged in their level of diculties.
Question 1 (20 points) Let X be a random variable taking values from X :=
Islamic University of Gaza
Faculty of Engineering
ECE Department
Fall 2007
Duration 1:15
Instructor: Dr. Ammar Abu Hudrouss
T.A. Eng.
Eng.
Information Theory and Coding
Midterm Exam
Student should attempt all questions.
This exam is a closed-book exam
Que
ECE 776 - Information theory Midterm Q1 (1 point). Given the channel p(y|x) 1/3 2/3 0 p(y|x) = 0 1/3 2/3 , 2/3 0 1/3
calculate the capacity.
Sol.: The channel is symmetric since each row is a permutation of the other rows and the sums on each column are
ECE 534 Information Theory - MIDTERM 1
09/29/2010, LH 305.
This exam has 4 questions, each of which is worth 25 points.
You will be given the full 1.25 hours. Use it wisely! Many of the problems have short answers; try
to nd shortcuts. Do questions that
Harvard SEAS
ES250 Information Theory
Midterm Exam
Solutions
1. (15 points) Let X be the waiting time for the rst heads to appear in successive ips of a fair coin.
Thus, for example, Prcfw_X = 3 = ( 1 )3 .
2
(a) (5 points) Find H (X ).
(b) (5 points) Find
Information Theory, Winter 2012
Communications Engineering Department
School of Electrical & Computer Engineering
University of Tehran
Assignment Set 1
Bahman 30, 1390
Due Date: Monday, Esfand 8, 1390
1.
( , ) + ( , ) + ( , ) 2( , , )
. Z Y X .2
H(X,Y,Z
Information Theory, Spring 2012
Communications Engineering Department
School of Electrical & Computer Engineering
University of Tehran
Assignment Set 2
Esfand 9, 1390
Due Date: Esfand 16, 1390
1. (AEP and Mutual Information) Let ( , ) be . . . ~(, ). we f
InformationTheory,Spring2012
CommunicationsEngineeringDepartment
SchoolofElectrical&ComputerEngineering
UniversityofTehran
Assignment Set 4
Channel Capacity
Farvardin27, 1391
Due date: Sunday, Ordibehesht 3, 1391
1. Compare the achieved information of
,
EE376B/Stat 376B
Information Theory
Prof. T. Cover
Handout #15
Thursday, May 26, 2011
Due Tuesday, May 31, 2011
Homework Set #7
1. Growth rate. Let
cfw_
X=
(1, a),
(1, 1/a),
with probability 1/2
,
with probability 1/2
where a > 1. This vector X represents