Information Theory
EE253
by
Hamid R. Sadjadpour
Email: [email protected]
Chapter 1
Pre-requisites:
Set theory, very little,
Probability theory,
Knowledge of random variables,
Knowledge of random process, very little.
P1 Notes.
UCBC
Chapter 1
UCBC
Information Theory
EE253
Asymptotic Equipartition Property (AEP)
by
Hamid R. Sadjadpour
Email: [email protected]
Chapter 3
UCBC
AEP in information theory is equivalent of law of larger numbers
in probability theory.
1 n
Law of large numbers : For i.i.d.
Information Theory
EE253
Entropy Rates Of A Stochastic Process
by
Hamid R. Sadjadpour
Email: [email protected]
EE 253
1
Chapter 4
UCBC
AEP states that nH(X) bits is sufficient on average to describe n i.i.d.
RVs.
Now what should we do if the RVs are depe
Information Theory
EE253
Data Compression
by
Hamid R. Sadjadpour
Email: [email protected]
1
Chapter 5-Data Compression
UCBC
We will now use our results in entropy and typical sequences
to represent data sequences with minimum number of bits, on average.
Information Theory
EE253
Channel Capacity
by
Hamid R. Sadjadpour
Email: [email protected]
Chapter 7-Channel Capacity
UCBC
In this chapter, we discuss the channel capacity.
By capacity, we mean the maximum amount of informatio n
that can be submitted thro