1
Colorado State University, Ft. Collins
Fall 2008
ECE 516: Information Theory
Lecture 9 September 25, 2008
Recap:
Write
( )
∑
=
=
X
1
0
i
i
i
e
p
α
, where
{
}
i
e
are the set of linear independent eigenvectors,
i
e
corresponding to eigenvalue
i
λ
. Assume
L
≥
>
=
2
1
1
λ
λ
. Then,
( )
(
)
( )
(
)
∑
∑
=
=
=
=
=
X
X
1
1
0
i
i
i
i
i
i
T
i
T
n
e
e
P
p
P
p
n
n
n
λ
α
α
As
∞
→
n
,
( )
1
1
e
p
α
=
n
.
(scaling is not important)
If a Markov chain is irreducible and aperiodic, and if it is run for a long time, so
( )
μ
p
→
n
, then it us a stationary ergodic process.
(
)
(
)
(
)
(
)
(
)
∑
∑
∈
∈
∞
→
−
∞
→
−
∞
→
=
=
=
=
′
=
X
X
i
ij
j
ij
i
n
n
n
n
n
n
n
P
P
X
X
H
X
X
H
X
X
X
H
H
H
1
log

lim

lim
,
,

lim
1
2
1
1
1
μ
L
X
X
(
)
(
)
X
log
≤
≤
n
X
H
H
X
5
Data Compression (Chp. 5)
5.1
Example of Codes
Definition: A source code
C
for a r.v.
X
is a mapping from
X
to
*
D
, the finite
length strings of symbols from a Dary alphabet.
Definition: The expected length
(
)
C
L
of the source code
( )
x
C
for a r.v.
X
with
( )
x
p
(
)
( ) ( )
∑
∈
=
X
x
x
l
x
p
C
L
Definition: A code is nonsingular if every element of
X
maps into a different
string in
*
D
, i.e.,
(
)
(
)
j
i
j
i
x
C
x
C
x
x
≠
⇒
≠
This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document