3
Definition: A discrete memoryless channel
is denoted by
()
Y
X
,

,
X
Y
p
,
where
X
and
Y
are finite sets,
( )
x
y
p

for
all
X
∈
x
and
Y
∈
y
, and
1

=
∑
y
x
y
p
for all
X
∈
x
.
Memoryless
( ) ( )
k
k
k
k
k
x
y
p
y
x
y
p

,

1
=
−
Assume no feedback
( ) ( )
1
1
1

,

−
−
−
=
k
k
k
k
k
x
x
p
y
x
x
p
DMC:
() ()
∏
=
=
n
i
i
i
n
n
x
y
p
x
y
p
1


(If you can get channel feedback, the above expression still holds, but it will not be
useful any more.)
Definition: An
n
M
,
code for the channel
( )
( )
Y
X
,

,
x
y
p
, consists of
1.
An index set
{}
M
,
,
1
L
2.
An encoding function:
{ }
n
M
X
→
,
,
1
L
(
)(
)
(
)
( )
() () () ()
M
x
M
x
M
x
M
X
M
x
x
x
X
x
x
x
X
n
n
n
n
n
n
,
,
,
2
,
,
2
,
2
2
2
1
,
,
1
,
1
1
1
2