ECE 255AN
Fall 2011
Homework set 5  solutions
Solutions to Chapter 7 problems
28.
Choice of Channels
(a) This is solved by using the very same trick that was used to solve problem 2.10.
Consider the following communication scheme:
X
=
(
X
1
Probability
α
X
2
Probability (1

α
)
Let
θ
(
X
) =
(
1
X
=
X
1
2
X
=
X
2
Since the output alphabets
Y
1
and
Y
2
are disjoint,
θ
is a function of
Y
as well, i.e.
X
→
Y
→
θ
.
I
(
X
;
Y, θ
)
=
I
(
X
;
θ
) +
I
(
X
;
Y

θ
)
=
I
(
X
;
Y
) +
I
(
X
;
θ

Y
)
Since
X
→
Y
→
θ
,
I
(
X
;
θ

Y
) = 0. Therefore,
I
(
X
;
Y
)
=
I
(
X
;
θ
) +
I
(
X
;
Y

θ
)
=
H
(
θ
)

H
(
θ

X
) +
αI
(
X
1
;
Y
1
) + (1

α
)
I
(
X
2
;
Y
2
)
=
H
(
α
) +
αI
(
X
1
;
Y
1
) + (1

α
)
I
(
X
2
;
Y
2
)
Thus, it follows that
C
= sup
α
{
H
(
α
) +
αC
1
+ (1

α
)
C
2
}
.
Maximizing over
α
one gets the desired result. The maximum occurs for
H
0
(
α
)+
C
1

C
2
= 0,
or
α
= 2
C
1
/
(2
C
1
+ 2
C
2
).
(b) If one interprets
M
= 2
C
as the effective number of noise free symbols, then the above
result follows in a rather intuitive manner: we have
M
1
= 2
C
1
noise free symbols from channel
1, and
M
2
= 2
C
2
noise free symbols from channel 2. Since at each step we get to chose which
This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
This is the end of the preview.
Sign up
to
access the rest of the document.
 Fall '09
 The Parallel, Greatest element, English Channel, Upper and lower bounds

Click to edit the document details