#21
ECE 253a
Digital Image Processing
Pamela Cosman
11/30/10
Solutions to homework 5
These solutions are written by Dawei Wang.
Problem 1: Huffman Coding
a) One possible codeword assignment is:
Symbol
Probability
Codeword Length
Codeword
A
0.5
1
0
B
0.25
2
10
C
0.1
3
110
D
0.05
4
1110
E
0.05
5
11110
F
0.05
5
11111
Other correct answers are also acceptable, and the average length is 2 bits.
The entropy of this
R.V. is
H
(
1
2
,
1
4
,
1
10
,
1
20
,
1
20
,
1
20
)
≈
1
.
98 bits.
b) For quaternary Huffman coding, we merge 4 nodes with smallest probability to one node in every
iteration. For this particular example, one possible code can be:
Symbol
Probability
Codeword Length
Codeword
A
0.5
1
a
B
0.25
1
b
C
0.1
2
ca
D
0.05
2
cb
E
0.05
2
cc
F
0.05
2
cd
and the average length is 1.25 quaternary units.
The quaternary alphabet symbol d is not used in the first position of the codeword, and this
encoding method is not efficient. In general, if the cardinality of input symbols is not in the form
of 3
N
+ 1,
N
∈
Q
+
for quaternary code ((
M

1)
N
+ 1,
N
∈
Q
+
for
M
ary code), we would waste
some codeword length in the final step of the iteration. To solve this problem, a zeroprobability
symbol padding is used.
Basically, we introduce some dummy symbols with zero probability to
make the starting number of input symbols be of the form 3
N
+ 1,
N
∈
Q
+
. For this particular
example:
Symbol
Probability
Codeword Length
Codeword
A
0.5
1
a
B
0.25
1
b
C
0.1
1
c
D
0.05
2
da
E
0.05
2
db
F
0.05
2
dc
G(dummy)
0
2
dd
1
This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
and the average length is 1.15 quaternary units.
Because the use of dummy symbols was not explained to you, both answers to this problem are
acceptable.
c) If we generate a binary code from the first quaternary code of part b), the codewords are:
Symbol
Probability
Codeword Length
Codeword
A
0.5
2
00
B
0.25
2
01
C
0.1
4
1000
D
0.05
4
1001
E
0.05
4
1010
F
0.05
4
1011
and the average length of this code is 2.5 bits, which is longer than 2 bits in part a).
This is the end of the preview.
Sign up
to
access the rest of the document.
 Fall '11
 Cosman
 Image processing, Allocation of resources, Digital image, Gonzalez Mask, Quternary Codeword

Click to edit the document details