1.3.4 An extended binary Hamming code
An extension of a binary Hamming code results from adding at the beginning of each
codeword a new symbol that checks the parity of the codeword. To the [7, 4] Hamming
code we add an initial symbol:
X0 is chosen to mak
Incomplete Maximum Likelihood Decoding When y is received, we must decode either to a codeword x that maximizes
Prob(y | x) or to the error detected symbol .
Of course, if we are only interested in maximizing our chance of successful decoding, then any gu
2.3. SHANNONS THEOREM AND THE CODE REGION 25
We divide by n and move the second term across the inequality to nd (C) = n1 logm(|C|) 1 n1 logm(|Ssn()|) .
The righthand side approaches 1 Hm(s) = Cm(p) as n goes to innity; so, for C to be a contributing memb
For a xed > (m 1)/m, the Plotkin bound 2.3.8 says that code size is bounded by a
constant. Thus as n goes to innity, the rate goes to 0, hence (1) of the corollary. Part (2)
is proven by applying the Plotkin bound not to the code C but to a related code C
Chapter 2
Sphere Packing and Shannons Theorem
In the rst section we discuss the basics of block coding on the m-ary symmetric channel. In the second section we see how
the geometry of the codespace can be used to make coding judgements. This leads to the
A code C that satises the three equivalent properties of Lemma 2.2.1 is called an e-error-correcting code. The lemma
reveals one of the most pleasing aspects of coding theory by identifying concepts from three distinct and impor- tant areas.
The rst prope
1.2.1 Message
Our basic assumption on messages is that each possible message k-tuple is as likely to be selected for broadcast as any
other.
We are thus ignoring the concerns of source coding. Perhaps a better way to say this is that we assume source codi
Claude Shannons 1948 paper A Mathematical Theory of Communication gave birth to the twin disciplines of
information theory and coding theory. The basic goal is efcient and reliable communication in an uncooperative (and possibly hostile) environment. To b
There are many situations in which we encounter other related types of com- munication. Cryptography is certainly
concerned with communication, however the emphasis is not on efciency but instead upon security. Nevertheless modern
cryptography shares cert
The Fundamental Problem Find codes with both reasonable information content and reasonable error handling
ability.
Is this even possible? The rather surprising answer is, Yes! The existence of such codes is a consequence of the
Channel Coding Theorem from