# Sphere - Chapter 2 Sphere Packing and Shannon's Theorem In...

This preview shows pages 1–3. Sign up to view the full content.

Chapter 2 Sphere Packing and Shannon’s Theorem In the ﬁrst section we discuss the basics of block coding on the m -ary symmetric channel. In the second section we see how the geometry of the codespace can be used to make coding judgements. This leads to the third section where we present some information theory and Shannon’s basic Channel Coding Theorem. 2.1 Basics of block coding on the m SC Let A be any ﬁnite set. A block code or code , for short, will be any nonempty block code subset of the set A n of n -tuples of elements from A . The number n = n ( C ) is the length of the code, and the set A n is the codespace . The number of members length codespace in C is the size and is denoted | C | . If C has length n and size | C | , we say that size C is an ( n, | C | ) code . ( n, | C | ) code The members of the codespace will be referred to as words , those belonging words to C being codewords . The set A is then the alphabet . codewords alphabet If the alphabet A has m elements, then C is said to be an m -ary code . In m -ary code the special case | A | =2 we say C is a binary code and usually take A = { 0 , 1 } binary or A = {- 1 , +1 } . When | A | =3 we say C is a ternary code and usually take ternary A = { 0 , 1 , 2 } or A = {- 1 , 0 , +1 } . Examples of both binary and ternary codes appeared in Section 1.3. For a discrete memoryless channel, the Reasonable Assumption says that a pattern of errors that involves a small number of symbol errors should be more likely than any particular pattern that involves a large number of symbol errors. As mentioned, the assumption is really a statement about design. On an m SC( p ) the probability p ( y | x ) that x is transmitted and y is received is equal to p d q n - d , where d is the number of places in which x and y diﬀer. Therefore Prob( y | x ) = q n ( p/q ) d , 15

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
16 CHAPTER 2. SPHERE PACKING AND SHANNON’S THEOREM a decreasing function of d provided q > p . Therefore the Reasonable Assumption is realized by the m SC( p ) subject to q = 1 - ( m - 1) p > p or, equivalently, 1 /m > p . We interpret this restriction as the sensible design criterion that after a symbol is transmitted it should be more likely for it to be received as the correct symbol than to be received as any particular incorrect symbol. Examples. (i) Assume we are transmitting using the the binary Hamming code of Section 1.3.3 on BSC( . 01). Comparing the received word 0011111 with the two codewords 0001111 and 1011010 we see that p (0011111 | 0001111) = q 6 p 1 . 009414801 , while p (0011111 | 1011010) = q 4 p 3 . 000000961 ; therefore we prefer to decode 0011111 to 0001111. Even this event is highly unlikely, compared to p (0001111 | 0001111) = q 7 . 932065348 . (ii) If m = 5 with A = { 0 , 1 , 2 , 3 , 4 } 6 and p = . 05 < 1 / 5 = . 2, then q = 1 - 4( . 05) = . 8; and we have p (011234 | 011234) = q 6 = . 262144 and p (011222 | 011234) = q 4 p 2 = . 001024 . For x , y A n , we deﬁne d H ( x , y ) = the number of places in which x and y diﬀer. This number is the Hamming distance between x and y . The Hamming distance Hamming distance is a genuine metric on the codespace A n . It is clear that it is symmetric and that d H ( x , y ) = 0 if and only if x = y . The Hamming distance d H ( x , y ) should be thought of as the number of errors required to change x into y (or, equally well, to change y into x ).
This is the end of the preview. Sign up to access the rest of the document.

## This note was uploaded on 06/13/2011 for the course CRYPTO 101 taught by Professor Na during the Spring '11 term at Harding.

### Page1 / 15

Sphere - Chapter 2 Sphere Packing and Shannon's Theorem In...

This preview shows document pages 1 - 3. Sign up to view the full document.

View Full Document
Ask a homework question - tutors are online