to Introduction Algebraic Coding Theory
With Gap Fall 2006
Sarah Spence Adams January 11, 2008
The rst versions of this book were written in Fall 2001 and June 2002 at Cornell University, respectively supported by an NSF VIGRE Grant and a Department of Mathematics Grant. The current version was written in 2005 at Franklin W. Olin College of Engineering supported by NSF CCLI DUE-0410517 Grant. Edits were made in 2006 under the same grant. If you wish to use this book, please simply let me know and keep me informed of any typos or other problems you nd with the book.
1
Contents
1 Introduction to Coding Theory 1.1 1.2 1.3 1.4 1.5 1.6 Introductory Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Important Code Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Correcting and Detecting Errors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Sphere-Packing Bound . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Answers to Chapter 1 Reading Questions . . . . . . . . . . . . . . . . . . . . . . . . Chapter 1 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 6 9 11 13 15 16 17 17 18 22 25 28 31 32 34 36 38 39 40 43 46 49 49 50 51 53 55 58 59 59 60
2 Linear Codes 2.1 2.2 2.3 2.4 2.5 2.6 2.7 2.8 2.9 Binary Linear Codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Fields, Vector Spaces, and General Linear Codes . . . . . . . . . . . . . . . . . . . . Encoding Linear Codes: Generator Matrices . . . . . . . . . . . . . . . . . . . . . . . Introduction to Parity Check Matrices . . . . . . . . . . . . . . . . . . . . . . . . . . Parity Check Matrices and Linear Decoding . . . . . . . . . . . . . . . . . . . . . . . Parity Check Matrices, Minimum Distance, and the Singleton Bound . . . . . . . . . Hamming Codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Answers to Chapter 2 Reading Questions . . . . . . . . . . . . . . . . . . . . . . . . Chapter 2 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3 Cyclic Codes, Rings, and Ideals 3.1 3.2 3.3 3.4 3.5 3.6 Introduction to Cyclic Codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Rings and Ideals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Ideals and Cyclic Codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Generator and Parity Check Polynomials . . . . . . . . . . . . . . . . . . . . . . . . Answers to Chapter 3 Reading Questions . . . . . . . . . . . . . . . . . . . . . . . . Chapter 3 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4 Classes of Powerful Cyclic Codes 4.1 4.2 4.3 4.4 Special Cases of BCH and RS Codes . . . . . . . . . . . . . . . . . . . . . . . . . . . Minimal Polynomials . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . BCH and Reed-Solomon codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Chapter 4 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5 Special Topics 5.1 5.2 Dual Codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Group of a Code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2
6 Solutions to Selected Problems 7 Bibliography
64 67
3
A Note from the Author
Using This Book This book serves as a fairly terse introduction to the exciting eld of coding theory. You can supplement your reading of this book with any of the books in the bibliography. The more you read, the more you will learn. The author believes that reading actively is innitely more productive than reading passively. Accordingly, as you read this book, you will notice reading questions interspersed throughout the text. These questions are meant to be straight-forward checks of your reading comprehension. You should work out each of these questions as you read. You can check your answers with the brief solutions provided at the end of each chapter. Be sure to try the questions before looking at the answers! Most sections incorporate examples using Gap to help students see the power of using software in the study of error-control codes. You should work through the Gap examples to get comfortable with this software. More challenging homework problems are found at the end of each chapter. Brief solutions to certain homework problems are found at the very end of the book. This book is a living document. It is recommended that you do not print sections far in advance, as the document is subject to continual improvement. In the electronic version, hyperlinks facilitate the navigation through the book. Constructive suggestions for the improvement of this book are always welcome. Acknowledgements The rst version of this book was written in the fall of 2001 at Cornell University, supported by an NSF VIGRE grant held by the Mathematics Department. The second version was written during June 2002, supported by the Cornell Mathematics Department. The second version was revised to accommodate several suggestions compiled by Ken Brown and Steph van Willigenburg while teaching Math 336, Applicable Algebra, during the Spring 2002 semester. It also incorporated some additional problems and a handout on the group of a code developed by Ken Brown during the Spring 2002 semester. The author thanks Ken and Steph for their helpful comments and contributions. The third version of this book was written during 2005, supported by an NSF CCLI A&I Grant. This version expands upon the second version and includes several additional sections designed to make the book appropriate for students without abstract algebra backgrounds. Additional problems have been adapted from the materials developed under NSF DUE-9980900. This version also incorporates extra examples, proofs, and other contributions compiled by the Olin students in the authors Spring 2005 MTH3140, Error Control Coding course. The Gap and Guava examples and exercises were designed by Michael Wu, a student assistant also supported by the NSF CCLI A&I grant. The author thanks Michael for his signicant contributions to the Gap and other aspects of the book. The fourth and current version of this book represents improvements such as solutions to all reading questions, solutions to selected homework problems, general clarications, and new examples. The author thanks Michael Foss for his detailed suggestions for improvement. The books in the following bibliography were of great help in writing this book. We borrowed some examples, problems, and proofs from these books, most often [2] and [8].
Sarah Spence Adams Fall 2006
4
Digital Source
Information Theory
Digital Link
Source Encoder
Source Decoder
Encryption
Cryptography Coding Theory
Channel
Decryption
Channel Encode
Channel Decode
Modulator
Demodulator
Noise
Figure 2: A Flowchart Representing the Various Stages of Data Processing
1
Introduction to Coding Theory
Imagine that you are using an infrared link to beam an mp3 le from your laptop to your PalmPilot. It is possible to model the transmitted data as a string of 0s and 1s. When a 0 is sent, your PalmPilot usually receives a 0. Occasionally, noise on the channel, perhaps in the form of atmospheric disturbances or hardware malfunctions, causes the 0 to be received as a 1. We would like to develop ways to combat these errors that can occur during data transmission. Error-control codes are used to detect and correct errors that occur when data are transmitted across some noisy channel or stored on some medium. When photographs are transmitted to Earth from deep space, error-control codes are used to guard against the noise caused by lightning and other atmospheric interruptions. Compact discs (CDs) use error-control codes so that a CD player can read data from a CD even if it has been corrupted by noise in the form of imperfections on the CD. Correcting errors is even more important when transmitting data that have been encrypted for security. In a secure cryptographic system, changing one bit in the ciphertext propagates many changes in the decrypted plaintext. Therefore, it is of utmost importance to detect and correct errors that occur when transmitting enciphered data. The study of error-control codes is called coding theory. This area of discrete applied mathematics includes the study and discovery of various coding schemes that are used to increase the number of errors that can be corrected during data transmission. Coding theory emerged following the publication of Claude Shannons seminal 1948 paper, A mathematical theory of communication, [6]. It is one of the few elds that has a dened beginning; you can nd a list of historical accomplishments over the last 50 years at http://www.ittc.ku.edu/ paden/reference/guides/ECC/history.html. Error control coding is only part of the processing done to messages that are to be transmitted across a channel or stored on some medium. Figure 2 shows a ow chart that illustrates how error control coding ts in with the other stages of data processing. The process begins with an information source, such as a data terminal or the human voice. The source encoder transforms the source output into a sequence of symbols which we call a message m; if the information source is continuous, the source encoding involves analog-to-digital conversion. Throughout this book, we usually assume that the symbols used are from the binary set {0, 1} and call these symbols bits. If security is desired, the message would next be encrypted using a 5
cipher, the subject of the eld of cryptography. The next step is the error-control coding, also called channel coding, which involves introducing controlled redundancy into the message m. The output is a string of discrete symbols (usually binary in this book) which we call a codeword c. Next, the modulator transforms each discrete symbol in the codeword into a wavelength to be transmitted across the channel. The transmission is subject to noise of various types, and then the processes are reversed. Historically, decreasing the error rates in data transmissions was achieved by increasing the power of the transmission. However, this is an inecient and costly use of power. Moreover, increasing the power only works well on certain channels such as binary symmetric channels; other channels, such as fading channels, present additional challenges. Hence, we will be studying more power- and costecient ways of reducing the error rates. In particular, we will be studying linear, cyclic, BCH, and Reed-Solomon codes. Through our study of error-control codes, we will model our data as strings of discrete symbols, often binary symbols {0, 1}. When working with binary symbols, addition is done modulo 2. For example, 1 + 1 0 (mod 2). We will study channels that are aected by additive white Gaussian noise, which we can model as a string of discrete symbols that get added symbol-wise to the codeword. For example, if we wish to send the codeword c = 11111, noise may corrupt the codeword so that the r = 01101 is received. In this case, we would say that the error vector is e = 10010, since the codeword was corrupted in the rst and fourth positions. Notice that c + e = r, where the addition is done component-wise and modulo 2. The steps of encoding and decoding that concern us are as follows: m Encode c N oise c + e = r Decode m where m is the message, c is the codeword, e is the error vector due to noise, r is the received word or vector, and m is the decoded word or vector. The hope is that m = m.
1.1
Introductory Examples
To get the ball rolling, we begin with some examples of error-control codes. Our rst example is an error-detecting, as opposed to error-correcting, code. A code that can only detect up to t errors within a codeword can be used to determine whether t or fewer errors occurred during the transmission of the codeword, but it cannot tell us exactly what error(s) occurred nor can it x the error(s). A code that can correct up to t errors can be used to actually correct up to t errors that occur during the transmission of a codeword. This means that the code detects that errors occurred, gures out what errors occurred in which positions, and then corrects the errors. Example 1.1.1. (The ISBN Code) The International Standard Book Number (ISBN) Code is used throughout the world by publishers to identify properties of each book. The rst nine digits (bounded between 0 and 9, inclusive) of each ISBN represent information about the book including its language, publisher, and title. In order to guard against errors, the nine-digit message is encoded as a ten-digit codeword. The appended tenth digit is a check digit chosen so that the whole ten-digit string x1 x2 x10 satises
10
ixi 0 (mod 11).
i=1
(1)
If x10 should be equal to 10, an X is used. The ISBN code can detect any single error and any double-error created by the transposition of two digits. A great course project would be to gure out why this code has this error-detecting capability and compare this with other check-digit schemes used on airplane tickets, in bank numbers on checks, in credit card numbers, in the Universal Product Code (UPC) found on groceries, etc. 6
Question 1. Are either of the following strings valid ISBNs: 0-13165332-6, 0-1392-4101-4? In some states, drivers license numbers include check digits in order to detect errors or fraud. In fact, many states generate license numbers through the use of complicated formulas that involve both check digits and numbers based on information such as the drivers name, date of birth, and sex. Most states keep their formulas condential, however Professor Joseph Gallian at the University of Minnesota-Duluth gured out how several states generate their license numbers. Some of his results and techniques are summarized in www.maa.org/mathland/mathtrek 10 19 98.html. Another website that has useful discussions about check digits is http://www.cs.queensu.ca/home/bradbury/checkdigit/index.html. Example 1.1.2. (The Repetition Code) This example is a simple error-correcting code. Suppose we would like to send a 1 to signify yes, and a 0 to signify no. If we simply send one bit, then there is a chance that noise will corrupt that bit and an unintended message will be received. A simple alternative is to use a repetition code. Instead of sending a single bit, we send 11111 to represent 1 (or yes), and 00000 to represent 0 (or no). Since the codewords consist of 0s and 1s, this is called a binary code. We say that the codes alphabet is the set {0, 1}, with all arithmetic done modulo 2. Alternatively, we can say that this code alphabet is the nite eld with two elements, GF (2). Finite elds will be formally dened in Section 2.2. Under certain reasonable assumptions about the channel in use, the receiver decodes a received 5-tuple using a majority vote: The received 5-tuple is decoded as the bit that occurs most frequently. This way, if zero, one, or two errors occur, we will still decode the received 5-tuple as the intended message. In other words, the receiver decodes a received 5-tuple as the closest codeword. This is an example of nearest neighbor decoding, which is discussed in more detail in Section 1.3. Question 2. How many errors can a binary repetition code of length 10 correct? How many errors can it detect? Notice that transmitting a message that has been encoded using this repetition code takes ve times longer than transmitting the uncoded message. However, we have increased the probability of decoding the received string as the correct message. A complete discussion on this probability is found in Section 1.3. Guava has a built-in function for creating repetition codes. For example, the command gap> C := RepetitionCode(5, GF(2));; creates a binary repetition code of length 5 and stores it as C. The second argument GF(2) indicates that each component of the codewords is in {0, 1} and the bitwise operations are performed modulo 2. For now, we will simply look at the generated codewords: The command AsSortedList will list all codewords of a code. With C dened by our rst command, we can now type: gap> cw := AsSortedList(C); which will output [[0000000], [1111111]] Question 3. Use Gap (Guava) to generate a repetition code of length 15 and list its codewords.
7
Example 1.1.3. (The 2-D Parity Check Code) In the ISBN code, we saw that a check digit could be appended to a message in order to detect common errors. It is possible to extend this idea to correct single errors by rst arranging a message into a matrix, or two-dimensional array. For example, suppose we want to encode the 20-bit message 10011011001100101011. First, we arrange this message into a 4 5 matrix 1 0 M = 1 0 0 1 1 1 0 1 0 0 1 0 0 1 1 0 . 1 1
Then, we look at the parity of each row, meaning that we check if there is an even or odd number of 1s in the row. In other words, we check if the sum of the entries in a row is even or odd. Then, we calculate and append a parity check bit to the end of each row, where a parity check bit is a bit chosen to ensure that the overall sum of the extended row is even. After appending a parity check bit to each row, we have four rows of length 6. We then repeat the process by calculating and appending parity check bits along along each of the original columns, which gives the following: C= 1 0 1 0 0 0 1 1 1 1 0 1 0 0 1 1 0 0 1 0 1 0 1 1 1 1 0 1 1 .
The entry in the lower right hand corner marked by is then calculated as the parity check bit of the new row (in other words, the entry is calculated as the parity of the parity bits that were calculated along the original columns). We therefore use the matrix C = 1 0 1 0 0 0 1 1 1 1 0 1 0 0 1 1 0 0 1 0 1 0 1 1 1 1 0 1 1 1
to send a stream of 30 bits across our noisy channel. To see how this extended matrix can correct a single error, suppose that the receiver arranges the received 30 bits into a 5 6 matrix and obtains R= 1 0 1 0 0 0 1 1 1 1 0 1 0 0 1 1 0 1 1 0 1 0 1 1 1 1 0 1 1 1 .
The receiver then checks the parity of each row and column: If all parities check to be even, then we assume there are no errors. (Is this a fair assumption?) However, in this example, we see that the parities of the third row and fourth column are odd. Therefore, we conclude that there is an error in the (3, 4) position, correct the error, and proceed to read the message. Traditionally, the alphabets used in coding theory are nite elds with q elements, GF (q), formally dened in Section 2.2. We say that a code is q-ary if its codewords are dened over the q-ary alphabet 8
GF (q). The most commonly used alphabets are binary extension elds, GF (2m ). This book focuses on codes with the familiar alphabet GF (2), which are known as binary codes. The ISBN code is 11-ary since its symbols come from the set {0, 1, . . . , 10}, while our repetition code and 2-D Parity Check Code are both binary.
1.2
Important Code Parameters
When developing codes, there must be a way to decide which codes are good codes. There are three main parameters used to describe and evaluate codes. The rst parameter is the code length, n. In the repetition code of Example 1.1.2, the code length is 5, since the codewords 00000 and 11111 each contain 5 bits. In this book, we restrict our discussion to block codes, which are codes whose codewords are all of the same length. Since error-control codes build redundancy into the messages, the code length, n, is always greater than the original message length, k. The next parameter that we consider is the total number of codewords, M . In Example 1.1.2, the total number of codewords is 2. The third parameter measures the distance between pairs of codewords in a code. In order to explain clearly the notion of distance between codewords, we need a few denitions. Denition 1.2.1. The Hamming weight w(c) of a codeword c is the number of nonzero components in the codeword. Example 1.2.2. w(00000) = 0, w(11111) = 5, w(1022001) = 4, w(1011001) = 4. The Guava function WeightCodeword(C) can be used to determine the weight of a given codeword: gap> WeightCodeword(Codeword("1011001")); 4 Question 4. Use Gap to conrm the weights given in Example 1.2.2. Denition 1.2.3. The Hamming distance between two codewords d(x, y) is the number of places in which the codewords x and y dier. In other words, d(x, y) is the Hamming weight of the vector x y, representing the component-wise dierence of the vectors x and y. Example 1.2.4. d(0001, 1110) = 4, since the two codewords dier in all four positions. d(1202, 1211) = 2, since the two codewords dier in the last two positions. The Guava function DistanceCodeword will return the Hamming distance of two codewords. For example: gap> a := Codeword("01111", GF(2));; gap> b := Codeword("11000",GF(2));; gap> DistanceCodeword(a,b); 4 Question 5. Calculate d(00111, 11001). How about d(00122, 12001)? Check your answers using Gap. Sometimes non-binary codes are evaluated using alternate distance metrics, but in this book, distance will always refer to Hamming distance. To read about other distance functions used to evaluate codes, see [7]. Denition 1.2.5. The minimum (Hamming) distance of a code C is the minimum distance between any two codewords in the code: d(C) = min{d(x, y) | x = y, x, y C}. 9
Example 1.2.6. The binary repetition code of length 5 has minimum distance 5 since the two codewords dier in all 5 positions. Question 6. What is the minimum distance between any two ISBN codewords? Denition 1.2.7. The Hamming distance d is a metric on the space of all q-ary n-tuples which means that d satises the following properties for any q-ary n-tuples x, y, z: 1. d(x, y) 0, with equality if and only if x = y. 2. d(x, y) = d(y, x) 3. d(x, y) d(x, z) + d(z, y) The third property above is called the triangle inequality, which should look familiar from Euclidean geometry. The proofs of these properties are left as problems at the end of this chapter. The notation (n, M, d) is used to represent a code with code length n, a total of M codewords, and minimum distance d. One of the major goals of coding theory is to develop codes that strike a balance between having small n (for fast transmission of messages), large M (to enable transmission of a wide variety of messages), and large d (to detect many errors). Example 1.2.8. Let C = {0000, 1100, 0011, 1111}. Then C is a (4, 4, 2) binary code. It is possible to construct a code in Gap by listing all of its codewords. To do this, we can use the function ElementsCode. For example: gap> C := ElementsCode(["0000", "1100", "0011", "1111"], GF(2));; constructs the code given in Example 1.2.8. If we need help determining the key parameters for a code, we can use built-in functions such as: gap> n := WordLength(C); 4 gap> M := Size(C); 4 gap> d := MinimumDistance(C); 2 The nal important code parameter is a measure of eciency: Denition 1.2.9. The rate of a code is the ratio Example 1.2.10. The rate of the ISBN code is message.
9 10 k n
of message symbols to coded symbols.
since each 9-digit message is encoded as a 10-digit
1 5
Example 1.2.11. The rate of the binary repetition code of length 5 is is encoded as a 5-bit message. Example 1.2.12. The rate of the 2-D Parity Check code encoded as a 30-bit message.
20 30
since each 1-bit message
or
2 3
since each 20-bit message is
High rate codes are desirable since a higher rate code implies a more ecient use of redundancy than a lower rate code. In other words, a higher rate code will transmit k message symbols more quickly than a lower rate code can transmit the same k message symbols. However, when choosing a code for a particular application, we must also consider the error-correcting capabilities of the code. A rate 1 code has the optimal rate, but has no redundancy and therefore oers no error control. 10
1.3
Correcting and Detecting Errors
Talking about the number of errors in a received codeword is equivalent to talking about the distance between the received word and the transmitted word. Suppose a codeword c = c0 c1 . . . cn1 is sent through a channel and the received vector is r = r0 r1 . . . rn1 . The error vector is dened as e = r c = e0 e1 . . . en1 . The job of the decoder is to decide which codeword was most likely transmitted, or equivalently, decide which error vector most likely occurred. Many codes use a nearest neighbor decoding scheme which chooses the codeword that minimizes the distance between the received vector and possible transmitted vectors. For example, the majority vote scheme described for our binary repetition code is an example of a nearest neighbor decoding scheme. A nearest neighbor decoding scheme for a q-ary code maximizes the decoders likelihood of correcting errors provided the following assumptions are made about the channel: 1. Each symbol transmitted has the same probability p (< 1/2) of being received in error (and 1 p of being received correctly) 2. If a symbol is received in error, that each of the q 1 possible errors is equally likely. Such a channel is called a q-ary symmetric channel, and we assume throughout this book that the channels involved are symmetric. This implies that error vectors of lower weight will occur with higher probability than error vectors of higher weight. When analyzing codes, we often need to talk about probabilities of certain events occurring. For example, we need to compare the probabilities of the occurrences of lower weight errors versus higher weight errors, and we need to ascertain the probability that a codeword will be received in error. Informally, the probability that an event occurs is a measure of the likelihood of the event occurring. Probabilities are numbers between 0 and 1, inclusive, that reect the chances of an event occurring. A probability near 1 reects that the event is very likely to occur, while a probability near 0 reects that the event is very unlikely to occur. When you ip a fair coin, there is a probability of 1/2 that you will get tails and a probability of 1/2 that you will get heads. But what about the probability of getting three heads in three ips of a fair coin? Since each of these events is independent, meaning that each of the three ips does not depend on the other ips, we can use the multiplicative property of probabilities: The probability of getting three heads in the 1 three coin tosses is 1 1 1 = 8 . 222 When dealing with a q-ary codeword of length n, we will often need to ask the following question: What is the probability that e errors occur within a codeword of length n? Lets start with some small examples that work up to this answer. Example 1.3.1. Given the assumptions of a q-ary symmetric channel with symbol error probability p, consider the probability that no errors will occur when transmitting a q-ary codeword of length n. Since each symbol has probability (1 p) of being received correctly, the desired probability is (1 p)n . Example 1.3.2. Given the assumptions of a q-ary symmetric channel, consider the probability that exactly one of the n symbols in a q-ary codeword of length n is received in error. The probability that the rst symbol is received in error and the remaining n 1 symbols are received correctly is p(1 p)n1 . Similarly, the probability that the second is received in error and the remaining n 1 symbols are received correctly is p(1 p)n1 . We can deduce that the overall probability of exactly one of the n symbols in a q-ary codeword of length n being received in error is np(1 p)n1 , since there are n ways that this can happen (namely, one way for each place that the error can occur). The next logical example should be to ask about the probability that exactly two of the n symbols in a q-ary codeword of length n are received in error. However, we rst need a little more experience with counting. 11
n The number m , read n choose m, counts the number of ways that we can choose m objects from n n! a pool of n objects, and m = (nm)!m! . Note that n! is read n factorial, and n! = n(n1) 21. n The numbers of the form m are called binomial coecients because of the binomial theorem which states that for any positive integer n,
(1 + x)n = 1 +
n2 nn n x+ x + + x. 1 2 n
Most calculators have a button that will compute binomial coecients. Lets do some examples to get comfortable with this idea. Example 1.3.3. How many ways can we choose the chairperson of a committee with four members 4! Alice, Bob, Cathy, and David? The answer should be 4 = 3!1! = 4. This makes sense since there 1 are clearly 4 ways to choose the chairperson from the group of 4. Example 1.3.4. How many ways can we choose two places from a codeword of length n? The answer is n . If n = 3, this simplies to 3 = 3. This makes sense since there are 3 ways to not 2 2 choose 1 symbol, or equivalently, 3 ways to choose 2 symbols. Question 7. By listing all possibilities and by using binomial coecients, determine how many ways we can choose two places from a codeword of length 4. We will now use our newfound knowledge on binomial coecients to resume our earlier discussion. Example 1.3.5. Given the assumptions of a q-ary symmetric channel, consider the probability that exactly two of the n symbols in a q-ary codeword of length n are received in error. This means that two of the symbols are received in error and n 2 of the symbols are received correctly. The probability that the rst two symbols are received in error is p2 (1 p)n2 . However, there are n 2 ways to choose where the 2 errors occur. Therefore, the probability that exactly two of the n symbols in a q-ary codeword of length n are received in error is n p2 (1 p)n2 . 2 Question 8. Recall that (assuming a binary symmetric channel), the binary repetition code of length 5 can correct up to two errors. Explain why the probability of decoding a received word correctly is (1 p)5 + 5p(1 p)4 + 10p2 (1 p)3 . (2) In general, a code that has minimum distance d can be used to either detect up to d 1 errors or correct up to (d 1)/2 errors. This is a consequence of the following theorem: Theorem 1.3.6. 1. A code C can detect up to s errors in any codeword if d(C) s + 1.
2. A code C can correct up to t errors in any codeword if d(C) 2t + 1. Proof. 1. Suppose d(C) s + 1. Suppose a codeword c is transmitted and that s or fewer errors occur during the transmission. Then, the received word cannot be a dierent codeword, since all codewords dier from c in at least s + 1 places. Hence, the errors are detected. 2. Suppose d(C) 2t + 1. Suppose a codeword x is transmitted and that the received word, r, contains t or fewer errors. Then d(x, r) t. Let x be any codeword other than x. Then d(x , r) t + 1, since otherwise d(x , r) t which implies that d(x, x ) d(x, r) + d(x , r) 2t (by the triangle inequality), which is impossible since d(C) 2t + 1. So x is the nearest codeword to r, and r is decoded correctly.
The remainder of this section will involve using error-correcting capabilities and code rate to compare two dierent codes proposed for the transmission of photographs from deep-space. 12
Example 1.3.7. (Transmission of photographs from deep-space) This example describes a method for transmitting photographs. It is the method that was used in the Mariner 1969 Mission which sent photographs of Mars back to Earth. In order to transmit a picture, the picture is rst divided into very small squares, known as pixels. Each pixel is assigned a number representing its degree of blackness, say in a scale of 0 to 63. These numbers are expressed in the binary system, where 000000 represents white and 111111 represents black. Each binary 6-tuple is encoded using a (32, 64, 16) code, called a Reed-Muller code. Reed-Muller codes are often used in practice because they are very easy to decode. Question 9. Although the details of the code are not given, use the codes parameters to determine how many errors the (32, 64, 16) Reed-Muller code correct. When using the Reed-Muller code, the probability that a length 32 codeword is decoded incorrectly is
32 i=8
32 i p (1 p)32i . i
(3)
Question 10. Explain how the expression in (3) was obtained. Suppose that instead of the Reed-Muller code, we use a repetition code of length 5 to transmit each bit of each pixels color assignment. Then each 6-bit message string (representing a color assignment) is encoded as a codeword of length 30, since each bit in the message string is repeated ve times. If we use a repetition code of length 5 to encode the 6-bit message strings, then the probability that one of the resulting codewords of length 30 is received correctly is equal to the expression in (2) raised to the 6th power: [(1 p)5 + 5p(1 p)4 + 10p2 (1 p)3 ]6 Question 11. Explain how the expression in (4) was obtained. Question 12. Suppose that p = .01 and compare the probability of decoding incorrectly when using the repetition code with the probability of decoding incorrectly when using the Reed-Muller code. Since each codeword of length 30 in the repetition code represents a 6-bit message, the rate of the code is 1/5. Since each codeword of length 32 in the Reed-Muller code in Example 1.3.7 represents a 6-bit message, the rate of the code is 6/32. Higher rate codes are faster and use less power, however sometimes lower rate codes are preferred because of superior error-correcting capability or other practical considerations. Question 13. Using code rate and probability of decoding incorrectly to compare the repetition code and the Reed-Muller code, explain which code you think is an overall better code. (4)
1.4
Sphere-Packing Bound
We know from Theorem 1.3.6 that if C has minimum distance d 2t + 1, then C can correct at least t errors. We also say that C can correct all error vectors of weight t. We now develop a geometric interpretation of this result. Since each codeword in C is at least distance 2t + 1 from any other codeword, we can picture each codeword c to be surrounded by a sphere of radius t such that the spheres are disjoint (non-overlapping). Any received vector that lies within the sphere centered at c will be decoded as c. This pictorially explains why received vectors that contain t or fewer errors are decoded correctly: They still lie within the correct sphere. We can use this picture to bound the total possible number of codewords in a code, as seen in the following theorem: 13
Theorem 1.4.1. (Sphere-packing bound ) A t-error-correcting q-ary code of length n must satisfy
t
M
i=0
n (q 1)i q n i
(5)
where M is the total number of codewords. In order to prove Theorem 1.4.1, we need the following lemma. Lemma 1.4.2. A sphere of radius r, 0 r n, in the space of all q-ary n-tuples contains exactly
r i=0
n (q 1)i = i
n n n n + (q 1) + (q 1)2 + + (q 1)r 0 1 2 r
vectors. Proof of Lemma 1.4.2. Let u be a xed vector in the space of all q-ary n-tuples. Consider how many vectors v have distance exactly m from u, where m n. The m positions in which v is to dier n from u can be chosen in m ways, and then in each of these m positions the entry of v can be chosen in q 1 ways to dier from the corresponding entry of u. Hence, the number of vectors at n distance exactly m from u is m (q 1)m . A ball of radius r centered at u contains vectors whose distance from u ranges from 0 to r. So, the total number of vectors in a ball of radius r, centered at u, must be n + n (q 1) + n (q 1)2 + + n (q 1)r . 0 1 2 r Proof of Theorem 1.4.1. Suppose C is a t-error-correcting q-ary code of length n. Then, in order that C can correct t errors, any two spheres of radius t centered on distinct codewords can have no vectors in common. Hence, the total number of vectors in the M spheres of radius t centered on the t M codewords of C is given by M i=0 n (q 1)i , by Lemma 1.4.2. This number of vectors must i be less than or equal to the total number of vectors in the space of all q-ary n-tuples, which is q n . This proves the sphere-packing bound. Denition 1.4.3. A code is perfect if it satises the sphere-packing bound of Theorem 1.4.1 with equality. Question 14. Show that the binary repetition code of length 5 is perfect. It is possible to use Gap to test if a code is perfect; the function IsPerfectCode return true if a code is perfect and false otherwise. For example, to answer the above question using Gap, we could type gap> C := RepetitionCode(5, GF(2));; gap> IsPerfectCode(C); true When decoding using a perfect code, every possible q-ary received word of length n is at distance less than or equal to t from a unique codeword. This implies that the nearest neighbor algorithm yields an answer for every received word r, and it is the correct answer when the number of errors is less than or equal to t. In Chapter 2, we will introduce a family of perfect codes called Hamming codes.
14
1.5
Answers to Chapter 1 Reading Questions
Answer 1. The rst string is valid since the required summation comes out to 198, which is divisible by 11. The second string is not valid since the summation results in 137, which is not divisible by 11. Answer 2. A binary repetition code of length 10 can correct up to 4 errors since it is a majority vote; at 5 errors, there is no way to determine whether 0 or 1 was intended; with 6 or more errors, the received codeword would be corrected to the wrong message. However, it can detect 9 errors. Answer 3. gap> C := RepetitionCode(15,GF(2));; gap> cw:= AsSortedList(C); This will output [[000000000000000],[111111111111111]] Answer 4. gap> WeightCodeword(Codeword("00000")); 0 gap> WeightCodeword(Codeword("11111")); 5 gap> WeightCodeword(Codeword("1022001")); 4 gap> WeightCodeword(Codeword("1011001")); 4 Answer 5. d(00111, 11001) is 4. d(00122, 12001) is 5. Answer 6. For any ISBN codeword, changing a single digit will cause the codeword to no longer be valid since the summation will no longer be divisible by 11. You can, however, change two digits and still follow the ISBN rules, as in 0-1316-5332-6 and 1-1316-5332-7. Therefore, the minimum distance between any two ISBN codewords is 2. Answer 7. By listing all possibilities: 1. rst position and second position 2. rst position and third position 3. rst position and fourth position 4. second position and third position 5. second position and fourth position 6. third position and fourth position This is a total of 6 ways to choose two place from a codeword of length 4. Similarly, using binomial 4! coecients, we conrm there are 4 = 2!2! = 6 ways to pick two places. 2 Answer 8. This formula gives the probability of getting 0, 1, or 2 errors in a binary repetition code of length 5. (1 p)5 is the probability of getting no errors. 5p(1 p)4 is the probability of getting one error. 10p2 (1 p)3 is the probability of getting two errors. Answer 9. The minimum distance of the code is 16, so this code can correct (16 1)/2 , or, 7 errors. Answer 10. This expression gives the probability of getting 8 or more errors (since the code can correct 7). 32 pi (1 p)32i is the probability that there are i errors, where i 8, 9, ..., 32. i Answer 11. (1p)5 +5p(1p)4 +10p2 (1p)3 is the probability that a single codeword of length 5 is decoded to the correct message bit. This expression is raised to the 6th power to nd the probability that all 6 message bits are decoded correctly. 15
Answer 12. The probability of decoding incorrectly using the repetition code is 5.9 105 . The probability of decoding incorrectly using the Reed-Muller code is about 8.5 1010 . Answer 13. The rate of the Reed-Muller code is 6/32, and the rate of the binary repetition code is 1/5. Answer 14. The binary repetition code of length 5 satises the sphere-packing bound with equality so it is perfect.You can check this using M = 2, n = 5, and q = 2.
1.6
Chapter 1 Problems
Problem 1.1. Complete the ISBN that starts as 0-7803-1025-. Problem 1.2. Let C = {00, 01, 10, 11}. Why cant C correct any errors? Problem 1.3. Let C = {(000000), (101110), (001010), (110111), (100100), (011001),(111101), (010011)} be a binary code of length 6. 1. What is the minimum distance of C? 2. How many errors will C detect? 3. How many errors will C correct? Problem 1.4. Let A = Z5 and n = 3. Let C = {1 (100) + 2 (011) | 1 , 2 Z5 } where all arithmetic is done componentwise and modulo ve. Write down the elements of C and determine the minimum distance of C. Problem 1.5. Find a way to encode the letters a, b, c, d, e, f, g, h using blocks of zeros and ones of length six so that if two or fewer errors occur in a block, Bob will be able to detect that errors have occurred (but not necessarily be able to correct them). Problem 1.6. Prove that there is no way to encode the letters a, b, c, d, e, f, g, h using blocks of zeros and ones of length ve so that Bob will be able to correct a single error. Problem 1.7. Let C be a code that contains 0, the codeword consisting of a string of all zeros. Suppose that C contains a string u as a codeword. Now suppose that the string u also occurs as an error. Prove that C will not always detect when the error u occurs. Problem 1.8. Prove that the Hamming distance satises the following conditions for any x, y, z GF (2)n : 1. d(x, y) 0, with equality if and only if x = y 2. d(x, y) = d(y, x) 3. d(x, y) d(x, z) + d(z, y) Problem 1.9. Is it possible to correct more than one error with the 2-D Parity Check Code shown in Example 1.1.3? How many errors can this code detect? Problem 1.10. Prove that any ternary (11, 36 , 5) code is perfect. Problem 1.11. Show that a q-ary (q + 1, M, 3) code satises M q q1 . Problem 1.12. Prove that all binary repetition codes of odd length are perfect codes. Problem 1.13. Prove that no binary repetition code of even length can be perfect. Problem 1.14. Prove that for n = 2r 1,
n 0 n 1
+
= 2r .
Problem 1.15. Prove that if C is a binary linear [n, k] code then the sum of the weights of all the elements of C is less than or equal to n2k1 . 16
2
Linear Codes
In this chapter, we study linear error-control codes, which are special codes with rich mathematical structure. Linear codes are widely used in practice for a number of reasons. One reason is that they are easy to construct. Another reason is that encoding linear codes is very quick and easy. Decoding is also often facilitated by the linearity of a code. The theory of general linear codes requires the use of abstract algebra and linear algebra, but we will begin with some simpler binary linear codes
2.1
Binary Linear Codes
Denition 2.1.1. A binary linear code C of length n is a set of binary n-tuples such that the componentwise modulo 2 sum of any two codewords is contained in C. Example 2.1.2. The binary repetition code of length 5 is a binary linear code. You can quickly conrm that it satises the requirement that the sum of any two codewords is another codeword: 00000 + 00000 = 00000 C, 00000 + 11111 = 11111 C, and 11111 + 11111 = 00000 C. Example 2.1.3. The set {0000, 0101, 1010, 1111} is a binary linear code, since the sum of any two codewords lies in this set. Notice that this is a (4, 4, 2) code because the codewords have length 4, there are 4 codewords, and the minimum distance between codewords is 2. Example 2.1.4. The set {0000000, 0001111, 0010110, 0011001, 0100101, 0101010, 0110011, 0111100, 1000011, 1001100, 1010101, 1011010, 1100110, 1101001, 1110000, 1111111} is a binary linear code known as a Hamming code. Notice that this is a (7, 16, 3) code because the codewords have length 7, there are 16 codewords, and the minimum distance is 3. We will study Hamming codes in more detail in Section 2.7. Question 1. How many errors can the above Hamming code correct? Question 2. Is {000, 100, 001} a binary linear code? How about {100, 001, 101}? You can use Gap to check if a given code is linear with the IsLinearCode function. For example, to automate the previous question, we can type gap> C := ElementsCode(["000","100", "001"], GF(2)); gap> IsLinearCode(C); and it will output true if C is linear and false if C is non-linear. Question 3. Redo Question 2 using Gap. Question 4. Show that the codeword 0 consisting of all zeros is always contained in a binary linear code. Theorem 2.1.5, below, implies one of the main advantages of linear codes: It is easy to nd their minimum distance. This is essential for evaluating the error-correcting capabilities of the code. Theorem 2.1.5. The minimum distance, d(C), of a linear code C is equal to w (C), the weight of the lowest-weight nonzero codeword. Proof. There exist codewords x and y in C such that d(C) = d(x, y). By the denition of Hamming distance, we can rewrite this as d(C) = w(xy). Note that xy is a codeword in C by the linearity of C. Therefore, since w (C) is the weight of the lowest weight codeword, we have w (C) w(xy) = d(C). 17
On the other hand, there exists some codeword c C such that w (C) = w(c). By the denition of the weight of a codeword, we can write w(c) = d(c, 0). Since c and 0 are both codewords, the distance between them must be greater than or equal to the minimum distance of the code: d(c, 0) d(C). Stringing together these (in)equalities, shows that w (C) d(C). Since we have now shown that both d(C) w (C) and d(C) w (C), we conclude that d(C) = w (C). Theorem 2.1.5 greatly facilitates nding the minimum distance of a linear code. Instead of looking at the distances between all possible pairs of codewords, we need only look at the weight of each codeword. Notice that the proof does not restrict the linear codes to be binary. Question 5. How many pairs of codewords would we need to consider if trying to nd the minimum distance of a nonlinear code with M codewords? Sometimes we can start with a known binary linear code and append an overall parity check digit to increase the minimum distance of a code. Suppose C is a linear (n, M, d) code. Then we can construct a code C , called the extended code of C, by appending a parity check bit to each codeword x C to obtain a codeword x C as follows. For each x = x0 x1 xn1 C, let x = x0 x1 xn1 0 if the Hamming weight of x is even, and let x = x0 x1 xn1 1 if the Hamming weight of x is odd. This ensures that every codeword in the extended code has even Hamming weight. Theorem 2.1.6. Let C be a binary linear code with minimum distance d = d(C). If d is odd, then the minimum distance of the extended code C is d + 1, and if d is even, then the minimum distance of C is d. Proof. In the problems at the end of this chapter, you will prove that the extended code C of a binary linear code is also a linear code. Hence by Theorem 2.1.5, d(C ) is equal to the smallest weight of nonzero codewords of C . Compare the weights of codewords x C with the weights of the corresponding codewords x C: w(x ) = w(x) if w(x) is even, and w(x ) = w(x) + 1 if w(x) is odd. By Theorem 2.1.5, we can conclude that d(C ) = d if d is even and d(C ) = d + 1 if d is odd.
2.2
Fields, Vector Spaces, and General Linear Codes
In order to develop the theory for general linear codes, we need some abstract and linear algebra. This section may be your rst introduction to abstract mathematics. Learning the denitions and getting comfortable with the examples will be very important. We begin with a discussion of the mathematical constructs known as elds. Although the following denition may seem daunting, it will be helpful to keep a couple of simple examples in mind. The binary alphabet, along with modulo 2 addition and multiplication, is the smallest example of a eld. A second familiar example is R, the real numbers along with ordinary addition and multiplication. Denition 2.2.1. Let F be a nonempty set that is closed under two binary operations denoted + and *. That is, the binary operations input any elements a and b in F and output other elements c = a + b and d = a b in F . Then (F, +, ) is a eld if: 1. There exists an additive identity labeled 0 in F such that a + 0 = 0 + a = a for any element a F. 2. The addition is commutative: a + b = b + a, for all a, b F . 3. The addition is associative: (a + b) + c = a + (b + c), for all a, b, c F .
18
4. There exists an additive inverse for each element: For each a F , there exists an element denoted a in F such that a + (a) = 0. 5. There exists a multiplicative identity denoted 1 in F such that a 1 = 1 a = a for any element a F. 6. The multiplication is commutative: a b = b a, for all a, b F . 7. The multiplication is associative: a (b c) = (a b) c, for all a, b, c F . 8. There exists a multiplicative inverse for each nonzero element: For each a = 0 in F , there exists an element denoted a1 in F such that a a1 = a1 a = 1. 9. The operations distribute: a (b + c) = a b + a c and (b + c) a = b a + c a, for all a, b, c F .
It is assumed that 0 = 1, so the smallest possible eld has two elements (namely, the binary alphabet that we have been using all along). We write this eld as GF (2), which means that it is the Galois eld (a fancy name for nite elds) with 2 elements. Take a moment to convince yourself that the binary alphabet GF (2) does indeed satisfy all of the eld axioms. Aspiring abstract algebra students will be interested to see that a eld is a set whose elements form a commutative group under addition (Criteria 14), whose nonzero elements form a commutative group under multiplication (Criteria 5-8), and whose operations distribute (Criterion 9). We will write ab as an abbreviation for a b. In addition to the familiar examples of the real numbers and the binary numbers, it is helpful to have a non-example in mind. (Z, +, ), the integers under ordinary addition and multiplication, is not a eld because most integers do not have multiplicative inverses: For example, there is no integer x such that 2x = 1. The integers under ordinary addition do form an additive group, but the nonzero integers under ordinary multiplication do not form a multiplicative group. Example 2.2.2. Lets determine if the set of integers Z6 = {0, 1, 2, . . . , 5} is a eld under addition and multiplication modulo 6. If the set is a eld, there must be a multiplicative inverse for each element of the set. For example, there must exist some k such that 4k 1 (mod 6). However, we know since k is always an integer, 4k is always even. Thus, when working modulo 6, 4k never be equivalent to 1. Thus, Z6 is not a eld under addition and multiplication modulo 6. Another way to see that this set is not a eld is the existence of zero divisors; these are non-zero numbers that multiply to zero. For example: 4 3 0 (mod 6). This would result in a contradiction, since if we multiply each side of this equation on the left by 41 , we would obtain the nonsensical 3 0 (mod 6). In contrast to the above example, Zp under addition and multiplication modulo p, where p is any prime, is a eld. The proof is left as a problem. In fact, it is the eld properties of Z11 that give the ISBN code from Example 1.1.1 its error-detecting capabilities. Question 6. Find additive inverses and multiplicative inverses for the elements in Z7 . We are now ready to use elds to introduce another algebraic construct: Denition 2.2.3. Let F be a eld. A set V of elements called vectors is a vector space if for any u, v, w V and for any c, d F : 1. u + v V . 2. cv V . 19
3. u + v = v + u. 4. u + (v + w) = (u + v) + w 5. There exists an element denoted 0 V such that u + 0 = u. 6. For every element u V , there exists an element denoted u such that u + (u) = 0. 7. c(u + v) = cu = cv. 8. (c + d)u = cu + du 9. c(du) = (cd)u. 10. 1u = u. We often say that F is the eld of scalars. Some familiar vector spaces are R2 and R3 . In these cases, the vectors are the sets of all real vectors of length 2 or 3, and the elds of scalars are the real numbers. An important example in coding theory is the vector space of all binary n-tuples. Here, the vectors are the binary n-tuples and the eld is GF (2). We say that 0 and 1 are the scalars. We can denote this set as GF (2)n , but when we are thinking of this as a vector space, we normally write V (n, 2), which indicates that our vectors are of length n over the alphabet eld of size 2. We will use the following results on vector subspaces more often than the above formal denition of a vector space: Denition 2.2.4. Let V be a vector space. A subset U of V is called a subspace of V if U is itself a vector space under the same operations as V . Theorem 2.2.5. Let V be a vector space and F be a eld. A subset U of V is a subspace if: 1. For any two vectors u, v U , u + v is also in U ; 2. For any c F , u U , cu is also in U . You may recall that R2 and R3 are known as 2-dimensional and 3-dimensional spaces, respectively. In order to make this concept more precise, we need to develop some mathematical theory. Denition 2.2.6. Two vectors are linearly independent if they are not scalar multiples of each other. Otherwise, the vectors are called linearly dependent. Example 2.2.7. When working in the familiar vector space of R3 , the vectors [100] and [030] are linearly independent, since they are clearly not scalar multiples of each other. However, [211] and [633] are linearly dependent, since the latter vector is 3 times the former vector. Denition 2.2.8. Vectors in a set are linearly independent if none of the vectors are linear combinations of other vectors in the set. More precisely, the vectors {v0 , v1 , . . . , vn1 } are linearly independent if the only linear combination c0 v0 + c1 v1 + + cn1 vn1 that gives 0 is the linear combination where all scalars ci are equal to 0. Otherwise, the vectors are called linearly dependent. Example 2.2.9. When working in the familiar vector space of R3 , the vectors [100], [010], and [001] are linearly independent, since none of these vectors are linear combinations of the others. More precisely, if c0 [100] + c1 [010] + c2 [001] = 0, then the only possible solution to this equation is when c0 = c1 = c2 = 0. However, [100], [010], and [550] are linearly dependent since the latter vector is a linear combination of the rst two. More precisely, if c0 [100] + c1 [010] + c2 [550] = 0, then we can solve this equation with c0 = 5, c1 = 5, and c2 = 1. Since the coecients of this linear combination are not all zero, the vectors are linearly dependent. (Notice that the abbreviation [xyz] is used for the vector [x, y, z] when no confusion should occur.) 20
Question 7. Are the following vectors independent in R3 : 1. {[1, 1, 1], [0, 3, 4], [3, 9, 11]}? 2. {[1, 0, 1], [12, 3, 0], [13, 3, 0]}? Denition 2.2.10. A basis is a (minimal cardinality) set of linearly independent vectors that span the vector space. It follows from this denition that any vector in the vector space is a linear combination of the basis vectors. For example, the standard basis for R3 is {[100], [010], [001]}, and any vector in R3 can be written as a linear combination of these 3 vectors. To illustrate this, notice that the vector [2, 3, ] can be written as 2[100] 3[010] + [001]. There are innitely many bases for R3 . To nd an alternate basis, we need to nd 3 linearly independent vectors such that their linear combinations produce all possible vectors in R3 . Question 8. Produce an alternate example for a basis for R3 . Similarly, the standard basis for R2 is {[10], [01]}. In the vector space V (4, 2), consisting of the 16 binary strings of length 4, the standard basis is {[1000], [0100], [0010], [0001]}, since any of the other binary strings can be obtained as binary linear combinations of these vectors. Denition 2.2.11. The dimension of a vector space V is number of vectors in any basis for V . We now understand precisely why R2 and R3 are said to be 2-dimensional and 3-dimensional vector spaces over the real numbers, respectively. Moreover, V (4, 2) is a 4-dimensional vector space over the binary numbers. An important concept for error-control coding is that of vector subspaces. Familiar subspaces from geometry are planes and lines in 3-space: Planes are 2-dimensional vector subspaces of R3 , and lines are 1-dimensional vector subspaces of R3 . Denition 2.2.12. Let V be a vector space over a eld F and let W be a subset of V . If W is a vector space over F , then W is a vector subspace of V . Theorem 2.2.13. If W is a subset of a vector space V over a eld F , then W is a vector subspace of V i u + v W for all , F and all u, v W . Proof. Suppose that W is a vector subspace of V , a vector space over the eld F . Then, W is itself a vector space over F . Therefore, by the denition of vector spaces, v + u W for any v, u W and sv W for any s F . This directly implies that u + v W for all , F and all u, v W . Now suppose that u + v W for all , F and all u, v W . Then, setting = = 1 F gives the rst requirement of a vector space and setting = 1 F and = 0 F gives the second requirement of a vector space. We can now begin to connect vector spaces to error-control codes. In Section 2.1, we discussed the linear code C = {000, 100, 001, 101}. We will now show that this code is a vector subspace of the vector space V (3, 2) of binary 3-tuples. Using Theorem 2.2.13, we must check if every linear combination of vectors in C remains in C. This should sound familiar! In fact, this was the dening condition for a linear code! We have just discovered one of the major links between coding theory and linear algebra: Linear codes are vector subspaces of vector spaces. Denition 2.2.14. A linear code of length n over GF (q) is a subspace of the vector space GF (q)n = V (n, q). Theorem 2.2.15 shows the consistency between the denitions for binary and general linear codes. 21
Theorem 2.2.15. Let C be a linear code. The linear combination of any set of codewords in C is a code word in C. Proof. We must show that (1) u + v C for all u, v in C, and (2) u C, for all u C and GF (q). Since C is a subspace of V (n, q), this follows from Theorem 2.2.13. We can deepen this connection between linear codes and vector subspaces by looking at the dimension of vector subspaces. Example 2.2.16. Lets continue working with our linear code C = {000, 100, 001, 101}. We can see that this code is spanned by the linearly independent vectors [100] and [001], since all other vectors in C can be obtained as binary linear combinations of these two vectors. In other words, these two vectors form a basis for C, so the dimension of C is 2. Denition 2.2.17. The dimension of a linear code C of length n is its dimension as a vector subspace of GF (2)n = V (n, 2). If C is a k-dimensional subspace of V (n, q), we say that C has dimension k and is an [n, k, d] or [n, k] linear code. (Be careful not to confuse the notation [n, k, d] with our earlier notation (n, M, d), where M represents the total number of codewords in the code.) The dimension of a code is very important because it is directly related to M , the number of codewords in a code. In particular, if the dimension of a binary code is k, then the number of codewords is 2k . In general, if the dimension of a q-ary code is k, then the number of codewords is q k . This means that C can be used to communicate any of q k distinct messages. We identify these messages with the q k elements of V (k, q), the vector space of k-tuples over GF (q). This is consistent with the aforementioned notion of encoding messages of length k to obtain codewords of length n, where n > k. We can now also fully understand the output that Gap gives when dening a code. Recall the command and output gap> C := RepetitionCode(5, GF(2)); a cyclic [5,1,5]2 repetition code over GF(2) We now see that [5,1,5]2 means that the binary repetition code of length 5 is a [5, 1, 5] linear code. The dimension is 1 since the code is spanned by the one basis vector [11111]. The 2 after this notation refers to the covering radius of the code. We will not be covering this concept in this course.
2.3
Encoding Linear Codes: Generator Matrices
Linear codes are used in practice largely due to the simple encoding procedures facilitated by their linearity. A k n generator matrix G for an [n, k] linear code C provides a compact way to describe all of the codewords in C and provides a way to encode messages. By performing the multiplication mG, a generator matrix maps a length k message string m to a length n codeword string. The encoding function m mG maps the vector space V (k, q) onto a k-dimensional subspace (namely the code C) of the vector space of V (n, q). Before giving a formal denition of generator matrices, lets start with some examples. Example 2.3.1. We will use the familiar repetition code example to illustrate how we can use matrices to encode linear codes. As you know, in the repetition code, 0 is encoded as 00000 and 1 is encoded as 11111. This process can be automated by using a generator matrix G to encode each message: If G = [11111], then for a message m, the matrix multiplication mG outputs the 22
corresponding codeword. Namely [0]G = [00000] and [1]G = [11111]. So, this 1 5 matrix G is working to map a message of length 1 to a codeword of length 5. Since we knew exactly how our two messages encode to our two codewords, it was very easy to nd this matrix. Example 2.3.2. We will now extend the work done in Example 2.2.16 on C = {000, 100, 001, 101}. We have already shown that C is a 2-dimensional vector subspace of the vector space V (3, 2). Therefore, C is a [3, 2] linear code and should have a 2 3 generator matrix. However, lets use our intuition to derive the size and content of the generator matrix. We want to nd a generator matrix G for C that will map messages (i.e. binary bit strings) onto the four codewords of C. But how long are are the messages that we can encode? We can answer this question using three facts: (1) we always encode messages of a xed length k to codewords of a xed length n; (2) a code must be able to encode arbitrary messages of length k; and (3) the encoding procedure must be one-toone, or in other words, no two messages can be encoded as the same codeword. Using these three facts, it is clear that the messages corresponding to the four codewords in C must be of length 2 since there are exactly 4 distinct binary messages of length 2: [00], [10], [01], and [11]. Hence, in order for the matrix multiplication to make sense, G must have 2 rows. Since the codewords are of length 3, properties of matrix multiplication tell us that G should have 3 columns. In particular, if 100 G= , then [00]G = [000], [10]G = [100], [01]G = [001], and [11]G = [101]. Hence, this 001 code C can encode any length 2 binary message. The generator matrix determines which length 2 message is encoded as which length 3 codeword. We have now seen how a generator matrix can be used to map messages onto codewords. We have also seen that an [n, k] linear code C is a k-dimensional vector subspace of V (n, q) and can hence be specied by a basis of k codewords, or q-ary vectors. This leads us to the following formal denition of a generator matrix: Denition 2.3.3. A k n matrix G whose rows form a basis for an [n, k] linear code C is called a generator matrix of the code C. Denition 2.3.3 shows that the dimension of a code C is equal to the number of rows in its generator matrix, which is consistent with our above examples. In Example 2.3.1, we found a 1 5 generator for our [5,1] repetition code (length 5 and dimension 1, since it is spanned by the vector [11111]). In Example 2.3.2, we indeed found a 2 3 generator matrix for the [3, 2] linear code C. Since Denition 2.3.3 stipulates that the rows of a generator matrix must form a basis for the code, the rows of a generator matrix must be linearly independent. In fact, a matrix G is a generator matrix for some linear code C if and only if the rows of G are linearly independent. Example 2.3.4. The following matrix is not a 11 1 0 0 1 01 generator matrix for any linear code: 000 1 1 0 . 0 1 1 110
To see this, notice that the rows cannot form a basis for a linear code because the rows are linearly dependent. There are several ways to discover this. For example, you can see that (bit-wise binary) adding the rst and second rows gives you the fourth row or reveal the dependency by obtaining a row of zeros through row-reduction. Alternately, you can note that the messages [1100] and [0001] are both encoded to the same codeword [01110]. Similarly, the messages [1101] and [0000] both encode to [00000]. Without a generator matrix to describe the code and dene the encoding of messages, it would be necessary to make a list of all of the codewords and their corresponding messages. This is very simple for the repetition code, however, this would become highly inconvenient as the number of codewords (and therefore messages) increases. 23
1000 Question 9. Given the generator matrix G = 0 1 0 0 , what is the dimension of the 0011 associated code? How long are the messages to be encoded by G? How long will the resulting codewords be? Use G to encode the all-1s message of the appropriate length. We can use Gap to facilitate the generation of linear codes from their matrices, and the subsequent encoding of messages. The command GeneratorMatCode generates a code based on a generator matrix. For example, suppose our generator matrix of a binary code is: 10001 G = 0 1 0 1 0 . 00111 Then, the basis vectors of this code are [1 0 0 0 1], [0 1 0 1 0], and [0 0 1 1 1]. We can now use Gap to construct our linear code from this generator matrix (or basis for the code): gap> C := GeneratorMatCode([[1,0,0,0,1],[0,1,0,1,0],[0,0,1,1,1]],GF(2));; To review the generator matrix for this code C, we can use a combination of the functions Display and GeneratorMat. For example: gap> 1. .1 .. Display(GeneratorMat(C)); ..1 .1. 111
The dots in the output represent 0s. We can also use Gap to encode messages. For example, to encode [1, 1, 1] we use: gap> m := Codeword("111", GF(2)); gap> m * C; [11100] Notice we do not have to use GeneratorMat to reproduce the generator matrix to encode the message m. We can encode the message directly by multiplying the message, m with the code, C. Question 10. Conrm the above encoding by hand. We have now seen how generator matrices can be used to encode messages. Generator matrices are also used to discover codes, as illustrated in the following example: Example 2.3.5. Suppose we need a [3,2] binary linear code C, that is, we need a 2-dimensional vector subspace of V (3, 2), the vector space of binary 3-tuples. In order to dene a 2-dimensional vector subspace of V (3, 2), we need 2 linearly independent basis vectors. Arbitrarily, lets choose these vectors to be 011 and 110. Let C be spanned by these vectors. Then the generator matrix is G= 110 011 .
Taking all the linear combinations of the rows in G, we generate the code C = {110, 011, 101, 000}. Question 11. Use matrix multiplication to convince yourself that taking all linear combinations of the rows of the 2 3 matrix G above is equivalent to taking all products mG, where m runs through all 4 binary 2-tuples. In particular, show that the product [11]G produces a vector which is the sum of all rows in G. 24
There was nothing special about the basis vectors [110] and [011] that we chose in the above example. In fact, there would be advantages to choosing permutations of these vectors obtained by swapping the rst two entries in each vector. This would produce the generator matrix G= 110 101 .
This is the systematic or standard form of G since it has the identity matrix on the right of G. When a data block is encoded using a generator matrix in standard form, the data block is embedded without modication in the last k coordinates of the resulting codeword. This facilitates decoding, especially when the decoding is implemented with linear feedback shift registers. When data is encoded in this fashion, it is called systematic encoding. Question 12. Using G above, encode all possible length 2 messages to conrm that the messages appear in the last 2 bits of each codeword. We can go one step further into the theory of linear algebra to deepen the understanding of the connections between codes and matrices. Denition 2.3.6. The rowspace of an matrix is the set of all vectors that can be obtained by taking linear combinations of the rows of the matrix. Recall that taking all linear combinations of the rows of an r c matrix G is equivalent to taking products vG, where v is any 1 r vector. It follows that the code C represented by a generator matrix G is equivalent to the rowspace of G. Example 2.3.7. The rowspace of the matrix 100 M = 0 1 0 001
is all of R3 , since any real vector [xyz] in R3 can be formed as a real linear combination of the rows of M . More precisely, any real vector [xyz] R3 can be written as [xyz]M , which represents a linear combination of the rows of M . Question 13. Find a 3 3 matrix whose rowspace represents a plane in R3 . Recall that a plane is a 2-dimensional subspace of R3 . Question 14. Is the matrix you dened in the previous question a generator matrix for a linear code? We have just introduced several equivalent constructs: vector subspaces, linear codes, and row spaces of matrices. As we delve further into the intriguing topics of coding theory, linear algebra, and abstract algebra, we will see that the lines between pure mathematics and real-world applications continues to blur.
2.4
Introduction to Parity Check Matrices
Denition 2.4.1. Let C be an [n, k] linear code. A parity check matrix for C is an (n k) n matrix H such that c C if and only if cH T = 0. Example 2.4.2. If C is the code {111, 000}, then a valid parity check matrix is H= 101 011 25 .
One way to verify this is to compute the product cH T for an arbitrary codeword c = [x, y, z]: 10 [xyz] 0 1 = [x + z, y + z] 11 So, c = [xyz] is a valid codeword i cH T = 0, ie. i [y + z, x + z] = [0, 0]. Algebraic manipulations over binary show that x = y = z. Thus the only valid codewords are indeed {000, 111}. Notice that this is the binary repetition code of length 3. Question 15. Find a parity check matrix for the binary repetition code of length 5. Example 2.4.3. The (10,9) binary parity check code (BPCC) is a code of length 10 and dimension 9 which encodes binary messages of length 9 into binary codewords of length 10 by appending an overall parity check bit. For example, the message [001100100] is encoded as [0011001001] and the message [100101100] is encoded as [1001011000]. The parity check matrix for this code is H = [1111111111] since then the product cH T forms the sum (modulo 2) of the bits in the length 10 codeword c; this product gives 0 precisely when the parity of the codeword is even, or in other words, precisely c is a member of the BPCC. Question 16. Find a parity check matrix for the (5, 4) binary parity check code, which encodes messages of length 4 to codewords of length 5 by appending a parity check bit. The Guava function CheckMat generates a parity check matrix for any given linear code. For example, we can generate a check matrix for a repetition code of length 5 as follows: gap> C := RepetitionCode(5, GF(2));; gap> H := CheckMat(C);; gap>Display(H); 11... .11.. ..11. ...11 Question 17. Show that any codeword c in the binary repetition code of length 5 indeed satises the equation cH T , where H is given by the Gap output above. The connection drawn above between the codewords of a code C and the parity check matrix of a code C can be strengthened by considering the nullspace of the parity check matrix. Denition 2.4.4. The nullspace of a matrix H is the set of all vectors x such that HxT = 0. In linear algebra, we study nullspaces of matrices, often in the context of nding the kernel of linear mappings. However, there is a simple connection to linear codes. Given the parity check matrix H for a code C, we have 0 = cH T for all c C, hence 0T = (cH T )T = HcT . Therefore, the codewords of C comprise the nullspace of its parity check matrix H. Question 18. Find the nullspace of B= 110 101 .
Theorem 2.4.5. An (n k) n parity check matrix H for a code C in systematic or standard form if H = (Ink | A). Then the corresponding generating matrix in systematic form is G = (AT | Ik ).
26
Notes: (1) Authors vary in their denitions of the systematic form of generator and parity check matrices; some put the identity matrix to the left for systematic generator matrices and to the right for systematic parity check matrices; and (2) When working in binary, the negative sign in G = (AT | Ik ) is moot. Question 19. Find systematic generator matrices for the codes dened by the parity check matrices in Examples 2.4.2 and 2.4.3. Note that these parity check matrices are given in systematic form. Any parity check matrix or generating matrix can be put in systematic form by performing row operations on the matrix or column permutations. This substantially changes neither the nullspace nor the rowspace of the matrix, which is important since C is the nullspace of H and the rowspace of G. Depending on the operations used to convert the dening matrices, either the same code or an equivalent code will be obtained. Denition 2.4.6. Two q-ary linear codes are called equivalent if one can be obtained from the other by a combination of operations of the following types: 1. Permutation of the positions of the codewords 2. Multiplication of the symbols appearing in a xed position by a nonzero scalar (i.e. element of GF (q)). Example 2.4.7. The two codes C1 and C2 below are equivalent since the second is merely the permutation of the rst two positions in the rst code. 000 000 101 011 C1 = C2 = 010 100 111 111 Theorem 2.4.8. Two k n matrices generate equivalent [n, k] linear codes over GF (q) if one matrix can be obtained from the other by a sequence of operations of the following types: 1. Permutation of the rows 2. Multiplication of a row by a nonzero scalar 3. Addition of a scalar multiple of one row to another 4. Permutation of the columns 5. Multiplication of any column by a nonzero scalar. Proof. The rst three types of operations preserve the linear independence of the rows of a generator matrix and simply replace one basis of the code by another basis of the same code. The last two types of operations convert a generator matrix for one code into a generator matrix for an equivalent code. Gap can be used to put parity check matrices and generator matrices into systematic or standard form. It is necessary to remember which standard form requires the identity matrix on the left. For our convention, we will put the identity on the right in generator matrices and on the left for parity check matrices. For example, suppose I have this check parity matrix 1010 H = 0 1 1 0 . 0001 27
We can represent this check matrix in a systematic form by using the PutStandardForm command which inputs two values, the matrix and a boolean variable, true or false. PutStandardForm(H, true) will arrange the input matrix H so that the identity matrix is on the left. PutStandardForm(H, false) will arrange the input matrix H so that the identity matrix is on the right. gap> H := [[1,0,1,0], [0, 1, 1, 0], [0, 0, 0, 1]]; gap> [[ [ [ Display(H); 1, 0, 1, 0 ], 0, 1, 1, 0 ], 0, 0, 0, 1 ] ]
gap> PutStandardForm(H, true);; gap> [[ [ [ Display(H); 1, 0, 0, 1 ], 0, 1, 0, 1 ], 0, 0, 1, 0 ] ]
Note that the (3,4) output indicates that the arranged matrix is 3 4, and then we use the Display command to view the standard form of H.
2.5
Parity Check Matrices and Linear Decoding
This section discusses a nearest neighbor decoding scheme that uses parity check matrices and cosets (which you rst encountered when cryptanalyzing the Vigenere cipher) to decode a linear code. The dening equation of a parity check matrix (c C cH T = 0) is used in the rst step of the decoding of linear codes. Upon receiving a word r, the product rH T is computed. If the result is 0, we assume that no errors have occurred and decode the received word as r. Question 20. Is this a fair assumption? We will now see how to decode if rH T = 0. Denition 2.5.1. Suppose that C is an [n, k] linear code over GF (q) and that a is any vector in V (n, q). Then the set a + C = {a + x | x C} is called a coset of C. Theorem 2.5.2. Suppose C is an [n, k] linear code over GF (q). Then every vector of V (n, q) is in some coset of C, every coset contains exactly q k vectors, and two cosets are either disjoint (non-overlapping) or equal. Example 2.5.3. Let C be the binary [4, 2] linear code with generator matrix G = Then C = {0000, 1011, 0101, 1110}, and the cosets of C are 0000 + C 1000 + C 0100 + C 0010 + C = {0000, 1011, 0101, 1110} = {1000, 0011, 1101, 0110} = {0100, 1111, 0001, 1010} = {0010, 1001, 0111, 1100}. 1011 0101 .
Notice that coset 0001 + C is equal to coset 0100 + C, and we have only listed this coset once. Denition 2.5.4. The coset leader of a coset is chosen to be one of the vectors of minimum weight in the coset. 28
Example 2.5.5. In Example 2.5.3, either 0100 or 0001 could be chosen as the coset leader for the corresponding coset. The coset decoding algorithm for an [n, k] linear code works as follows. We partition GF (q)n into cosets of C. There are q n /q k = q nk cosets, each of which contains q k elements. For each coset, pick a coset leader. There may be an arbitrary choice involved at this step. Then, if r is received, nd the coset that contains r. This coset is of the form e + C, and we guess that r e was sent. Note that coset leaders, dened to be of least weight, represent error vectors, and this decoding algorithm assumes that lowest weight error vectors are the most probable. In order to carry out the above decoding algorithm, we make a standard array by listing the cosets of C in rows, where the rst entry in each row is the coset leader. More specically, the rst row of the standard array is 0 + C, with 0, the coset leader, listed rst. We next pick a lowest-weight vector of GF (q)n that is not in C and put it as a coset leader for the second row. The coset in the second row is obtained by adding its leader to each of the words in the rst row. This continues until all cosets of C are entered as rows. The standard array for Example 2.5.3 is: 0000 1000 0100 0010 1011 0011 1111 1001 0101 1101 0001 0111 1110 0110 1010 1100
We decode r as the codeword at the top of the column containing r. The error vectors that will be corrected are precisely the coset leaders. By choosing a minimum weight vector in each coset as coset leader we ensure that standard array decoding is a nearest neighbor decoding scheme. Since the code in the above example has minimum distance equal to 2, we cannot even correct all single errors. In fact, this code corrects all single errors that occur within the rst 3 positions, but it does not correct errors that occur in the fourth position. Question 21. Explain why the above code, with the above decoding scheme, cannot correct errors in the fourth position. Question 22. Construct a standard array for the binary repetition code of length 4. Syndrome decoding is a related decoding scheme for linear codes that uses the parity check matrix H of a code. Suppose x is transmitted and r is received. Compute rH T , which we call the syndrome of r and write S(r). We know from the denition of a parity check matrix that rH T = 0 if and only if r C. So, if S(r) = rH T = 0, then we conclude that most likely no errors occurred. (It is possible that suciently many errors occurred to change the transmitted codeword into a dierent codeword, but we cannot detect or correct this.) If rH T = 0, then we know that at least one error occurred, and so r = x + e, where x is a codeword and e is an error vector. Since xH T = 0, we see that rH T = (x + e)H T = xH T + eH T = eH T . So the syndrome of the received word r is equal to the syndrome of the error vector e that occurred during transmission. It is known that S(u) = S(v) if and only if u and v are in the same coset of C. It follows that there is a one-to-one correspondence between cosets and syndromes. This means that every word in a particular coset (i.e. in a particular row of the standard array) has the same syndrome. Thus, we can extend the standard array by listing the syndromes of each coset leader (and hence of each element in the coset) in an extra column. Example 2.5.6. When the standard array of Example 2.5.3 is expanded to allow for syndrome
29
decoding, it looks as follows: 0000 1000 0100 0010 1011 0011 1111 1001 0101 1101 0001 0111 1110 0110 1010 1100 00 11 . 01 10
The syndrome decoding algorithm works as follows. When a vector r is received, calculate the syndrome S(r) = rH T . Locate the syndrome in the table, which requires looking through only one column. Decode r as r e where e is the coset leader in the row containing S(r). This is equivalent to decoding r as the codeword at the top of the column containing r. Using syndromes saves storage space and time. Instead of storing the entire standard array, we need only store the error vectors and the syndromes of the error vectors for each error vector that we hope to detect. Also, when n is large, locating a received vector within a standard array (as compared to locating a syndrome within the syndrome column) can be dicult and time consuming. Example 2.5.7. Suppose that we are decoding using the expanded standard array shown in Example 2.5.6. Suppose 1111 is received, and compute S(1111) = 01. We nd this syndrome in the third row of the syndrome column. The coset leader for this row is 0100, and so we decode 1111 as 1111 0100 = 1011. The following theorem will help connect the ideas of syndromes and extended arrays to parity check matrices: Theorem 2.5.8. For a binary code, the syndrome of a received codeword is a vector that is equal to the sum of the columns of H that correspond to the positions where errors occurred. Proof. Let r be a received codeword with syndrome S(r) = rH T . Since r = c + e, where c is a valid codeword and e is an error vector, we have
S(r) = rH T = (c + e)H T = cH T + eH T = 0 + eH T = eH T
(6)
Notice that (6) follows from the fact that H is the parity check matrix for C. Since eH T is a combination of the columns in H that correspond to the positions where errors occurred, this proves the result. Decoding linear codes, calculating syndromes, and computing standard and extended arrays can be done using Gap. If all we want to know is whether a codeword is in our code or not, we can use the in operator. For example: gap> c := Codeword("10111", GF(2));; gap> C := RepetitionCode(5,GF(2));; gap> c in C; false 30
We can also use the built-in function for calculating syndromes. We illustrate this with an example of computing the syndrome of the codeword [11111] in the binary repetition code of length 5: gap> C := RepetitionCode(5, GF(2));; gap> c := Codeword("11111", GF(2));; gap> s := Syndrome(C,c); [0000] The function StandardArray is a function that will return the standard array of a linear code C. For example, we can construct a standard array for a repetition code of length 4. gap> StandardArray(RepetitionCode(4, GF(2))); [ [ [ 0 0 0 0 ], [ 1 1 1 1 ] ], [ [ 0 0 0 1 ], [ [ 0 0 1 0 ], [ 1 1 0 1 ] ], [ [ 0 0 1 1 ], [ [ 0 1 0 0 ], [ 1 0 1 1 ] ], [ [ 0 1 0 1 ], [ [ 0 1 1 0 ], [ 1 0 0 1 ] ], [ [ 1 0 0 0 ],
[ [ [ [
1 1 1 0
1 1 0 1
1 0 1 1
0 0 0 1
] ] ] ]
], ], ], ]]
Be sure to look at this output carefully to notice the placement of brackets. If we were to write out this standard array by hand, we would write eight rows with two entries each. The function SyndromeTable is a function that will return an abbreviated version of our extended standard array. The rst column in the output of SyndromeTable consists of the error vectors that correspond to the syndrome vectors in the second column. So, the output of SyndromeTable contains only the rst and last columns of our extended standard arrays. To make this clear, we will now produce the syndrome table for the same above repetition code: gap> SyndromeTable(RepetitionCode(4, GF(2))); [ [ [ 0 0 0 0 ], [ 0 0 0 ] ], [ [ 0 0 0 1 ], [ [ [ 0 0 1 1 ], [ 0 1 0 ] ], [ [ 0 0 1 0 ], [ [ [ 1 0 0 0 ], [ 1 0 0 ] ], [ [ 0 1 1 0 ], [ [ [ 0 1 0 0 ], [ 1 1 0 ] ], [ [ 0 1 0 1 ], [
0 0 1 1
0 1 0 1
1 1 1 1
] ] ] ]
], ], ], ]]
You will notice that the output of SyndromeTable is sorted according to the syndrome vectors. Although this may be confusing when you compare the output of SyndromeTable(RepetitionCode(4, GF(2))); to the output of StandardArray(RepetitionCode(4, GF(2)));, Gap does this to facilitate the search of the syndrome vectors, which is of utmost importance in decoding linear codes.
2.6
Parity Check Matrices, Minimum Distance, and the Singleton Bound
In addition to contributing to the decoding scheme for linear codes, parity check matrices can be used to determine the minimum distance of the code: Theorem 2.6.1. Let C have parity check matrix H. The minimum distance of C is equal to the minimum nonzero number of columns in H for which a nontrivial linear combination of the columns sums to zero. Proof. Since H is a parity check matrix for C, c C if and only if 0 = cH T . Let the column vectors of H be {d0 , d1 , . . . , dn1 }. The matrix equation 0 = cH T can be reexpressed as follows:
31
0
= cH T = (c0 , c1 , . . . , cn1 )[d0 d1 dn1 ]T = c0 d0 + c1 d1 + + cn1 dn1
This shows that c is a weight d > 0 codeword if and only if there is a nontrivial linear combination of d columns of H which equals zero. It now follows from Theorem 2.1.5 that the minimum distance of C is equal to the minimum nonzero number of columns in H for which a nontrivial linear combination of the columns sums to zero. 1010 Question 23. Suppose that 0 1 1 0 is a parity check matrix for a binary code C. What 0001 is the minimum distance of C? Theorem 2.6.1 can be used to bound the possible minimum distance for a code, as explained in the following theorem. Theorem 2.6.2. (Singleton bound) The minimum distance d for an [n, k] linear code satises d n k + 1. Proof. An [n, k] linear code has a parity check matrix H containing (n k) linearly independent rows. Linear algebra results then imply that any such H also has exactly (nk) linearly independent columns. It follows that any collection of (n k + 1) columns of H must be linearly dependent. The result now follows from Theorem 2.6.1. Codes that achieve the Singleton bound with equality are called maximum distance separable (MDS). For their given parameters n and k, they achieve the best possible minimum distance. Question 24. Show that the binary repetition code of length 5 is MDS. Gap can be used to check if a code is MDS. The following commands and output conrm that the binary repetition code of length 5 is MDS: gap> C := RepetitionCode(5, GF(2));; gap> IsMDSCode(C); true In Chapter 4.3, we will study some more interesting MDS codes.
2.7
Hamming Codes
The fact that the syndrome of a received vector is equal to the sum of the columns of the parity check matrix H where errors occurred gives us some insight about how to construct a parity check matrix for a binary code that will undergo coset or syndrome decoding. First, the columns of H should all be nonzero, since otherwise an error in the corresponding position would not aect the syndrome and would not be detected by the syndrome decoder. Second, the columns of H should be distinct, since if two columns were equal, then errors in those two positions would be indistinguishable. We use these observations to build a family of codes known as the binary Hamming codes, Hr , r 2. Any parity check matrix for the Hamming code Hr has r rows, which implies that each column of 32
the matrix has length r. There are precisely 2r 1 nonzero binary vectors of length r, and in order to construct a parity check matrix of Hr , we use all of these 2r 1 column vectors. Denition 2.7.1. A binary Hamming code Hr of length n = 2r 1, r 2, has parity check matrix H whose columns consist of all nonzero binary vectors of length r, each used once. This gives an [n = 2r 1, k = 2r 1 r, d = 3] linear code. Question 25. How many errors can a Hamming code correct? One parity check matrix for the binary [7, 4, 3] Hamming natural order of increasing binary numbers is as follows: 000111 0 1 1 0 0 1 101010 code H3 , where columns are taken in the 1 1 . 1
We now rearrange the columns to get a parity check matrix H in standard form: 1000111 0 1 0 1 0 1 1 0011101 A generator matrix G in standard form is: 01 1 0 1 1 11 0 0 0 1
1 1 0 1
1 0 0 0
0 1 0 0
0 0 1 0
Question 26. Encode the messages 0000 and 1010 using G. Check that the resulting codewords are valid by using H. Guava has built-in functions for Hamming codes. The [7, 4, 3] binary Hamming code is dened by the parameter r = 3, and is generated as follows: gap> C := HammingCode(3, GF(2)); a linear [7,4,3]1 Hamming (3,2) code over GF(2) To view the full list of codewords in the [7, 4, 3] binary Hamming code, we use the following command: gap> cw := AsSortedList(C); [[0000000], [0001111], [0010110], [0100101], [0101010], [0110011], [1000011], [1001100], [1010101], [1100110], [1101001], [1110000],
[0011001], [0111100], [1011010], [1111111]]
Notice that the output is sorted. The list returned is also immutable, meaning that elements of the list cannot be changed. It will sometimes be useful to be able to access the elements in the outputted list. For example, in what follows, we will add together two of the codewords in the list and check that their sum is also a codeword (guaranteed by the linearity of the code): gap> d := cw[2] + cw[3]; [0011001] gap> d in cw; true 33
Question 27. Write down a parity check matrix for the binary Hamming code with r = 4 by using the mathematical denition of Hamming codes. Check your answer with Gap. It is easy to decode Hamming codes, which are used when we expect to have zero or one error per codeword. If we receive the vector r, compute the syndrome S(r). If S(r) = 0, then assume that r was the codeword sent. If S(r) = 0, then, assuming a single error, S(r) is equal to the column of H that corresponds to the coordinate of r where the error occurred. Find the column of H that matches S(r), and then correct the corresponding coordinate of r. This decoding scheme is further simplied if the columns of H are arranged in order of increasing binary numbers. Example 2.7.2. In Section 1.3, we studied extended codes, which are built from existing codes by adding an overall parity check bit. This example shows a parity check matrix of the extended code of the binary Hamming code H3 : 01111000 1 0 1 1 0 1 0 0 1 1 0 1 0 0 1 0 11111111 Notice that the last row of the extended parity check matrix gives an overall parity-check equation on the codewords: x0 + x1 + + xn = 0. Compare this parity check matrix with the parity check for the corresponding Hamming code.
2.8
Answers to Chapter 2 Reading Questions
Answer 1. This Hamming code can correct 1 error. Answer 2. The rst code is not linear because it does not contain 101, the sum of 100 and 001. The second code is not because it does not contain 000 (which is the binary sum of any codeword with itself). Answer 3. gap> C := ElementsCode(["000", "100", "001"], GF(2));; gap> IsLinearCode(C): false gap> C := ElementsCode(["100", "001", "101"], GF(2));; gap> IsLinearCode(C): false Answer 4. Let C be a binary linear code. Let c be a codeword in C. Since C is linear, (c + c) = 0 must also be in C. Answer 5. We would need to consider code with M codewords.
M 2
pairs of codewords to nd the minimum distance of a
Answer 6. The additive inverse of 0 is 0. The additive inverse of 1 is 6 and its multiplicative inverse is 1. The additive inverse of 2 is 5, and its multiplicative inverse is 4. The additive inverse of 3 is 4, and its multiplicative inverse is 5. The additive inverse of 4 is 3, adn its multiplicative inverse is 2. The additive inverse of 5 is 2, and its multiplicative inverse is 3. The additive inverse of 6 is 1, and its multiplicative inverse is 6. Answer 7. {[1, 1, 1], [0, 3, 4], [3, 9, 11]} are linearly dependent since, for example, 3[1, 1, 1]+2[0, 3, 4]+ [3, 9, 11] = [0, 0, 0]. {[1, 0, 1], [12, 3, 0], [13, 3, 0]} are independent. linearly Answer 8. {[2, 0, 0], [0, 4, 0], [0, 0, 5]} is an alternate basis. Answer 9. The associated code has dimension 3 since there are three (linearly independent) rows in G and thus three basis vectors. The messages that are to be encoded by G are of length 3. The resulting codewords have length 4, which is the number of columns in G. The all-1s message will be encoded as [1111]. 34
Answer 10. (111)G = (11100) Answer 11. The four binary 2-tuples are {[00], [01], [10], [11]}. When performing matrix multiplication, notice that the above four binary 2-tuples designate all the dierent ways that we can add rows together. For example, we can see that [11]G is equivalent to adding all the rows of G, resulting in [110] + [011] = [101]. Answer 12. [00]G 1 Answer 13. 0 1 = [000], [01]G = [101], [10]G = [110], [11]G = [011] 00 1 0 00
Answer 14. No, this matrix is not a generator matrix for a linear code. Its rows are not linearly independent so they do not form a basis for a linear code. Answer 15. We know that H is an (nk)n matrix by denition, so the rst step is to determine n and k. We know that the dimension of any repetition code is 1 since it contains only two codewords: a string of ones and a string of zeroes. We also know that n = 5 since the length was given in the problem statement. So we must determine a 4 5 matrix such that the product of 11111 or 00000 with H T yields the zero vector. 00000 will work with any matrix, so we can craft the matrix based on what 11111 must multiply with to get the zero vector. This works when every column of H T has an even number of ones, or, when every row of H has an even number of ones. One way of creating this matrix is shown below. 1 0 H= 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1
Answer 16. Since every codeword has an even number of ones, any codeword multiplied by the all-ones vector will yield zero, as in Example 2.4.3. H= Answer 17. (00000)H T = (0000) (11111)H T = (0000) Answer 18. We know that the nullspace consists of every vector x such that HxT = 0. So we have x 0 1 1 0 1 x2 = . 0 101 x3 Then, x1 + x2 = 0 x1 + x3 = 0. By algebra, x1 = x2 = x3 , so the nullspace of B is 000, 111. Answer 1 1 1 1 G= 1 1 1 1 1 19. 1 0 0 0 0 0 0 0 0 G= 00 10 01 00 00 00 00 00 00 1 0 0 0 1 0 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 11111
35
Answer 20. Yes, this is a fair assumption. It is true unless so many errors have occurred that we started with one valid codeword and ended up with a dierent valid codeword. In that case, there would be no way to detect that anything went wrong and the received vector would be decoded incorrectly. Answer 21. There are a total of 16 binary length 4 vectors, and this code contains 4 of these vectors. So our standard array can only have 4 rows. Thus, only three weight 1 vectors can be coset leaders. In this case, (0001) is not a coset leader, so a single error in this position will not be decoded correctly. Answer 22. 0000 1000 0100 0010 0001 1100 1010 1001 1111 0111 1011 1101 1110 0011 0101 0110
By examining the coset leaders, we can see that this corrects all single-errors but only three types of double-errors. Answer 23. No single column is zero, so the minimum distance is not 1. No two columns have a linear combination that sums to 0, so the minimum distance is not 2. However, the third column is a linear combination of the rst two, so the minimum distance of C is 3. Answer 24. Since the binary repetition code of length 5 satises the Singleton bound with equality, it is MDS. Check this using d = 5, n = 5, and k = 1. Answer 25. A Hamming code can correct one error since it has a distance of 3. Answer 26. (0000)G = (0000000) (1010)G = (1011010) (0000000)H T = (000) (1011010)H T = (000) Answer 27. We can form this matrix by writing down as columns all of the length 4 binary vectors except for the zero vector. We put the identity on the left side in order for the parity check matrix 100000111001111 0 1 0 0 0 1 1 0 0 1 1 1 1 0 1 to be in standard form. 0 0 1 0 1 1 0 0 1 0 1 1 0 1 1 000110010110111
2.9
Chapter 2 Problems
Problem 2.1. Are the following sets of vectors linearly independent? 1. [101111], [000001], [110111] Z6 . 2 2. [000], [101] Z3 . 2 3. [10000], [01001], [00111] Z5 . 2 4. [0212], [0010], [2212] Z4 . 3 10110 Problem 2.2. Let G = 0 1 0 1 1 . Determine whether or not the following words are in 11010 the rowspace of G: 36
1. [11111] 2. [11101] 3. [00111] Problem 2.3. Let C = {[000000], [101110], [001010], [110111], [100100], [011001], [111101], [010011]} Z6 . Find a generator matrix for C and a parity check matrix for C. 2 Problem 2.4. List the cosets of the linear code having the following generator matrix: 1 0 0 0 1 1 0 0 0 1 1 0 1 0 1 1 0 1 0 1 0 0 1 0 0 0 0 1
Problem 2.5. Construct a standard array for a binary code having the following generator matrix: 10110 . Decode the received vectors 11111 and 01011. Give examples of (a) two errors 01011 occurring in a codeword and being corrected and (b) two errors occurring in a codeword and not being corrected. Problem 2.6. Construct a syndrome look-up table for the perfect binary [7, 4, 3] code which has 1000111 0 1 0 0 1 1 0 generator matrix 0 0 1 0 1 0 1 . Use the table to decode the following received vectors: 0001011 0000011, 1111111, 1100110, and 1010101. Repeat this exercise using Gap. 100110 Problem 2.7. Let C Z6 be the linear [6, 3] code with generator matrix. G = 0 1 0 0 1 1 2 001111 1. List the elements of C. 2. Find the minimum distance of C. 3. How many errors will C correct? 4. Find a check matrix for C. Problem 2.8. Let C be a q-ary linear code with parity check matrix H. Let u, v GF (q)n . Prove directly that uH T = vH T if and only if u and v lie in the same coset of C. Problem 2.9. Dene an encoding map E: Z4 Z6 by E((x1 , x2 , x3 , x4 )) = (x1 , x2 , x2 , x3 , x4 , x4 ). 2 2 Show that the encoding map E represents the encoding of a linear code. In other words, show that the codewords produced under this map satisfy the requirements of a linear code. Problem 2.10. Dene an encoding map E: Z6 Z6 by E((x1 , x2 , x3 , x4 , x5 , x6 )) = (x3 , x4 , x1 , x5 , x2 , x6 ). 2 2 Show that the encoding map E represents the encoding of a linear code. In other words, show that the codewords produced under this map satisfy the requirements of a linear code. Problem 2.11. Let C be a linear [n, k] code over Zp , where p is a prime number. How many elements does C contain? Explain your answer. 1011 0112 codewords of C. What is the minimum distance of C? Is C a perfect code? Problem 2.12. Let C be a ternary linear code with generator matrix Problem 2.13. Extended Codes: 37 . List the
1. Show that if C is a binary linear code, then the extended code obtained by adding an overall parity check to C is also linear. 2. Suppose that C is a binary code with parity check matrix H. Write down the parity check matrix for the extended code of C. (Your new parity check matrix will involve the matrix H.) 3. Generalize the idea of extending binary codes with an overall parity check matrix to dene extended codes of q-ary codes. What condition should the weight of each codeword in the extended code satisfy? Problem 2.14. Prove that either all of the codewords in a binary linear code have even weight or exactly half have even weight. Problem 2.15. Let U (n) Zn be the set of elements of Zn , that are relatively prime to n. Show that U (n) is a group under multiplication modulo n. Problem 2.16. Prove that the identity element in a group is unique Problem 2.17. Prove that if G is a group and a G, then a1 is unique. Problem 2.18. Show that a binary code can correct all single errors if and only if any parity check matrix for the code has distinct nonzero columns. Problem 2.19. Prove that any Hamming code has minimum distance equal to 3. Problem 2.20. Prove that binary Hamming codes are perfect. Problem 2.21. Let C be an [n, k] binary linear code with generator matrix G. If G does not have a column of zeroes, show that the sum of the weights of all the codewords is n 2k1 . [Hint: How many codewords are there with a 1 in coordinate i for xed i?] Generalize this to linear codes over the eld GF (q). Problem 2.22. Let A = {0, 1}. 1. Does there exist a code C A8 where C has exactly four codewords and such that C corrects 2 errors? If so, nd such a C. If not, prove it 2. Does there exist a code C A8 where C has exactly ve codewords and such that C corrects 2 errors? If so, nd such a C. If not, prove it. Problem 2.23. Alice and Bob have used the Bennett-Brassard protocol to generate two strings of bits which ideally are identical but which in fact contain some discrepancies. They have determined that it is very unlikely that a block of seven bits will contain more than one error; so they have decided to use the [7,4] binary Hamming code to correct their errors. 1. Bobs rst block of seven bits is 1110110. Alice sends him the syndrome 111. What does Bobs rst block become after error correction? 2. Bobs second block is 0000111. Alice sends him the syndrome 111. What does Bobs second block become after error correction? Problem 2.24. Prove that any linear ternary code (i.e. its alphabet has three symbols, 0, 1, 2 under modulo 3) with a check matrix H can correct all single errors if and only if no two columns of H have sum or dierence 0.
3
Cyclic Codes, Rings, and Ideals
This chapter studies more advanced topics from both abstract algebra and coding theory. 38
3.1
Introduction to Cyclic Codes
Denition 3.1.1. An [n, k, d] linear code C is cyclic if whenever (c0 , c1 , . . . , cn1 ) is a codeword in C, then (cn1 , c0 , . . . , cn2 ) is also a codeword in C. Example 3.1.2. The binary code C = {000, 110, 011, 101} is a cyclic code. Cyclic codes can be implemented eciently via simple hardware devices called shift registers. This is of great interest in applications involving ber optics, where high-speed data rates are possible. When working with cyclic codes, it is convenient to convert codeword vectors c = (c0 , c1 , . . . , cn1 ) of length n into code polynomials c(x) = c0 + c1 x + + cn1 xn1 of degree less than n. Note that the left-most bit in a codeword is associated with the constant term in the code polynomial. Gap can be used to convert a codeword to a codeword polynomial. For example: gap> c := Codeword("0111000", GF(2));; gap> PolyCodeword(c); x_1^3+x_1^2+x_1 Question 1. What is the corresponding code polynomial for the codeword (1, 0, 1, 1)? Question 2. For a codeword of length 5, what is the maximum degree of the associated code polynomial? From now on, we will use the terms codeword and code polynomial interchangeably. This abuse reects the fact that you should be thinking about codewords and code polynomials as representing the same thing. The power of using code polynomials will be apparent when we develop some additional structure. We will parallel the development of Zp , the integers modulo p, from the regular integers Z. Recall that with the integers modulo p, we constantly wrap around according to the modulus p. In other words, the largest number we can use is p 1, since as soon as we hit p or larger, we are reduced modulo p. We think of p as being equivalent to 0, so all multiples of p are ignored. Now consider the code polynomials {0, 1 + x, x + x2 , 1 + x2 } corresponding to the code C = {000, 110, 011, 101}. Notice that the highest power appearing among the code polynomials is x2 , since a higher power term would indicate a codeword with length greater than 3. Similarly to working with integers modulo a specic prime integer p, we can work with polynomials modulo a specic polynomial, for example p(x) = x3 1. In this situation, we think of x3 1 being equivalent to 0, or in other words, x3 is equivalent to 1. Lets see what this means with an example. Example 3.1.3. Consider the code polynomial c(x) = 1+x2 corresponding to the codeword c = 101 from C = {000, 110, 011, 101}. Lets multiply c(x) by x to obtain c(x)x = c (x) = x + x3 . However, if we are working modulo p(x) = x3 1, then x3 is equivalent to 1. So, c (x) is equivalent to x + 1 modulo p(x). Notice that rearranging this new polynomial with terms from lowest to highest powers gives 1 + x, which is the code polynomial for the code word 110 C. Notice furthermore that 110 is the right cyclic shift of 101. Coincidence? Lets try this again. This time, start with the code polynomial d(x) = x + x2 corresponding to the codeword 011 in C. Multiplying d(x) by x gives d (x) = x2 + x3 . Taking this result modulo x3 1 gives x2 + 1, which (upon rearranging terms from lowest to highest powers) corresponds to the codeword 101 C and is the right cyclic shift of 011. Interesting! Example 3.1.3 suggests a true fact about cyclic codes. In a cyclic code C of length n, the product xc(x) modulo xn 1 produces another code polynomial in C, namely the right cyclic shift of 39
c(x). More precisely, when working with code polynomials of degree less than n corresponding to codewords of length n, by working modulo xn 1, we can achieve a right cyclic shift of a codeword by multiplying the associated code polynomial by x: Consider the code polynomial c(x) = c0 +c1 x+ +cn1 xn1 . Multiplying c(x) by x modulo xn 1 gives c (x) = c0 x+c1 x2 + +cn1 xn c0 x + c1 x2 + + cn1 modulo xn 1. The codeword associated with c (x) is (cn1 , c0 , . . . , cn2 ), which is clearly the right cyclic shift of the codeword associated with c(x). Question 3. Show that multiplying c(x) = 1+x2 by x2 corresponds to doubly-shifting the associated codeword. We now use the language of code polynomials to characterize cyclic codes: Theorem 3.1.4. A linear code of length n over GF (q) is cyclic if and only if C satises the following two conditions: 1. If a(x) and b(x) are code polynomials in C, then a(x) b(x) C 2. If a(x) is a code polynomial in C and r(x) is any polynomial of degree less than n, then r(x)a(x) C Proof. Suppose C is a cyclic code of length n over GF (q). Then C is linear, so Condition (1) holds. Now suppose that a(x) C and r(x) = r0 + r1 x + . . . rn1 xn1 , where ri GF (q). As discussed above, multiplication of a code polynomial by x corresponds to a cyclic shift of the corresponding codeword. Since C is a cyclic code, it contains the cyclic shifts of all codewords, so xa(x) C. Similarly, xi c(x), for 0 < i < n, is the ith cyclic shift of c(x), so xi c(x) C. Now, by the linearity of C, r(x)a(x) = r0 a(x) + r1 xa(x) + + rn1 xn1 a(x) is also in C since each summand is in C. Therefore, Condition (2) also holds. Hence, if C is a cyclic code, then Conditions (1) and (2) hold. On the other hand, suppose that Conditions (1) and (2) hold. If we take r(x) to be a scalar in GF (q), the conditions imply that C is a linear code. Then, if we take r(x) = x, Condition (2) implies that C is a cyclic code. Hence, if Conditions (1) and (2) hold, then C is a cyclic code. We can use Gap to check if a given code C is cyclic using the IsCyclicCode(C); command.
3.2
Rings and Ideals
Above, in working with cyclic codes of length n, we used code polynomials of degree less than n and polynomial multiplications modulo xn 1. This set of elements (code polynomials of degree less than n) and this operation (polynomial multiplication modulo xn 1) forms what is known as a ring. Rings are mathematical structures that are slightly less restrictive than elds. The only dierence between elds and the type of rings that we will consider in this course is that our rings do not require the existence of multiplicative inverses. The other 8 criteria from the Denition 2.2.1 of elds apply. For completeness, we give the formal denition of a ring: Denition 3.2.1. Let R be a nonempty set that is closed under two binary operations denoted + and *. Then (R, +, ) is a ring if: 1. There exists an additive identity denoted 0 in R such that a + 0 = 0 + a = a for any element a R. 2. The addition is commutative: a + b = b + a, for all a, b R. 3. The addition is associative: (a + b) + c = a + (b + c), for all a, b, c R. 40
4. There exists an additive inverse for each element: For each a R, there exists an element denoted a in R such that a + (a) = 0. 5. The multiplication is associative: a (b c) = (a b) c, for all a, b, c R. 6. The operations distribute: a (b + c) = a b + a c and (b + c) a = b a + c a, for all a, b, c R. If a ring R has the additional property that the multiplication is commutative, then R is called a commutative ring. If R has the additional property of including a multiplicative identity, then R is called a ring with unity. The rings of interest to us are commutative rings with unity. In the event that a commutative ring R with unity includes multiplicative inverses for all of its nonzero elements, then R is a eld. Example 3.2.2. The integers Z under ordinary addition and multiplication form a commutative ring with unity, but you will recall that they do not form a eld. Example 3.2.3. The set Zn of integers modulo n under addition and multiplication is a commutative ring with unity, but you will recall that Zn is only a eld in the case that n is a prime. Example 3.2.4. The set 2Z of even integers under ordinary addition and multiplication is a commutative ring without unity. Example 3.2.5. The set GF (2)[x] of all polynomials in the variable x with coecients from the binary eld GF (2) under ordinary polynomial addition and multiplication is a commutative ring with unity polynomial e(x) = 1. Example 3.2.6. The set GF (2)[x]/(xn 1), which can be informally dened as the set of all polynomials of degree less than n 1 in the variable x with coecients from the binary eld GF (2) under polynomial addition and multiplication modulo xn 1, is a commutative ring with unity polynomial e(x) = 1. Example 3.2.6 is the most important ring in coding theory, and we have already worked with the elements of this ring in the context of code polynomials. We will soon understand the mathematics behind the notation GF (2)[x]/(xn 1), as we delve deeper into the connections between cyclic codes and and the theory of rings and ideals. Denition 3.2.7. A nonempty subset A of a ring R is an ideal of R if a b A whenever a, b A and ra, ar A whenever a A and r R. When R is commutative, ar = ra, hence we need only check that ra A. We say that an ideal A has closure since a + b A whenever a, b A. We say that an ideal A absorbs elements from R, since ra, ar A whenever a A and r R. Example 3.2.8. For any ring R, {0} and R are ideals of R. This is trivial to check. Example 3.2.9. For any positive integer n, the set nZ = {0, n, 2n . . .} is an ideal of the ring of integers Z. To see this, rst notice that for any a, b nZ, we have a = a n and b = b n for some integers a and b . Then, a b = a n b n = n(a b ), which is clearly in nZ. Next, notice that for any r Z, we have ar = a nr = n(a r), which is also clearly in nZ. By the denition of ideal (and since Z is commutative), we have shown that nZ is an ideal in Z. Denition 3.2.10. Let R be a commutative ring with unity and let g R. The set g = {rg|r R} is an ideal of R called the principal ideal generated by g. The element g is called the generator of the principal ideal. So, A is a principal ideal of a commutative ring R with unity if there exists g A such that every element a A can be written as rg for some r R. 41
Example 3.2.11. The ideal nZ of the integers from Example 3.2.9 is clearly a principle ideal with generator n. Example 3.2.12. Let 3 = {3r|r Z36 }. We will prove that 3 is an ideal of the ring Z36 . To do this, we must show that (1) a b 3 for all a, b 3 , and (2) ar 3 whenever a 3 and r Z36 . (Since Z36 is commutative, we need not separately address ra Z36 .) To prove (1), consider two arbitrary elements a, b 3 . These elements are of the form a = 3a and b = 3b , for some a , b Z36 , by the denition of 3 . Then a b = 3a 3b = 3(a b ), where a b is some other element in Z36 , since a , b Z36 and therefore b Z36 . So, again by the denition of 3 , a b Z36 . To prove (2), consider two arbitrary elements a 3 and r Z36 . Then ar = (3a )r = 3(a r), where a r Z36 since a , r Z36 . So, again by the denition of 3 , ar Z36 whenever a 3 and r Z36 . Therefore, 3 is an ideal of the ring Z36 . Question 4. Show that x + 1 is an ideal in GF (2)[x], the ring of polynomials with binary coecients. To do this, you must show that (1) a(x) b(x) x + 1 for all a(x), b(x) x + 1 , and (2) a(x)r(x) x + 1 whenever a(x) x + 1 and r(x) GF (2)[x]. In fact, for any q-ary polynomial p(x), p(x) is an ideal in GF (q)[x]. Theorem 3.2.13. Let R be a ring and A be an ideal of R. Then the set of cosets R/A = {r + A|r R} is a ring under the operations (s + A) + (t + A) = s + t + A and (s + A)(t + A) = st + A. The operations s + t and st are done according to the operation of the ring R. Example 3.2.14. We have shown that 2Z is an ideal of the ring Z, so we can consider the ring of cosets Z/2Z = Z/ 2 . Elements of this ring are cosets of the form r + 2Z where r Z. For example, elements include 0 + 2Z, 1 + 2Z, 23 + 2Z, and 45 + 2Z. However, we will see that there are only two distinct cosets in the ring. First, consider the coset 23 + 2Z. This represents the set {23 + 2n|n Z}. Since 23 = 1 + 2(11), we can write this set as {1 + 2(11) + 2n|n Z} = {1 + 2(11 + n)|n Z} = {1 + 2m|m Z}. This latter representation clearly represents the coset 1 + 2Z. In fact, any coset of the form r + 2Z where r is an odd integer is equivalent to the coset 1 + 2Z. Similarly, any coset of the form r + 2Z where r is an even integer is equivalent to the coset 0 + 2Z. Now, consider adding two specic elements in this ring of cosets: 23 + 2Z and 45 + 2Z. Adding together these elements gives 22 + 2Z, which represents the set {22 + 2n|n Z} = {0 + 2(11 + n)|n Z}= {0 + 2m|m Z}, which is the coset 0 + 2Z. So, (23 + 2Z) + (45 + 2Z) = 0 + 2Z. Notice then that adding elements of the ring of cosets Z/2Z is essentially the same thing as adding elements of Z modulo 2 (since 23 + (45) 0 (mod 2)). The above discussion suggests the true fact that Z/2Z is regarded as isomorphic to, or essentially the same mathematical structure as, Z2 . Question 5. Compute (24 + 2Z)(4 + 2Z) inside the ring of cosets Z/2Z. Example 3.2.15. We showed above that 3 = {3r|r Z36 } is an ideal of the ring Z36 . We can therefore consider the ring of cosets Z36 / 3 = {r + 3 |r Z36 }. The elements in this ring of cosets have the form r + 3 , where r Z36 . For example, two specic elements are 12 + 3 and 28 + 3 . Adding these elements together gives 4 + 3 , since 12 + 28 4 (mod 36). However, lets look more closely at 4 + 3 . This element represents the set {4 + 3n|n Z36 }, which can be rewritten as {1 + 3 + 3n|n Z36 } = {1 + 3m|m Z36 }. This represents the coset 1 + 3 . So, (12 + 3 ) + (28 + 3 ) = 1 + 3 . Notice then that adding elements in the ring Z36 / 3 is essentially the same thing as adding elements of Z36 under addition modulo 3. In fact, the only three distinct cosets in Z36 / 3 are 0 + 3 , 1 + 3 , and 2 + 3 , and this ring is isomorphic to Z3 . Question 6. Compute (10 + 3 )(5 + 3 ) inside the ring of cosets Z36 / 3 . Example 3.2.16. Consider the ring of cosets GF (2)[x]/(x3 1). The elements of this ring have the form r(x) + x3 1 , where r(x) GF (2)[x]. Two specic elements are 1 + x2 + x3 1 and x + x3 1 . Multiplying these elements gives (1 + x2 + x3 1 )(x + x3 1) = x + x3 + x3 1 , 42
which represents the set {x + x3 + (x3 1)g(x)|g(x) GF (2)[x]}. We can rewrite this as the set {x + 1 + (x3 1) + (x3 1)g(x)|g(x) GF (2)[x]} = {x + 1 + (x3 1)g (x)|g (x) GF (2)[x]}, which represents the coset 1 + x + x3 1 . Comparing this with Example 3.1.3, we see that multiplying two elements in the ring of cosets GF (2)[x]/(x3 1) is essentially equivalent to multiplying two polynomials modulo x3 1. Question 7. Use multiplication in the ring of cosets GF (2)[x]/(x3 1) to compute the product (x + x2 + x3 1 )(x + x3 1 ), and then compare your result to the discussion in Example 3.1.3. Example 3.2.17. We will now look at the more general ring of cosets dened by GF (q)[x]/(xn 1) = {f (x) + xn 1 |f (x) GF (q)[x]}. Elements of this ring of cosets have the form f (x) + xn 1 , where f (x) GF (q)[x]. Since f (x) GF (q)[x], the Division Algorithm implies that we may write f (x) in the form q(x)(xn 1) + r(x), where q(x) is the quotient and r(x) is the remainder obtained when dividing f (x) by xn 1. We can therefore write elements of GF (q)[x]/(xn 1) as q(x)(xn 1) + r(x) + xn 1 , where q(x), r(x) GF (q)[x]. However, since xn 1 is an ideal in GF (q)[x], q(x)(xn 1) is an element of xn 1 , by the absorption property of ideals. This allows a simplication to obtain GF (q)[x]/(xn 1) = {r(x) + xn 1 |r(x) GF (q)[x]}. By the Division Algorithm, r(x) has degree less than n, and we can nally write GF (q)[x]/(xn 1) = {c0 + c1 x + . . . + cn1 xn1 + xn 1 |ci GF (q)}. We can now understand that elements in the ring of cosets GF (q)[x]/(xn 1) have the form c0 + c1 x + . . . + cn1 xn1 + xn 1 , where ci GF (q). The absorption property of ideals implies that adding and multiplying elements of GF (q)[x]/(xn 1) is equivalent to adding and multiplying polynomials of degree less than n using modulo xn 1 polynomial arithmetic. Example 3.2.17 is fundamental to the study of cyclic codes. We showed in Example 3.2.17 that technically, the elements of the ring of cosets GF (q)[x]/(xn 1) have the form r(x) + xn 1 , where r(x) is a polynomial of degree less than n. However, we will abuse notation and say that elements of GF (q)[x]/(xn 1) are simply polynomials r(x) of degree less than n and understand that we are working modulo xn 1.
3.3
Ideals and Cyclic Codes
The following theorem motivates our study of ideals: Theorem 3.3.1. Cyclic codes of length n over GF (q) correspond precisely to the ideals in the ring GF (q)[x]/(xn 1). Proof. Suppose C is a cyclic code of length n over GF (q). Then, the corresponding set of code polynomials I(C) is contained in GF (q)[x]/(xn 1) since the polynomials are of degree less than n over GF (q). By Denition 3.2.7, to show that I(C) forms an ideal in GF (q)[x]/(xn 1), we must show that (1) c(x) d(x) I(C) for any code polynomials c(x) and d(x) in I(C); and (2) r(x)c(x) I(C) for any polynomial r(x) GF (q)[x]/(xn 1) and c(x) I(C). By Theorem 3.1.4, every cyclic code C does indeed satisfy these two conditions. On the other hand, suppose that I is an ideal in GF (q)[x]/(xn 1). Then its elements are polynomials of degree less than n, and by Denition 3.2.7, its elements satisfy (1) a(x) b(x) I whenever a(x), b(x) I; and (2) r(x)a(x) I whenever r(x) GF (q)[x]/(xn 1) and a(x) I. Theorem 3.1.4 then shows that these conditions imply that the set of polynomials I represents the code polynomials of a cyclic code. Therefore, cyclic codes of length n over GF (q) are precisely the ideals in the ring GF (q)[x]/(xn 1). Example 3.3.2. In Example 3.2.16, we considered the rings of cosets GF (2)[x]/(x3 1). A trivial example of an ideal in this ring is the entire ring itself. The elements in this ideal can be viewed 43
as all binary polynomials of degree less than 3: 0, 1, x, x2 , x + 1, x2 + 1, x2 + x, x2 + x + 1, }. These polynomials correspond respectively to the binary codewords {000, 001, 010, 100, 011, 101, 110, 111}, which clearly form a trivial example of a binary cyclic code. In order to study nontrivial cyclic codes, we will have to study nontrivial ideals of the ring GF (q)[x]/(xn 1). The next theorem will help us nd these: Theorem 3.3.3. Let C be an [n, k] cyclic code corresponding to an ideal I(C) in GF (q)[x]/(xn 1). The following statements are true: 1. There exists a unique monic polynomial g(x) I(C) of minimal degree r < n. (A monic polynomial has a coecient of 1 on its highest power term.) This polynomial is called the generator polynomial of C. 2. I(C) is a principal ideal with generator g(x), so that every code polynomial c(x) can be expressed uniquely as c(x) = m(x)g(x), where g(x) is the generator polynomial and m(x) is a polynomial of degree less than (n r) in GF (q)[x]. 3. The generator polynomial g(x) divides xn 1 in GF (q)[x]. Proof. 1. Since there is at least one non-empty ideal I(C) of Rn = GF (q)[x]/(xn 1) (why?), and since the degree of polynomials is bounded below by 0, there is certainly at least one polynomial of minimum degree in I(C). Since the scalar multiplication of any polynomial in an ideal I(C) remains in I(C), we can nd a monic polynomial of minimum degree in I(C); call it g(x) of degree r < n. Suppose that h(x) is another monic polynomial of minimum degree d in I(C). Since I(C) is an ideal, h(x) g(x) I(C). Since these polynomials have the same degree and are monic, their dierence must be of lower degree. However, this contradicts the minimality of r. Therefore, there cannot be two monic polynomials of minimum degree in I(C). 2. We must show that every element in I(C) can be written as a multiple of g(x), where g(x) is the unique monic polynomial of minimum degree in I(C). Consider f (x) an arbitrary element of I(C), so that f (x) is represented by a polynomial of degree less than n. By the Division Algorithm in GF (q)[x] (and since f (x) must have degree larger than the degree of g(x)), we can write f (x) = q(x)g(x) + r(x), where q(x), r(x) GF (q)[x] and deg r(x) < deg g(x). Since f (x) has degree less than n, clearly q(x) and r(x) (and g(x)) must also have degree less than n. Then, by Theorem 3.1.4, since I(C) corresponds to a cyclic code and since g(x) is a code polynomial, q(x)g(x) I(C). Since f (x) is also in I(C), by the linearity of the code (or by the closure property of ideals), f (x)q(x)g(x) = r(x) is also in I(C). However, deg r(x) < deg g(x) contradicts the minimality of the degree of g(x) in I(C). Therefore, r(x) must be equivalent to 0, implying that f (x) is indeed a multiple of g(x). 3. Suppose that the unique monic polynomial of minimum degree in I(C) does not divide xn 1 in GF (q)[x]. By the Division Algorithm in GF (q)[x], we can write xn 1 = q(x)g(x)+r(x), where q(x), r(x) GF (q)[x] and deg r(x) < deg g(x). Since rearranging gives r(x) = xn 1q(x)g(x) in GF (q)[x], we see that in GF (q)[x]/xn 1, r(x) is congruent to q(x)g(x). Since g(x) I(C) and q(x) GF (q)[x], we have q(x)g(x) I(C), by the absorption property of ideals. Therefore, r(x) must also be in I(C). However, by the Division Algorithm, deg r(x) < deg g(x), which contradicts the minimality of the degree of g(x) in I(C). Therefore, r(x) must be 0, implying that g(x) indeed divides xn 1.
Theorem 3.3.3 allows us to think of ideals and cyclic codes as equivalent and allows us to talk about the unique generator polynomial for a cyclic code. Although each cyclic code contains a unique monic 44
generating polynomial of minimal degree, it may also contain other polynomials that generate the code. These other polynomials are either not monic, not of minimal degree, or not divisors of the appropriate xn 1. Example 3.3.4. Suppose we would like to generate a binary cyclic code C of length 4. According to Theorem 3.3.1, this code must correspond to an ideal I(C) in the ring GF (2)[x]/(x4 1). Then, according to Theorem 3.3.3, the ideal I(C) must be generated by a divisor of x4 1 in GF (2)[x]. Using Gap or paper and pencil, we nd that g(x) = 1 + x is a divisor of x4 1 in GF (2)[x]. Then, the ideal generated by g(x) consists of the elements {0, 1 + x, 1 + x2 , x + x2 , 1 + x3 , x + x3 , x2 + x3 , 1 + x + x2 + x3 }. (Check this!). These elements correspond respectively to the binary code polynomials {0000, 0011, 0101, 0110, 1001, 1010, 1100, 1111}. Inspection shows that these code polynomials indeed form a binary cyclic code of length 4. Question 8. Show that the ideal 1 + x3 in GF (2)[x]/(x4 1) generates the same binary cyclic code of length 4 as generated by 1 + x above. Explain why this does not contradict Theorem 3.3.3. Since the ideals corresponding to cyclic codes are generated by divisors of xn 1, there are precisely as many q-ary cyclic codes of length n as there are divisors of xn 1 in GF (q)[x]/(xn 1). More formally: Theorem 3.3.5. There is a one-to-one correspondence between monic divisors of xn 1 of degree r in GF (q)[x] and q-ary cyclic codes of length n and dimension n r. Proof. By Theorems 3.3.1 and 3.3.3, any q-ary [n, k] cyclic code C is an ideal in GF (q)[x]/(xn 1) with a unique monic generating polynomial g(x) that divides xn 1. Hence, to any q-ary [n, k] cyclic code, we can associate a unique monic divisor of xn 1. On the other hand, suppose g(x) is a monic divisor of xn 1 in GF (q)[x]. Consider the code C = g(x) . Clearly, the code polynomial g(x) generates C. Suppose for contradiction that h(x) is actually the generator polynomial for C, so that h(x) is a monic polynomial of minimum degree r in C. Since h(x) C = g(x) , h(x) must be a polynomial multiple of g(x) inside GF (q)[x]/(xn 1), in other words, h(x) = m(x)g(x) inside GF (q)[x]/xn 1. So, h(x) = m(x)g(x) + s(x)(xn 1) inside GF (q)[x]. But, since g(x) is a divisor of xn 1, we can write g(x)t(x) = xn 1, for some t(x) GF (q)[x]. So, h(x) = g(x)m(x) + s(x)t(x)g(x). This implies h(x) = g(x)(m(x) + s(x)t(x)), or simplied, that h(x) = g(x)a(x), where a(x) GF (q)[x]. This leaves two cases: Either the degree of g(x) is less than the degree of h(x), or these polynomials are scalar multiples of each other. In the rst case, we arrive at a contradiction, since h(x) was assumed to be of minimum degree in C. In the second case, since both polynomials were assumed to be monic, they must be equal. In either case, we can conclude that g(x) is in fact the unique monic generator polynomial of minimum degree for C.
Theorem 3.3.5 facilitates listing all of the cyclic codes of a given length, n. If we can factor xn 1 over the appropriate alphabet, then the resulting factors (and combinations thereof) are equivalent to the generator polynomials for all cyclic codes of length n. To nd an [n, k] cyclic code, we need a factor of xn 1 of degree n k. Unfortunately, as n becomes larger than 3, factoring xn 1 over arbitrary alphabets (or even just over GF (2)), becomes increasingly more dicult. Fortunately, in this course, the factorizations of xn 1 will be provided or can be obtained using software. Gap can factor xn 1 with over dierent elds. For example, we can factor x1 0 1 over GF(2). gap> x := Indeterminate(GF(2), "x");; gap> Display(Factors(x^7-1)); [ x+Z(2)^0, x^3+x+Z(2)^0, x^3+x^2+Z(2)^0 ] 45
Gap displays the factors as powers of Z(n), which represents an element of order n 1 inside Zn . If n is prime, we can use the function Int to gure out what Z(n) is equal to. For example, gap> Int(Z(2)); 1 shows that the above factors are x + 1, x3 + x + 1, and x3 + x2 + 1. Question 9. Use Gap to factor x7 1 over GF (2). Question 10. Suppose that g(x) = 1 + x + x2 + x4 is the generator matrix for a length 7 binary cyclic code. What is the dimension of this code? Use Gap to conrm that g(x) indeed divides x7 1. Example 3.3.6. In order to nd all of the binary cyclic codes of length 3, we must factor x3 1 into irreducible polynomials over GF (2). In other words, we must factor x3 1 into polynomials that can no longer be factored. It can be shown using Gap (or higher-level mathematics) that the irreducible factors of x3 1 are (x + 1) and (x2 + x + 1). Therefore, there are four possible generator polynomials: g0 (x) = 1, g1 (x) = x + 1, g2 (x) = x2 + x + 1, and g3 (x) = x3 + 1. To obtain the length 3 code generated by the degree 1 polynomial g1 (x) = x + 1, we must multiply g1 (x) by every message polynomial corresponding to messages of length 2. We see 0 g1 (x) = 0, 1 g1 (x) = x + 1, x g1 (x) = x2 + x, and (x + 1) g1 (x) = x2 + 1. Therefore, g1 (x) = x + 1 generates the code {0, 1 + x, x + x2 , 1 + x2 } = {000, 110, 011, 101}. Similarly, g0 (x) = 1 generates the code which is equal to all of GF (2)/(x3 1) or all binary 3-tuples; g2 (x) = x2 + x + 1 generates the code {0, 1 + x + x2 } = {000, 111}; and g3 (x) = x3 + 1 generates the code {0} = {000}. You can verify that the code C = {0, 1 + x, x + x2 , 1 + x2 } generated by g1 (x) = x + 1 in the above example is also generated by x2 + 1. However, g1 (x) is the unique monic generator polynomial of minimal degree. Example 3.3.7. Use Gap to nd all possible generator polynomials for binary cyclic codes of length 5. We can use Gap to generate cyclic codes given a generator polynomial. To declare a codeword polynomial, or in particular a generator polynomial, we must rst declare the eld in which the coecients lie. For example, suppose we want a binary code of length 7 generated by generator polynomial 1 + x + x2 : gap> x := Indeterminate(GF(2), "x");; gap> gx := 1+x+x^2;; gap> C := GeneratorPolCode(gx, 7, GF(2)); a cyclic [7,7,1]0 code defined by generator polynomial over GF(2) Notice that since this code has minimum distance 1, it can correct 0 errors. Question 11. Describe the set of codewords in the above code - how many codewords are there?
3.4
Generator and Parity Check Polynomials
In Example 3.3.6, we saw that the code C = {0, 1 + x, x + x2 , 1 + x2 } is generated by g1 (x) = x + 1. We can now expand upon this result and rewrite C as {0, g1 (x), xg1 (x), g1 (x) + xg1 (x)}. This shows that g1 (x) = 1 + x and xg1 (x) = x + x2 form a basis for C; in other words, all codewords in C 46
are linear combinations of g1 (x) and xg1 (x). This implies that the corresponding vectors (110) and (011) can be used to form a generating matrix for C (recall Section 2.3). In particular, a generating matrix for C has the form: 110 011 where the rst row is the vector corresponding to the generator polynomial g1 (x), and the second row is the vector corresponding to its cyclic shift xg1 (x). We next show that we can always use the generator polynomial to dene the generator matrix of a cyclic code in this way. Theorem 3.4.1. Suppose C is a cyclic code with generator polynomial g(x) = g0 + g1 x + + gr xr of degree r. Then the dimension of C is nr, and a generator matrix for C is the following (nr)n matrix: g0 0 G= 0 . . . 0 g1 g0 0 . . . 0 g1 g0 .. . gr g1 .. . 0 0 gr .. . g0 0 0 gr g1 .. . . ... .. 0 0 . . .
0 gr
Proof. First, note that g0 is nonzero: Otherwise, (0, g1 , . . . , gr1 ) C which implies that (g1 , . . ., gr1 , 0) C which implies that g1 + g2 x + + gr1 xr1 C, which contradicts the minimality of the degree r of the generating polynomial. Now, we see that the n r rows of the matrix G are linearly independent because of the echelon of nonzero g0 s with 0s below. These n r rows represent the code polynomials g(x), xg(x), x2 g(x), . . ., xnr1 g(x). In order to show that G is a generator matrix for C we must show that every code polynomial in C can be expressed as a linear combination of g(x), xg(x), x2 g(x), . . ., xnr1 g(x). Part 2 of Theorem 3.3.3 shows that if c(x) is a code polynomial in C, then c(x) = m(x)g(x) for some polynomial m(x) of degree less than n r in GF (q)[x]. Hence, c(x) = m(x)g(x) = (m0 + m1 x + + mnr1 xnr1 )g(x) = m0 g(x) + m1 xg(x) + + mnr1 xnr1 g(x) which shows that any code polynomial c(x) in C can be written as a linear combination of the code polynomials represented by the n r independent rows of G. We conclude that G is a generator matrix for C and the dimension of C is n r. Example 3.4.2. In order to construct a binary cyclic code C of length n = 7, we need to nd a generator polynomial that is a factor of x7 1 in GF (2)[x]. Choose g(x) = (1 + x + x3 )(1 + x) = 1 + x2 + x3 + x4 . Since g(x) has degree r = 4, it follows that C has dimension k = n r = 3. So our generator matrix G will be a 3 7 matrix: 1011100 0 1 0 1 1 1 0 . 0010111
We now have an easy way to write down a generator matrix for a cyclic code. As with linear codes, we can encode a cyclic code by performing a matrix multiplication. However, Part (2) of Theorem 3.3.3 suggests a dierent encoding scheme for cyclic codes. We can use the generating polynomial g(x) to encode a message a = (a0 , . . . , ak1 ) = a(x) by performing the polynomial multiplication a(x)g(x). This simple polynomial multiplication can be used instead of storing and using an entire generator matrix for the cyclic code. 47
Question 12. Let g(x) = 1 + x2 + x3 be the generator polynomial for a binary cyclic code of length 7. Encode the message (1, 0, 0, 1) using the generator polynomial. You may have noticed that the above way of obtaining a generator matrix of a cyclic code gives a matrix that is not in standard form. Therefore, we cannot automatically write down the parity check matrix, as we can when we have a generator matrix in standard form. However, we next dene parity check polynomials that can be used to easily construct parity check matrices. Denition 3.4.3. The parity check polynomial h(x) for an [n, k] cyclic code C is the polynomial such that g(x)h(x) = xn 1, where g(x) is the degree r generator polynomial for C. Furthermore, h(x) is monic of degree k = n r. Since c(x) is a code polynomial if and only if it is a multiple of g(x), it follows that c(x) is a code polynomial if and only if c(x)h(x) 0 modulo (xn 1). Question 13. Prove the above statement. Given the generator matrix of a code, the function CheckPol returns the corresponding check polynomial. For example: gap> H := CheckPol(GeneratorMatCode([[1,1,0], [0,1,1]], GF(2))); x^2+x+Z(2)^0 You can also play around with Gap to conrm that given a code, CheckPol can also nd the check polynomial. Theorem 3.4.4. Suppose C is an [n, k] cyclic code with parity check polynomial h(x) = h0 + h1 x + + hk xk . Then, a parity check matrix for C is the following (n k) n matrix: hk hk1 h0 0 0 0 0 hk hk1 h0 0 0 . .. . . 0 hk hk1 h0 . H= 0 . . .. .. .. .. . . . . . . 0 . . 0 0 0 hk hk1 h0 The proof of this theorem uses the concept of dual codes, which are introduced in the special topics portion of this book. Proof. You proved above that if c(x) C, then c(x)h(x) 0 modulo (xn 1). The coecient of xj n1 in the product c(x)h(x) is i=0 ci hji , where the subscripts are taken modulo n. So, if c(x) C, we have
n1 i=0
ci hji = 0, for 0 j n 1.
(7)
The n equations represented in (7) are parity check equations satised by the code. The last n k of these equations imply that if c C, then cH T = 0. On the other hand, suppose that cH T = 0. Then the rows of H are vectors in C . Since h(x) is monic, H contains an echelon of 1s with 0s underneath; hence, the n k rows of H are linearly independent. Since dim C = k, we have dim C = n k, so the n k independent rows of H form a basis for C . It follows that c h = 0 for all h C . So c (C ) = C. Using Denition 2.4.1 of a parity check matrix, we have shown that H is a parity check matrix for C. 48
Example 3.4.5. The [7, 3] code C constructed in Example 3.4.2 has parity check polynomial h(x) = (x7 1)/g(x) = 1 + x2 + x3 . The following [(n k) n] = [4 7] matrix is a parity check matrix for C: 1101000 0 1 1 0 1 0 0 0 0 1 1 0 1 0 . 0001101 Question 14. What condition (equation) should the generator matrix from Example 3.4.2 and parity check matrix from Example 3.4.5 together satisfy? Check that these matrices do satisfy that equation. We dened Hamming codes Hr in Section 2.7 by constructing their parity check matrices. We now show that these codes are equivalent to cyclic codes. Theorem 3.4.6. The binary Hamming code Hr is equivalent to a cyclic code. The proof of this theorem assumes knowledge of eld theory and is provided for those students who have already taken a course in eld theory. Proof. Let p(x) be an irreducible polynomial of degree r in GF (2)[x]. Then, the ring GF (2)[x]/p(x) of polynomials modulo p(x) is actually a eld of order 2r . Since every nite eld has a primitive element (recall the Primitive Root Theorem), there exists an element GF (2)[x]/p(x) such that r r GF (2)[x]/p(x) = {0, 1, , 2 , . . . , 2 2 }. Consider the matrix H = [1, , , 2 2 ]. Since each entry of H is an element of the eld GF (2)[x]/p(x) of order 2r , we can express each entry i , 0 i 2r 1 as a binary column vector (ai,0 , ai,1 , . . . , ai,r1 )T where i = ai,0 + ai,1 x + + ai,r1 xr1 . This allows us to think of H as an r (2r 1) binary matrix. Let C be the linear code having H as its parity check matrix. Since the columns of H are exactly the 2r 1 nonzero binary vectors of length r, C is a length n = 2r 1 binary Hamming code Hr . We now show that C is cyclic. Since H is the parity check matrix for C, we have c C if and only if cH T = 0, i.e., C = {(c0 , c1 , . . . , cn1 ) V (n, 2) | c0 + c1 + + cn1 n1 = 0} = {c(x) GF (2)[x]/(xn 1) | c() = 0 in GF (2)[x]/p(x)}.
If c(x) C and r(x) GF (2)[x]/(xn 1), then r(x)c(x) C because r()c() = r() 0 = 0. It now follows from Theorem 3.1.4 that this Hamming code is cyclic.
3.5
Answers to Chapter 3 Reading Questions
Answer 1. The corresponding code polynomial is c(x) = 1 + x2 + x3 . Answer 2. For a codeword of length 5, the maximum degree of the associated code polynomial is 4. Answer 3. (1 + x2 )(x2 ) = x2 + x4 x2 + x The codeword corresponding to this polynomial looks like 011, which is two right shifts of 101, the code polynomial we started with.
3.6
Chapter 3 Problems
Problem 3.1. Suppose you are working with an arbitrary cyclic code C. Let m = (m0 , m1 , m2 ) be a message that you want to encode using the generator polynomial g(x) = g0 + g1 x. (Note that we 49
know g0 is nonzero and g1 is 1 since generator polynomials are monic, but do the problem out with the general notation.) 1. Encode the message using polynomial multiplication. 2. What are (n, k) for this code? 3. Use g(x) to produce a non-systematic generator matrix G for C. 4. Now encode the same message m using the non-systematic generator matrix G. 5. Manipulate your answers to (a) and (d) to see that they are equal. Problem 3.2. Let C be the binary cyclic code of length 9 with generator polynomial g(x) = 1 + x3 . 1. Find the parity check polynomial h(x) for C. 2. Use h(x) to conrm that 010100110 is a codeword in C. 3. Use h(x) to nd a (non-systematic) parity check matrix H for C. 4. Use H to again conrm that 010100110 is a codeword in C. 5. What message was encoded to 010100110? 6. What are (n, k) for the code C? Problem 3.3. Find a basis for the smallest binary linear cyclic code of length 7 containing the codeword 1101000. Problem 3.4. Show that the subring of rational numbers is not an ideal in the Reals. Problem 3.5. Let R be a commutative ring with unity and let a R. Prove that the set a = {ra|r R} is an ideal of R. Problem 3.6. Suppose that I is an ideal of R and that 1 I. Prove that I = R Problem 3.7. Show that every ideal in Z is a principal ideal. Problem 3.8. Consider the polynomial ring GF (2)[x]/(xn 1). Let f (x), g(x) be two xed polynomials in GF (2)[x]/(xn 1). Consider the subset generated by these two xed polynomials: f (x), g(x) = {a(x)f (x) + b(x)g(x) | a(x), b(x) GF (2)[x]/(xn 1)}. Using the denition of an ideal, prove that this subset is an ideal of GF (2)[x]/(xn 1). Problem 3.9. Gap: List all of the distinct ideals in GF (2)[x]/ (x7 1) by listing their generators. Problem 3.10. Gap: List by dimension all of the binary cyclic codes of length 31. Do the same for length 63 and 19.
4
Classes of Powerful Cyclic Codes
This Chapter uses more advanced abstract algebra to develop some of the most powerful error-control codes: BCH and Reed-Solomon (RS) codes. Binary BCH codes were introduced by R. C. Bose and D. K. Ray-Chaudhuri (1960) and independently by A. Hocquenghem (1959). Gorenstein and Zierler later extended the concepts of BCH codes to arbitrary nite elds (1961). Reed-Solomon (RS) codes were rst introduced by Irving Reed and Gustave Solomon in a ve-page paper, Polynomial Codes over Certain Finite Fields, in 1960 [5], and they were given their present name by W. Wesley Peterson in 1961 [4]. Reed-Solomon codes are the most used codes today, due to their use in compact disc digital audio systems. They are also well known for their role in providing pictures of Saturn and Neptune during space missions such as the Voyager. In fact, RS codes are incorporated in the NASA Standard. These and several other applications of RS codes are described in [9]. 50
4.1
Special Cases of BCH and RS Codes
In this section, we provide an introduction to the powerful classes of BCH and RS codes by focusing on restricted cases. The following sections develop the mathematical theory needed for the general theory of BCH and RS codes. Denition 4.1.1. The multiplicative order of a non-zero eld element is dened as the smallest positive integer r such that r 1. Example 4.1.2. In the eld Z7 , the element 3 has order 6, since 36 1, but 3x is not equivalent to 1 for any x < 6. (Conrm this by computing powers of 3 modulo 7) Question 1. Find the order of 2 inside Z7 . We can also use Gap to nd the order of a eld element. For example, to nd the multiplicative order of 3 inside Z7 , we type: gap> OrderMod(3, 7); 6 OrderMod(<n>, <m>) returns the multiplicative order of the integer n modulo m when n and m are relatively prime. Otherwise, the order is not dened and the command outputs 0. Question 2. Use Gap to nd the order of 2 inside Z7 . Theorem 4.1.3. (The Modied BCH Bound ) In order to build a p-ary cyclic code C of length p 1, where p is prime, begin by nding an element of order p 1 in Zp . Next, dene a generator polynomial g(x) for C as (x b )(x b+1 ) (x b+2 ) for some integers b 0 and 1. Hence, g(x) contains 1 consecutive powers of as zeros or roots. Then, the cyclic code C dened by this generator polynomial g(x) has minimum distance dmin . The original BCH Bound, used to dene general BCH codes, does not restrict the alphabet to be a prime eld and does not restrict the length of the code to be one less than the size of the alphabet. Reed-Solomon (RS) codes are q m -ary BCH codes of length q m 1. Our Modied BCH Bound will allow us to construct RS codes in the special case where the alphabet is a prime eld. Example 4.1.4. Lets nd a generator polynomial for a 7-ary Reed-Solomon code of length 6 and minimum distance at least 4. First, we need to nd an element Z7 with order 6. Example 4.1.2 shows that 3 satises our requirement. Since we need a minimum distance of 4, we need to build our generator polynomial so that it has three consecutive powers of 3 as its roots. We choose to dene g(x) = (x 3)(x 6)(x 2) (since 32 6 and 33 2). Multiplying out g(x) over Z gives g(x) = x3 + 3x2 + x + 6. This polynomial is used to generate a (6, 4, 4) Reed-Solomon code over Z7 . Since the degree of g(x) is 3 and the codelength is 6, the dimension is 6 3 = 3. When using our Modied BCH Bound, we will always be in search of elements of order p 1 inside Zp , where p is prime. In mathematics, these elements are known as primitive roots, and the Gap command PrimitiveRootMod(p) nds these primitive roots for us. For example, gap> PrimitiveRootMod(7); 3 Although there are often many primitive roots for a given prime, this Gap command will return the smallest such primitive root. We can also use Gap to automate the construction of the generator matrix for a RS code. To illustrate, we will redo Example 4.1.4 using Gap: 51
gap> x := Indeterminate(GF(7), "x");; gap> gx := (x-3^1)*(x-3^2)*(x-3^3);; gap> C := GeneratorPolCode(gx, 6, GF(7)); a cyclic [6,3,1..4]2..3 code defined by generator polynomial over GF(7) The rst command denes the variable x. The second command denes the generator polynomial gx using three consecutive powers of 3. The third command tells Gap to generate a cyclic code of length 6 over GF (7) using generator polynomial gx. The output conrms that we have generated a cyclic length 6 code of dimension 3. The minimum distance is between 1 and 4, but can determine the exact minimum distance via the following input: gap> MinimumDistance(C); 4 Guava also has a built-in function for generating Reed-Solomon Codes. ReedSolomonCode(<n>, <d>) produces a length n RS code over GF (n + 1) with minimum distance d. Question 3. Generate and compare the generator matrices for the above code C and the Gapgenerated RS code of the same parameters. It is quite simple to increase the minimum distance of a RS code. In order to increase the minimum distance of the RS code in Example 4.1.4 from 4 to 5, we need to include 2 more factors in the generator matrix. Question 4. Write down a generator polynomial for a 7-ary RS code of length 6 and minimum distance 5. Since the RS code in Example 4.1.4 has dimension k = n r = 6 3 = 3,, this 7-ary code has 37 = 2187 codewords. Question 5. How many codewords does the RS code constructed in Question 4 have? The preceding examples, comments, and questions highlight the trade-o that exists between minimum distance and total number of codewords. It should also be intuitively clear that by including fewer codewords, it is possible to increase the minimum distance between the codewords. Striking the right balance on this scale is of utmost importance in practical implementations. Theorem 4.1.5. Reed-Solomon codes are maximum-distance separable (MDS); hence the parameters of an [n, k, d] RS code satisfy the equation d = n k + 1. Proof. By the Singleton bound given in Theorem 2.6.2, we know that d (n k + 1). Since C is an [n, k] code, its generator polynomial has (n k) roots. Since RS codes of designed distance have generator polynomials with ( 1) roots, we conclude that (n k) = ( 1). So, = n k + 1, and by the BCH bound, we have d = n k + 1. Now, we can combine both inequalities to see that d = n k + 1. Theorem 4.1.5 shows that RS codes satisfy the Singleton bound of Theorem 2.6.2 with equality. This means that given their length and dimension, RS codes are optimal in their distance properties. One notable property of MDS codes is that if a code is MDS, then so is its dual code, as dened in Section 5.1. We can use the Gap IsMDSCode command to conrm that RS codes are MDS. 52
One of the advantages of BCH and RS codes is the ability to build codes with huge numbers of codewords. The (Modied) BCH Bound provides a very simple way to construct codes with optimal distance properties and many codewords. Importantly, there exist ecient decoding algorithms for these codes, making them very desirable in practice.
4.2
Minimal Polynomials
In order to study general BCH and RS codes, we need to learn how to nd minimal polynomials of elements in a eld. We begin with several denitions. Denition 4.2.1. Let be an element in GF (q m ). The minimal polynomial of with respect to the subeld GF (q) is the monic polynomial p(x) GF (q)[x] of minimal degree such that p() = 0. The above denition depends on the phrase with respect to the subeld GF (q), and it is meaningless without specifying a subeld. The monic minimal polynomial of an element with respect to a subeld GF (q) is unique: Suppose f (x) and g(x) are distinct monic minimal polynomials of GF (q m ) with respect to GF (q). Since f (x) = g(x), there exists a nonzero polynomial r(x) such that f (x) = g(x) + r(x), r() = 0, and deg r(x) < deg f (x). This contradicts the minimality of degree of the minimal polynomial. Question 6. Prove that the minimal polynomial of an element is always irreducible. Example 4.2.2. If GF (q...