Coursehero >> California >> Berkeley >> STAT 206a

Course Hero has millions of student submitted documents similar to the one below including study guides, homework solutions, papers, exam answer keys and textbook solutions.

206A: STAT Gibbs Measures 10 Lecture 10 Lecture date: Sept 28 Scribe: Aleksandr Simma 1 Introduction to Linear Codes For the purposes of coding, we will be working with linear algebra on nite elds. We will n only need to work with the eld F2 , composed of the elements {0, 1}n and the operations and . n De nition 1 A linear code C is a linear subspace of F2 Claim 2 For every linear code C, there exist the parity check matrix H and the generator n matrix G which satisfy C = {x F2 : Hx = 0} and C = imageG = {Gy : y {0, 1}m } These matrices H and G allow us to characterize and study codes. However, for a particular code, H and G are not necessarily unique. The parity check matrix has a clear interpretation as an adjacency matrix of the code s factor graph: each row represents a factor node, and the value of each column in the row indicates whether the corresponding variable node is connected to the factor node or not. De nition 3 The factor graph of a parity check matrix H has n variable nodes x1 , . . . , xn and m factor nodes. Each factor node corresponds to a row of H and is connected to all the x nodes which have non-zero coe cients in the row. Each factor node (and row of H) prevents some values of x from being code words. Thus, any x which is not restricted by any of the factors is a valid code word, and so the uniform distribution factorizes into a product of indicators, one for each factor nodes. P H (x) = m 1 a=0 1(Ha X = 0) Z Claim 4 Observe that for any linear subspace, the addition of a constraint linearly independent from the previous constraints eliminates half the elements in the subspace, so it follows that |C| = 2N Rank(H) 2N M 10-1 2 Ensembles of factor graphs with a given degree distribution. n n=0 n x n n=0 Pn x Recall that (x) = Analogously, P (x) = where n is the fraction of variable nodes of degree N. where Pn is the fraction of factor nodes of degree n. One way of de ning a uniform distribution over factor nodes is by stating that De nition 5 DN ( , P ) is the uniform distribution over a graph containing N variable nodes, and the degree pro le given by ( , P ) However, this de nition raises several questions. First, does a single graph satisfying these constraints exist? How can we sample from this distribution? It turns out that it is very di cult to sample from the distribution coming from this de nition, even on regular graphs. Thus, this de nition is not very useful. An alternative de nition is the Con guration Model, which results in a di erent, but close, distribution over graphs. De nition 6 De ne a uniform distribution over random graphs with N i verteces of degree i, and M Pi factor nodes of degree i. We can sample this model with the following algorithm: 1. For each of the N i variable nodes, create i variable sub-nodes. 2. For each the of M Pi factor nodes, create i factor sub-nodes. 3. Randomly, connect each variable sub-node to a factor sub-node to produce a graph. The a diagram of the generating process is in Figure 1. Although this is not the same model as the prior, intractable de nition, the models are closely related when the degrees of the nodes are not too high. For purposes of coding, when there are multiple (m) edges between a particular variable and factor node, they are replaced by m mod 2. This produces the same code, as double-xor is always 0, but eliminates the problem of duplicate edges. Exercise 7 Suppose (1), P (1) < 0 and let m be the number of parallel edges. Show that E(m) = O(1) 10-2 M P1 of N 1 of M P2 of N 2 of ... M Pk of (w/ k sub-nodes) ... N k of (w/ k sub-nodes) Figure 1: Generating a random factor graph. The edges in the middle are random, but every small circle on the left has to be paired with a small square on the right. Exercise 8 Let be the degree distribution of the variable nodes before removing the par allel edges, and be the degree distribution after. Similarly, let P be the degree distribution of factor nodes before parallel edge removal, and P be the degree distribution after. Show that E( l | l l | + l 1 |Pl Pl |) = O( n ) De nition 9 A low density parity check code (LDPCC) is an ensemble of codes of DN ( , P ) Underlying this de nition is the assumption that we truncate , P at n(N ) such that n << n(N ) << N The design rate of the code is N M N =1 (1) P (1) Since the code is linear, the distribution of code words around a particular code word X is the same for all X. Thus, in order to study the properties of the code, it is su cient to consider X = 0. One desired property of a code is that the code words be far apart, so that drift from errors 10-3 in transmission would be lower than the spacing between code words. De nition 10 For a linear code C, the weight-enumerator function WN ( ) is the number of words of weight , where w(x) = dH (x, 0) = {i : xi = 1} For LDP CCN ( , P ), let W ( ) = E(W ( )). We write W (N ) for W (N ) exp(N ( )) (N ) meaning ( ) = log W plus lower-order terms. N w w Figure 2: The bipartite graph used for counting LDPCCs for estimating W It is not easy to analyze W but analyzing W is much harder. Computing W involves looking at an x such that W (x) = w and counting LDP CCN ( , P ) for which x is a code word. In order to do that, we construct a bipartite graph with variable nodes on the left, and factors on the right. We represent edges from xi = 1 to a factor node with red edges, and edges from xi = 0 by black edges. In that case, it is su cient to count the number of factor graphs where every factor node is adjacent to an even number of red edges. This construction can be seen in Figure 2. 10-4

Find millions of documents here - Study Guides, Homework Solutions, Papers, Exam Answer Keys and more. Course Hero has millions of course related materials that will enable you to learn better, faster and get an A in all your courses.
Below is a small sample set of documents: