This preview shows pages 1–3. Sign up to view the full content.
This preview has intentionally blurred sections. Sign up to view the full version.View Full Document
Unformatted text preview: Homework 2, Stat 541: Linear Algebra, Due Fri, Sept 25, 2009, 12 Noon Your Name: (replace this with your name) October 5, 2009 Instructions: Edit this LaTex file by inserting your solutions after each problem statement. Generate a PDF file from it and e-mail the PDF to the usual class gmail address. You should not just answer the questions but also give evidence or even proof. You can discuss the homework with each other in general terms, but not with previous years students of Stat 541. Also, you must write your own solutions and not copy from anyone. 1. Interpretations of matrix multiplication: Assume X is of size n p and B of size p q . (a) If X = ( x 1 , x 2 ,..., x p ), B = ( b 1 , b 2 ,..., b p ) T , express XB = X j =1 ,...,p x j b T j Interpret the summands: Size? Rank? How would you compute one in R? A: The term x j b T j is called a rank-one matrix. It is what the name says: a matrix of rank one. In fact, every rank-one matrix can be written this way. It is the outer product of x j and b j ; in R it would be computed by outer(X[,j],B[j,]) . The above formula says that every matrix can be written as a sum of rank-one matrices in an obvious way: the rank-one summands are the outer products of the colums of the first factor and the rows of the second factor. (b) If X = ( x 1 , x 2 ,..., x n ) T , B = ( b 1 , b 2 ,..., b q ), express ( XB ) i,k = x T i b k 1 Interpret these terms. What would it mean if all of them were zero? How is a term computed in R? A: The term x T i b k is the inner product of the i th row of X and the k th column of B . In R it would be computed by sum(X[i,]*B[,k] . The above formula says that a matrix product is the collection of inner products of all rows of X and all columns of B . If they all were zero, it would mean the rows of X and the columns of B are orthogonal. 2. Write the Euclidean inner product as h x , y i = x T y and the Euclidean squared norm as k x k 2 = x T x . What is the interpretation of y 7 h y , x i k x k 2 x ? A: This is the orthogonal projection of y on the direction given by x . The scalar factor written as h y , x i / k x k 2 = y i x i / x 2 i should remind you of the formula for the slope of the simple linear regression coefficient when regressing the response y on the predictor x ; in fact, this is the slope formula when there is no intercept term. On the side, note that replacing x with any non-zero multiple c x produces the same mapping ( c cancels out), hence the mapping really depends only on the direction of x , not its length or sign. How do you see that this is the orthogonal projection? You can prove that the residual vector y- h y , x i k x k 2 x is orthogonal to x (its not difficult)....
View Full Document