426/CISC Math 410 08F, All Sections R.J. Braun Homework 7 Solutions, Hints and Answers
Problem 3.2.1. Here we use the normal equation approach. AT A = 8 -6 -6 6 , AT b = -10 6 .
Solving AT Ax = AT b gives x = [-2 - 1]T . 3.2.2. We can compute all the quantities we need to get the pseudoinverse, even if there is only one column in A. Here AT A = 12 + (-2)2 + 32 = 14. Then A+ = (AT A)-1 AT = 1 [1 - 2 3] . 14
3.2.9. Let P and Q be orthogonal matrices; then for either of them, the inverse is the same as the transpose. If the product P Q is orthogonal then (P Q)T (P Q) = (P Q)(P Q)T = I. Let's try it. (P Q)T (P Q) = QT P T P Q = QT IQ = QT Q = I; we can do this if both P and Q are orthogonal. Also (P Q)(P Q)T = P QQT P T = P IP T = P P T = I. Because the order of the products doesn't matter and we get the identity matrix as a result, the product P Q of orthogonal matrices is still an orthogonal matrix. 3.2.10. (a) For an orthogonal matrix, Q-1 = QT ; that is, QQT = QT Q = I. Let P = QT , and now try the same approach as in the last problem. P P T = QT (QT )T = QT Q = I, and P T P = (QT )T QT = QQT = I; thus, QT is also an orthogonal matrix. (b) Now 2 (Q) = ||Q-1 ||2 ||Q||2 = ||QT ||2 ||Q||2 . Recall that ||Q|| = max||x||2 =1 ||Qx||2 is the definition of the 2-norm of the matrix. Then for an orthogonal matrix, ||Qx||2 = (Qx)T (Qx) = xT QT Qx = xT Ix = xT x = ||x||2 . 2 2 Because in the definition of the norm we must have ||x||2 = 1, then ||Q||2 = 1. Now ||QT ||2 = 1 because QT is also an orthogonal matrix. Then 2 (Q) = (1)(1) = 1. 3.3.4. The QR factorization can be written as follows. ^ A = QR = Q Q0 ^ R 0 .
1
^ ^ Note that A is m-by-n, Q is m-by-n and R0 is n-by-n; the others fill out the dimensions with Q0 being m-by-(m - n) and the zero matrix is (m - n)-by-n. Because of that zero matrix, ^^ we can reconstruct A with the compressed QR facorization: A = QR. Now use this in the formula for the pseudoinverse. A+ = (AT A)-1 AT ^^ ^^ ^^ = [(QR)T QR]-1 [QR]T , ^ ^ ^^ ^ ^ = [RT QT QR]-1 RT QT , ^ ^ ^ ^ = [RT R]-1 RT QT , ^ ^ ^ ^ = R-1 [RT ]-1 RT QT , ^ ^ ^ ^ = R-1 I QT = R-1 QT . 3.3.6. (b) Use the definition of the norm. ||A||2 = = = =
||x||2 =1 ||x||2 =1 ||x||2 =1 ||x||2 =1
max ||Ax||2
max ||QRx||2
max ||Q(Rx)||2 max ||Rx||2
= ||R||2 . (a) Count the number of flops in the qrfact function; use m = n. Note that there are no flops in lines 9 to 13, but there is important there: information the sizes of everything are established. For example, v and z are n - k + 1 long. Let s = n - k + 1 for convenience. Line 14: There are 3 flops before the semicolon and there are s - 1 flops (multiplication by -1 after it, for a total of s - 1 + 3 = s + 2. Line 15: This line is non-trivial to count! Start with the parentheses. The last term is v'*v, which is a dot product; the result is a scalar that takes 2s-1 flops. The other term in parentheses is v*v', which is an outer product giving an s-by-s matrix as output; this matrix takes s2 flops. (You have to write out the matrix to see this.) Now the division sign takes s2 flops, as does the multiplication by 2, and the substraction from the s-by-s identity matrix. Thus the total flops count is 4s2 + 2s - 1 for this line. Line 16: This is multiplication of two matrices of size s-by-s; there are s2 locations and each one takes a dot product requiring 2s - 1 flops, so there are s2 (2s - 1) flops required here. Line 17: Same as line 16, so s2 (2s - 1) flops here too. That's it for the flops. Now we need to total them up, for each k we have (s + 2) + (4s2 + 2s - 1) + [s2 (2s - 1)] + [s2 (2s - 1)] = 4s3 - 2s2 + 3s + 1, s = n - k + 1.
2
Now summing over the loop gives
n n
4s3 - 2s2 + 3s - 3 =
k=1 k=1 n
4(n - k + 1)3 - 2(n - k + 1)2 + 3(n - k + 1) + 1, 4n3 - 4k 3 + O(n2 ) + O(k 2 ),
k=1 n n
= 4
n3 -
k=1 k=1
k3 ,
4(n4 - n4 /4), = 3n4 . Thus,...