{[ promptMessage ]}

Bookmark it

{[ promptMessage ]}

TrefethenLecture1

# TrefethenLecture1 - Lecture 1 Matrix-Vector Multiplication...

This preview shows pages 1–8. Sign up to view the full content.

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: Lecture 1. Matrix-Vector Multiplication You already know the formula for matrix—vector multiplication. Nevertheless, the purpose of this ﬁrst lecture is to describe a way of interpreting such prod— ucts that may be less familiar. If I) : Am, then b is a linear combination of the columns of A. Familiar Deﬁnitions Let a: be an n—dimensional column vector and let A be an m x n matrix (m rows, n columns). Then the matrix—vector product I) : A51: is an m—dimensional column vector deﬁned as follows: bZZZa/ijjj 7:: 1,...,m. (1.1) j:1 Here b, denotes the ith entry of b, (1,,- denotes the z',j entry of A (ith row, jth column), and x, denotes the jth entry of :L’. For simplicity, we assume in all but a few lectures of this book that quantities such as these belong to (D, the ﬁeld of complex numbers. The space of m—vectors is (Em, and the space of m X n matrices is Cm”. The map a: »—> A70 is linear, which means that, for any 56,3; 6 C“ and any 04 E (U, A(a: + 9) Ax + Ay, A(oaa:) : oan. 4 PART I FUNDAMENTALS Conversely, every linear map from C” to (Um can be expressed as multiplication by an m X 71 matrix. A Matrix Times a Vector Let aj denote the jth column of A, an m—vector. Then (1.1) can be rewritten b2A\$=Zmau (1.2) b = a1 a2 a. : = 1:1 (11 +902 a2 +'~+mn an In (1.2), b is expressed as a linear combination of the columns aj. Nothing but a slight change of notation has occurred in going from (1.1) to (1.2). Yet thinking of A9: in terms ofthe form (1.2) is essential for a proper understanding of the algorithms of numerical linear algebra. One way to summarize these different ways of viewing matrix—vector prod— ucts is like this. As mathematicians, we are used to viewing the formula Ax : b as a statement that A acts on a: to produce b. The formula (1.2), by contrast, suggests the interpretation that x acts on A to produce b. Example 1.1. Fix a sequence of numbers {951, . . . ,xm}. lfp and q are polyno— mials of degree < n and 04 is a scalar, then p + q and 0439 are also polynomials of degree < 71. Moreover, the values of these polynomials at the points as, satisfy the following linearity properties: (19 + (1)031) = We) + (1(a), (04p)(5’3¢) : 0419017»)- Thus the map from vectors of coeﬂicients of polynomials p of degree < n to vectors (p(m1),p(m2), . . . ,p(:1:m)) of sampled polynomial values is linear. Any linear map can be expressed as multiplication by a matrix; this is an example. In fact, it is expressed by an m x n Vandermonde matrix 2 11—1 1 1:1 1:1 m1 2 mil A— 1 1:2 {1:2 {1:2 2 n—l 1 Jim Jim mm LECTURE 1 MATRIX—VECTOR MULTIPLICATION 5 If c is the column vector of coeﬂicients of p, n—l , 13(13):co+clx+62x2+~-+cn,1x , Cn—l then the product Ac gives the sampled polynomial values. That is, for each 73 from 1 to m, we have n71 (Ac)i : c0 + 01:17) + c23312 + ' - - + 6,1111% 2 p(a:z) (1.3) In this example, it is clear that the matrix—vector product Ac need not be thought of as m distinct scalar summations, each giving a different linear combination of the entries of c, as (1.1) might suggest. Instead, A can be viewed as a matrix of columns, each giving sampled values of a monomial, A: 1 :13 51:2 {En—1 , (1.4) and the product Ac should be understood as a single vector summation in the form of (1.2) that at once gives a linear combination of these monomials, Ac : c0 + 01:17 + 02.702 + - - - + 0,111.1:“1 : p(:z:) D The remainder of this lecture will review some fundamental concepts in linear algebra from the point of View of (1.2). A Matrix Times a Matrix For the matrix—matrix product B 2 AC, each column ofB is a linear com- bination of the columns of A. To derive this fact7 we begin with the usual formula for matrix products. If A is E X m and C is m X n, then B is E X n, with entries deﬁned by k:1 Here bij, am, and CM are entries of B, A, and C, respectively. Written in terms of columns, the product is b1 b2 0‘ | | 0’1 012 ... a , 6 PART I FUNDAMENTALS and (1.5) becomes bj = ch = Z ckjak. (1.6) 19:1 Thus bj is a linear combination of the columns ak with coefﬁcients ckj. Example 1.2. A simple example of a matrix—matrix product is the outer product. This is the product of an m—dimensional column vector u with an n—dimensional row vector u; the result is an m X n matrix of rank 1. The outer product can be written 711 v2 u Ulul “#1 u = ulu 112uuu ulu u u 7Tb 71m The columns are all multiples of the same vector u, and similarly, the rows are all multiples of the same vector u. Example 1.3. As a second illustration, consider B : AR, where R is the upper—triangular n x n matrix with entries rij : 1 for 2' g j and rij : 0 for 2' > j. This product can be written 1 1 b on = a an 1 1 1 The column formula (1.6) now gives 1 b] : ATj : Zak. (1.7) k:1 That is, the jth column of B is the sum of the ﬁrst 3’ columns of A. The matrix R is a discrete analogue of an indeﬁnite integral operator. Range and Nullspace The range of a matrix A, written range(A), is the set of vectors that can be expressed as A30 for some 1:. The formula (1.2) leads naturally to the following characterization of range(A): Theorem 1.1. range(A) is the space spanned by the columns of A. LECTURE 1 MATRIX—VECTOR MULTIPLICATION 7 Proof. By (1.2), any A1: is a linear combination of the columns of A. Con— versely, any vector y in the space spanned by the columns of A can be written as a linear combination of the columns, 3/ : 22;, ijaj. Forming a vector 1: out of the coefﬁcients 17,-, we have y 2 A33, and thus y is in the range of A. In view of Theorem 1.1, the range of a matrix A is also called the column space of A. The nullspace of A E Cm”, written null(A), is the set of vectors :1: that satisfy Azr: : 0, where 0 is the 0—vector in 03’”. The entries of each vector x E null(A) give the coefficients of an expansion of zero as a linear combination of columns of A: 0 = mlol + {132% + . - ' + :Irnan. Rank The column rank of a matrix is the dimension of its column space. Similarly, the row rank of a matrix is the dimension of the space spanned by its rows. Row rank always equals column rank (among other proofs, this is a corollary of the singular value decomposition, discussed in Lectures 4 and 5), so we refer to this number simply as the rank of a matrix. An m X n matrix of full rank is one that has the maximal possible rank (the lesser of m and n). This means that a matrix of full rank with m 2 n must have n linearly independent columns. Such a matrix can also be characterized by the property that the map it deﬁnes is one—to—one: Theorem 1.2. A matrix A E Cm” with in 2 n has full rank if and only if it maps no two distinct vectors to the some vector. Proof. (:>) If A is of full rank, its columns are linearly independent, so they form a basis for range(A). This means that every I) E range(A) has a unique linear expansion in terms of the columns of A, and therefore, by (1.2), every b E range(A) has a unique 33 such that b 2 A93. (<2) Conversely, if A is not of full rank, its columns a,- are dependent, and there is a nontrivial linear combination such that 2;, cjaj = 0. The nonzero vector c formed from the coefﬁcients cj satisﬁes Ac = 0. But then A maps distinct vectors to the same vector since, for any ac, A1: : A(IL‘ + c). D Inverse A nonsingviar or invertible matrix is a square matrix of full rank. Note that the rn columns of a nonsingular in X in matrix A form a basis for the whole space 03’". Therefore, we can uniquely express any vector as a linear 8 PART I FUNDAMENTALS combination of them. In particular, the canonical unit vector with 1 in the jth entry and zeros elsewhere, written ej, can be expanded: m i:1 Let Z be the matrix with entries 2,], and let zj denote the jth column of Z. Then (1.8) can be written ej : Azj. This equation has the form (1.6); it can be written again, most concisely, as The matrix Z is the inverse of A. Any square nonsingular matrix A has a unique inverse, written A‘l, that satisﬁes AA‘1 : A‘lA : I. The following theorem records a number of equivalent statements that hold when a square matrix is nonsingular. These conditions appear in linear algebra texts, and we shall not give a proof here. Concerning (f), see Lecture 5. Theorem 1.3. For A E (Dmxm, the following conditions are equivalent: (a) A has an inverse A’l, (b) rank(A) = m, (c) range(A) : (Um, a) nu11(A> = {0}, (e) 0 is not an eigenvalue of A, (f) 0 is not a singular value ofA, (9) detM) 7E 0- Concerning (9), we mention that the determinant, though a convenient notion theoretically, rarely ﬁnds a useful role in numerical algorithms. A Matrix Inverse Times a Vector When writing the product a = A‘lb, it is important not to let the inverse— matrix notation obscure what is really going on! Rather than thinking of a as the result of applying A—1 to b, we should understand it as the unique vector that satisﬁes the equation Ax = b. By (1.2), this means that a: is the vector of coefﬁcients of the unique linear expansion of b in the basis of columns of A. This point cannot be emphasized too much, so we repeat: A’lb is the vector of coefﬁcients of the expansion ofb in the basis of columns of A. LECTURE 1 MATRIX—VECTOR MULTIPLICATION 9 Multiplication by A—1 is a change of basis operation: Multiplication by A—1 A‘lb: coefﬁcients of the expansion of b in {a1,...,am} b: coefﬁcients of the expansion of b in {el,...,em} Multiplication by A In this description we are being casual with terminology, using “b” in one instance to denote an m—tuple of numbers and in another as a point in an abstract vector space. The reader should think about these matters until he or she is comfortable with the distinction. A Note on m and n Throughout numerical linear algebra, it is customary to take a rectangular matrix to have dimensions m X n. We follow this convention in this book. What if the matrix is square? The usual convention is to give it dimensions n x 71, but in this book we shall generally take the other choice7 m X m. Many of our algorithms require us to look at rectangular subrnatrices formed by taking a subset of the columns of a square matrix. If the submatrix is to be m x n, the original matrix had better be m X m. Exercises Let B be a 4 x 4 matrix to which we apply the following operations: . double column 17 . halve row 3, . add row 3 to row 1, . interchange columns 1 and 4, . subtract row 2 from each of the other rows, . replace column 4 by column 3, 7. delete column 1 (so that the column dimension is reduced by 1). [\3>—\ @Ull-DQJ (a) Write the result as a product of eight matrices. (b) Write it again as a product ABC (same B) of three matrices. Suppose masses m1, m2, m3, m4 are located at pos1tions x1, 1:2, 1:3, 1:4 in a line and connected by springs w1th spr1ng constants k12,k23,k34 whose natural 10 PART I FUNDAMENTALS lengths of extension are €12,623,€34. Let f1,f2,f3,f4 denote the rightward forces on the masses, e.g., f1 : k12(m2 — 1:1 — £12). (a) Write the 4 X 4 matrix equation relating the column vectors f and :13. Let K denote the matrix in this equation. (b) What are the dimensions of the entries of K in the physics sense (e.g., mass times time, distance divided by mass, etc)? (0) What are the dimensions of det(K)7 again in the physics sense? (01) Suppose K is given numerical values based on the units meters, kilograms, and seconds. Now the system is rewritten with a matrix K’ based on centime— ters, grams, and seconds. What is the relationship of K’ to K? What is the relationship of det(K’) to det(K )? Generalizing Example 1.3, we say that a square or rectangular matrix R with entries 7'” is upper-triangular if 7”,, : 0 for i > j. By considering what space is spanned by the ﬁrst 71 columns of R and using (1.8), show that if R is a nonsingular m X m upper-triangular matrix, then R‘1 is also upper—triangular. (The analogous result also holds for lower—triangular matrices.) Let f1,...,f8 be a set of functions deﬁned on the interval [1,8] with the property that for any numbers d1,...,d8, there exists a set of coefficients 01,... ,08 such that 8 j:1 (a) Show by appealing to the theorems of this lecture that d1, . . . , d8 determine c1, . . . , c8 uniquely. (b) Let A be the 8 X 8 matrix representing the linear mapping from data 611,. . . ,d8 to coefficients 01,... ,cS. What is the z',j entry of A’l? ...
View Full Document

{[ snackBarMessage ]}

### Page1 / 8

TrefethenLecture1 - Lecture 1 Matrix-Vector Multiplication...

This preview shows document pages 1 - 8. Sign up to view the full document.

View Full Document
Ask a homework question - tutors are online