This preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Unformatted text preview: Lecture 1. MatrixVector Multiplication You already know the formula for matrix—vector multiplication. Nevertheless,
the purpose of this ﬁrst lecture is to describe a way of interpreting such prod—
ucts that may be less familiar. If I) : Am, then b is a linear combination of
the columns of A. Familiar Deﬁnitions Let a: be an n—dimensional column vector and let A be an m x n matrix (m rows,
n columns). Then the matrix—vector product I) : A51: is an m—dimensional
column vector deﬁned as follows: bZZZa/ijjj 7:: 1,...,m. (1.1)
j:1 Here b, denotes the ith entry of b, (1,, denotes the z',j entry of A (ith row,
jth column), and x, denotes the jth entry of :L’. For simplicity, we assume in
all but a few lectures of this book that quantities such as these belong to (D,
the ﬁeld of complex numbers. The space of m—vectors is (Em, and the space of
m X n matrices is Cm”. The map a: »—> A70 is linear, which means that, for any 56,3; 6 C“ and any
04 E (U, A(a: + 9) Ax + Ay,
A(oaa:) : oan. 4 PART I FUNDAMENTALS Conversely, every linear map from C” to (Um can be expressed as multiplication
by an m X 71 matrix. A Matrix Times a Vector Let aj denote the jth column of A, an m—vector. Then (1.1) can be rewritten b2A$=Zmau (1.2) b = a1 a2 a. : = 1:1 (11 +902 a2 +'~+mn an In (1.2), b is expressed as a linear combination of the columns aj. Nothing
but a slight change of notation has occurred in going from (1.1) to (1.2). Yet
thinking of A9: in terms ofthe form (1.2) is essential for a proper understanding
of the algorithms of numerical linear algebra. One way to summarize these different ways of viewing matrix—vector prod—
ucts is like this. As mathematicians, we are used to viewing the formula
Ax : b as a statement that A acts on a: to produce b. The formula (1.2), by
contrast, suggests the interpretation that x acts on A to produce b. Example 1.1. Fix a sequence of numbers {951, . . . ,xm}. lfp and q are polyno—
mials of degree < n and 04 is a scalar, then p + q and 0439 are also polynomials
of degree < 71. Moreover, the values of these polynomials at the points as,
satisfy the following linearity properties: (19 + (1)031) = We) + (1(a), (04p)(5’3¢) : 0419017») Thus the map from vectors of coeﬂicients of polynomials p of degree < n to
vectors (p(m1),p(m2), . . . ,p(:1:m)) of sampled polynomial values is linear. Any
linear map can be expressed as multiplication by a matrix; this is an example.
In fact, it is expressed by an m x n Vandermonde matrix 2 11—1 1 1:1 1:1 m1
2 mil A— 1 1:2 {1:2 {1:2
2 n—l 1 Jim Jim mm LECTURE 1 MATRIX—VECTOR MULTIPLICATION 5 If c is the column vector of coeﬂicients of p, n—l , 13(13):co+clx+62x2+~+cn,1x , Cn—l
then the product Ac gives the sampled polynomial values. That is, for each 73
from 1 to m, we have n71 (Ac)i : c0 + 01:17) + c23312 + '   + 6,1111% 2 p(a:z) (1.3) In this example, it is clear that the matrix—vector product Ac need not
be thought of as m distinct scalar summations, each giving a different linear
combination of the entries of c, as (1.1) might suggest. Instead, A can be
viewed as a matrix of columns, each giving sampled values of a monomial, A: 1 :13 51:2 {En—1 , (1.4) and the product Ac should be understood as a single vector summation in the
form of (1.2) that at once gives a linear combination of these monomials, Ac : c0 + 01:17 + 02.702 +    + 0,111.1:“1 : p(:z:) D The remainder of this lecture will review some fundamental concepts in
linear algebra from the point of View of (1.2). A Matrix Times a Matrix For the matrix—matrix product B 2 AC, each column ofB is a linear com
bination of the columns of A. To derive this fact7 we begin with the usual
formula for matrix products. If A is E X m and C is m X n, then B is E X n,
with entries deﬁned by k:1 Here bij, am, and CM are entries of B, A, and C, respectively. Written in terms of columns, the product is b1 b2 0‘
  0’1 012 ... a , 6 PART I FUNDAMENTALS and (1.5) becomes
bj = ch = Z ckjak. (1.6)
19:1
Thus bj is a linear combination of the columns ak with coefﬁcients ckj. Example 1.2. A simple example of a matrix—matrix product is the outer
product. This is the product of an m—dimensional column vector u with an
n—dimensional row vector u; the result is an m X n matrix of rank 1. The outer
product can be written 711 v2 u Ulul “#1 u = ulu 112uuu ulu u u 7Tb 71m The columns are all multiples of the same vector u, and similarly, the rows
are all multiples of the same vector u. Example 1.3. As a second illustration, consider B : AR, where R is the
upper—triangular n x n matrix with entries rij : 1 for 2' g j and rij : 0 for
2' > j. This product can be written 1 1
b on = a an
1 1 1
The column formula (1.6) now gives
1
b] : ATj : Zak. (1.7)
k:1 That is, the jth column of B is the sum of the ﬁrst 3’ columns of A. The
matrix R is a discrete analogue of an indeﬁnite integral operator. Range and Nullspace The range of a matrix A, written range(A), is the set of vectors that can be
expressed as A30 for some 1:. The formula (1.2) leads naturally to the following
characterization of range(A): Theorem 1.1. range(A) is the space spanned by the columns of A. LECTURE 1 MATRIX—VECTOR MULTIPLICATION 7 Proof. By (1.2), any A1: is a linear combination of the columns of A. Con—
versely, any vector y in the space spanned by the columns of A can be written
as a linear combination of the columns, 3/ : 22;, ijaj. Forming a vector 1:
out of the coefﬁcients 17,, we have y 2 A33, and thus y is in the range of A. In view of Theorem 1.1, the range of a matrix A is also called the column
space of A. The nullspace of A E Cm”, written null(A), is the set of vectors :1: that
satisfy Azr: : 0, where 0 is the 0—vector in 03’”. The entries of each vector
x E null(A) give the coefficients of an expansion of zero as a linear combination
of columns of A: 0 = mlol + {132% + .  ' + :Irnan. Rank The column rank of a matrix is the dimension of its column space. Similarly,
the row rank of a matrix is the dimension of the space spanned by its rows.
Row rank always equals column rank (among other proofs, this is a corollary
of the singular value decomposition, discussed in Lectures 4 and 5), so we refer
to this number simply as the rank of a matrix. An m X n matrix of full rank is one that has the maximal possible rank (the
lesser of m and n). This means that a matrix of full rank with m 2 n must
have n linearly independent columns. Such a matrix can also be characterized
by the property that the map it deﬁnes is one—to—one: Theorem 1.2. A matrix A E Cm” with in 2 n has full rank if and only if
it maps no two distinct vectors to the some vector. Proof. (:>) If A is of full rank, its columns are linearly independent, so they
form a basis for range(A). This means that every I) E range(A) has a unique
linear expansion in terms of the columns of A, and therefore, by (1.2), every
b E range(A) has a unique 33 such that b 2 A93. (<2) Conversely, if A is
not of full rank, its columns a, are dependent, and there is a nontrivial linear
combination such that 2;, cjaj = 0. The nonzero vector c formed from the
coefﬁcients cj satisﬁes Ac = 0. But then A maps distinct vectors to the same vector since, for any ac, A1: : A(IL‘ + c). D Inverse A nonsingviar or invertible matrix is a square matrix of full rank. Note
that the rn columns of a nonsingular in X in matrix A form a basis for the
whole space 03’". Therefore, we can uniquely express any vector as a linear 8 PART I FUNDAMENTALS combination of them. In particular, the canonical unit vector with 1 in the
jth entry and zeros elsewhere, written ej, can be expanded: m i:1 Let Z be the matrix with entries 2,], and let zj denote the jth column of
Z. Then (1.8) can be written ej : Azj. This equation has the form (1.6); it can be written again, most concisely, as The matrix Z is the inverse of A. Any square nonsingular matrix A has a
unique inverse, written A‘l, that satisﬁes AA‘1 : A‘lA : I. The following theorem records a number of equivalent statements that hold
when a square matrix is nonsingular. These conditions appear in linear algebra
texts, and we shall not give a proof here. Concerning (f), see Lecture 5. Theorem 1.3. For A E (Dmxm, the following conditions are equivalent: (a) A has an inverse A’l, (b) rank(A) = m, (c) range(A) : (Um, a) nu11(A> = {0}, (e) 0 is not an eigenvalue of A,
(f) 0 is not a singular value ofA, (9) detM) 7E 0 Concerning (9), we mention that the determinant, though a convenient notion
theoretically, rarely ﬁnds a useful role in numerical algorithms. A Matrix Inverse Times a Vector When writing the product a = A‘lb, it is important not to let the inverse— matrix notation obscure what is really going on! Rather than thinking of a as the result of applying A—1 to b, we should understand it as the unique vector that satisﬁes the equation Ax = b. By (1.2), this means that a: is the vector of coefﬁcients of the unique linear expansion of b in the basis of columns of A.
This point cannot be emphasized too much, so we repeat: A’lb is the vector of coefﬁcients of the expansion ofb
in the basis of columns of A. LECTURE 1 MATRIX—VECTOR MULTIPLICATION 9 Multiplication by A—1 is a change of basis operation: Multiplication by A—1 A‘lb:
coefﬁcients of
the expansion of b in {a1,...,am} b:
coefﬁcients of
the expansion of b in {el,...,em} Multiplication by A In this description we are being casual with terminology, using “b” in one
instance to denote an m—tuple of numbers and in another as a point in an
abstract vector space. The reader should think about these matters until he
or she is comfortable with the distinction. A Note on m and n Throughout numerical linear algebra, it is customary to take a rectangular
matrix to have dimensions m X n. We follow this convention in this book. What if the matrix is square? The usual convention is to give it dimensions
n x 71, but in this book we shall generally take the other choice7 m X m. Many
of our algorithms require us to look at rectangular subrnatrices formed by
taking a subset of the columns of a square matrix. If the submatrix is to be
m x n, the original matrix had better be m X m. Exercises Let B be a 4 x 4 matrix to which we apply the following operations: . double column 17 . halve row 3, . add row 3 to row 1, . interchange columns 1 and 4, . subtract row 2 from each of the other rows, . replace column 4 by column 3, 7. delete column 1 (so that the column dimension is reduced by 1). [\3>—\ @UllDQJ (a) Write the result as a product of eight matrices.
(b) Write it again as a product ABC (same B) of three matrices. Suppose masses m1, m2, m3, m4 are located at pos1tions x1, 1:2, 1:3, 1:4 in a line
and connected by springs w1th spr1ng constants k12,k23,k34 whose natural 10 PART I FUNDAMENTALS lengths of extension are €12,623,€34. Let f1,f2,f3,f4 denote the rightward
forces on the masses, e.g., f1 : k12(m2 — 1:1 — £12). (a) Write the 4 X 4 matrix equation relating the column vectors f and :13. Let
K denote the matrix in this equation. (b) What are the dimensions of the entries of K in the physics sense (e.g.,
mass times time, distance divided by mass, etc)? (0) What are the dimensions of det(K)7 again in the physics sense? (01) Suppose K is given numerical values based on the units meters, kilograms,
and seconds. Now the system is rewritten with a matrix K’ based on centime—
ters, grams, and seconds. What is the relationship of K’ to K? What is the
relationship of det(K’) to det(K )? Generalizing Example 1.3, we say that a square or rectangular matrix R with
entries 7'” is uppertriangular if 7”,, : 0 for i > j. By considering what space
is spanned by the ﬁrst 71 columns of R and using (1.8), show that if R is a
nonsingular m X m uppertriangular matrix, then R‘1 is also upper—triangular.
(The analogous result also holds for lower—triangular matrices.) Let f1,...,f8 be a set of functions deﬁned on the interval [1,8] with the
property that for any numbers d1,...,d8, there exists a set of coefficients
01,... ,08 such that 8
j:1 (a) Show by appealing to the theorems of this lecture that d1, . . . , d8 determine
c1, . . . , c8 uniquely. (b) Let A be the 8 X 8 matrix representing the linear mapping from data
611,. . . ,d8 to coefficients 01,... ,cS. What is the z',j entry of A’l? ...
View
Full Document
 Spring '10
 Johnson

Click to edit the document details