Manual Solution for:
Linear Algebra
by Gilbert Strang
John L. Weatherwax
January 1, 2006
Introduction
A Note on Notation
In these notes, I use the symbol to denote the results of elementary elimination matrices
used to transform a given matrix into its reduced row echelon form. Thus when looking for
the eigenvectors for a matrix like
002
A= 0 1 0
002
rather than say, multiplying A on the left by
E33
produces
1 00
= 0 1 0
1 0 1
002
E33 A = 0 1 0
000
we will use the much more compact notation
002
002
A= 0 1 0 0 1 0 .
002
000
[e-mail]
1
Derivation of the decomposition (Page 170)
Combining the basis for the row space and the basis for the nullspace into a common matrix
T
to assemble a general right hand side x = a b c d
from some set of components
T
c = c1 c2 c3 c4
we must have
a
c1
10 1
0
c1
c2 b
c2 0 1 0
1
A
c3 = 1 0 1 0 c3 = c
d
c4
0 1 0 1
c4
Inverting the coecient matrix A by using the teaching code elim.m or augmentation and
inversion by hand gives
10 1
0
1 0 1 0
1
.
A1 =
2 1 0 1 0
0 1 0 1
So the coecients of c1 , c2 , c3 , and c4 are given by
a
c1
a+c
b 1 b+d
c2
1
c3 = A c = 2 a c
d
c4
bd
As veried by what is given in the book.
Chapter 1 (Introduction to Vectors)
Section 1.1 (Vectors and Linear Combinations)
Problem 16 (dimensions of a cube in four dimensions)
We can generalize Problem 15 by stating that the corners of a cube in four dimensions are
given by
n(1, 0, 0, 0) + m(0, 1, 0, 0) + l(0, 0, 1, 0) + p(0, 0, 0, 1) ,
for indices n, m, l, p taken from {0, 1}. Since the indices n, m, l, p can take two possible values
each the total number of such vectors (i.e. the number of corners of a four dimensional cube)
is given by 24 = 16.
To count the number of faces in a four dimensional cube we again generalize the notion
of a face from three dimensions. In three dimensions the vertices of a face is dened by
a conguration of n, m, l where one component is specied. For example, the top face is
specied by (n, m, 1) and the bottom face by (n, m, 0), where m and n are allowed to take
all possible values from {0, 1}. Generalizing to our four dimensional problem, in counting
faces we see that each face corresponds to rst selecting a component (either n, m, l, or p)
setting it equal to 0 or 1 and then letting the other components take on all possible values.
The component n, m, l, or p can be chosen in one of four ways, from which we have two
choices for a value (0 or 1). This gives 2 4 = 8 faces.
To count the number of edges, remember that for a three dimensional cube an edge is
determined by specifying (and assigning to) all but one elements of our three vector. Thus
selecting m and p to be 0 we have (n, 0, 0) and (n, 0, 0), where n takes on all values from
{0, 1} as vertices that specify one edge. To count the number of edges we can rst specifying
the one component that will change as we move along the given edge, and then specify a
complete assignment of 0 and 1 to the remaining components. In four dimensions, we can
pick the single component in four ways and specify the remaining components in 23 = 8,
ways giving 4 8 = 32 edges.
Problem 17 (the vector sum of the hours in a day)
Part (a): Since every vector can be paired with a vector pointing in the opposite direction
the sum must be zero.
Part (b): We have
vi
i=4
v4 = 0 v4 = v4 ,
vi
vi =
1
v1
v1
v1 + v1 = 0
= ,
2
2
2
all i
with v4 denoting the 4:00 vector.
Part (c): We have
i=1
1
vi + v1 =
2
all i
with v1 denoting the 1:00 vector.
Problem 18 (more clock vector sums)
We have from Problem 17 that the vector sum of all the vi s is zero,
vi = 0 .
i{1,2,...12}
Adding twelve copies of (0, 1) = to each vector gives
j
i{1,2,...12}
(vi ) = 12 .
j
j
But if in addition to the transformation above the vector 6:00 is set to zero and the vector
12:00 is doubled, we can incorporate those changes by writing out the above sum and making
the terms summed equivalent to the specication in the book. For example we have
(vi ) + (v6 ) + (v12 ) = 12
j
j
j
j
(vi ) + (0 ) + (2v12 ) = v6 + v12 12
j
j
j
j
(vi ) + (0 ) + (2v12 ) = (0, 1) + (0, 1) 12(0, 1) = 10(0, 1) .
j
j
j
i={6,12}
i={6,12}
i={6,12}
The left hand side now gives the requested sum. In the last equation, we have written out
the vectors in terms of their components to perform the summations.
Problem 26 (all vectors from a collection )
Not if the three vector are not degenerate, i.e. are not all constrained to a single line.
Problem 27 (points in common with two planes)
Since the plane spanned by u and v and the plane spanned by v and w intersect on the line
v , all vectors cv will be in both planes.
Problem 28 (degenerate surfaces)
Part (a): Pick three vectors collinear, like
u = (1, 1, 1)
v = (2, 2, 2)
w = (3, 3, 3)
Part (b): Pick two vectors collinear with each other and the third vector not collinear with
the rst two. Something like
u = (1, 1, 1)
v = (2, 2, 2)
w = (1, 0, 0)
Problem 29 (combinations to produce a target)
Let c and d be scalars such that combine our given vectors in the correct way i.e.
c
1
2
+d
3
1
=
14
8
which is equivalent to the system
c + 3d = 14
2c + d = 8
which solving for d using the second equation gives d = 8 2c and inserting into the rst
equation gives c + 3(8 2c) = 14, which has a solution of c = 2. This with either of the
equations above yields d = 2.
Section 1.2 (Lengths and Dot Products)
Problem 1 (simple dot product practice)
We have
uv
uw
vw
wv
=
=
=
=
.6(3) + .8(4) = 1.4
.6(4) + .8(3) = 0
3(4) + 4(3) = 24
24 .
Chapter 2 (Solving Linear Equations)
Section 2.2 (The Idea of Elimination)
Problem 1
We should subtract 5 times the rst equation. After this step we have
2x + 3y = 11
6y = 6
or the system
The two pivots are 2 and -6.
23
0 6
Problem 2
the last equation gives y = 1, then the rst equation gives 2x 3 = 1 or x = 2. Lets check
the multiplication
1
3
2
(1)
(1) =
(2) +
11
9
10
If the right hand changes to
4
44
(2)
then -5 times the rst component added to the second component gives 44 20 = 24.
Chapter 3 (Vector Spaces and Subspaces)
Section 3.1
Problem 5
Part (a): Let M consist of all matrices that are multiples of
10
00
.
Part (b): Yes, since the element 1 A + (1) B = I must be in the space.
Part (c): Let the subspace consist of all matrices dened by
a
10
00
+b
00
01
Problem 6
We have h(x) = 3(x2 ) 4(5x) = 3x2 20x.
Problem 7
Rule number eight is no longer true since (c1 + c2 )x is interpreted as f ((c1 + c2 )x) and
c1 x + c2 x is interpreted as f (c1 x) + f (c2 x), while in general for arbitrary functions these two
are not equal i.e. f ((c1 + c2 )x) = f (c1 x) + f (c2 x).
Problem 8
The rst rule x + y = y + x is broken since f (g (x)) = g (f (x)) in general.
The second rule is correct.
The third rule is correct with the zero vector dened to be x.
The fourth rule is correct if we dene x to be the inverse of the function f (), because
then the rule f (g (x)) = x states that f (f 1 (x)) = x, assuming an inverse of f exists.
The seventh rule is not true in general since c(x + y ) is cf (g (x)) and cx + cy is cf (cg (x))
which are not the same in general.
The eighth rule is not true since the left hand side (c1 + c2 )x is interpreted as (c1 +
c2 )f (x), while the right hand side c1 x + c2 x is interpreted as c1 f (c2 f (x)) which are not
equal in general.
Problem 9
Part (a): Let the vector
x
y
1
1
=
+c
1
0
+d
0
1
.
For c 0 and d 0. Then this set is the upper right corner in the rst quadrant of the
xy plane. Now note that the sum of any two vectors in this set will also be in this set but
scalar multiples of a vector in this set may not be in this set. Consider
1
2
1
1
=
1
2
1
2
,
which is not be in the set.
Part (b): Let the set consist of the x and y axis (all the points on them). Then for any
point x on the axis cx is also on the axis but the point x + y will almost certainly not be.
Problem 10
Part (a): Yes
1
Part (b): No, since c(b1 , b2 , b3 ) = c(1, b2 , b3 ) is not in the set if c = 2 .
Part (c): No, since if two vectors x and y are such that x1 x2 x3 = 0 and y1 y2 y3 = 0 there is
no guarantee that x + y will have that property. Consider
0
1
x = 1 and y = 0
1
1
Part (d): Yes, this is a subspace.
Part (e): Yes, this is a subspace.
Part (f): No this is not a subspace since if
b1
b = b2 ,
b3
has this property then cb should have this property but cb1 cb2 cb3 might not be true.
Consider
100
b = 10 and c = 1 .
1
Then b1 b2 b3 but cb1 cb2 cb3 is not true.
Problem 11
Part (a): All matrices of the form
ab
00
for all a, b R.
Part (b): All matrices of the form
aa
00
for all a R.
Part (c): All matrices of the form
a0
0b
or diagonal matrices.
Problem 12
1
4
5
Let the vectors v1 = 1 and v2 = 0 , then v1 + v2 = 1 but 5 + 1 2(2) =
2
0
2
10 = 4 so the sum is not on the plane.
Problem 13
1
The plane parallel to the previous plane P is x + y 2z = 0. Let the vectors v1 = 1
1
1
2
0 , which are both on P0 . Then v1 + v2 = 1 . We then check that this
and v2 =
1
2
3
2
point is on our plane by computing the required sum. We nd that 2 + 1 2
see that it is true.
3
2
= 0, and
Problem 14
Part (a): Lines, R2 itself, or (0, 0, 0).
Part (b): R4 itself, hyperplanes of dimension four (one linear constraining equation among
four variables) that goes through the origin like the following
ax1 + bx2 + cx3 + dx4 = 0 .
Constraints involving two linear equation like toe above (going through the origin)
ax1 + bx2 + cx3 + dx4 = 0
Ax1 + Bx2 + Cx3 + Dx4 = 0 ,
which is eectively a two dimensional plane. In addition, constraints involving three equations like above and going through the origin (this is eectively a one dimensional line).
Finally, the origin itself.
Problem 15
Part (a): A line.
Part (b): A point (0, 0, 0).
Part (c): Let x and y be elements of S T . Then x + y S T and cx S T since x
and y are both in S and in T , which are both subspaces and therefore x + y and cx are both
in S T .
Problem 16
A plane (if the line is in the plane to begin with) or all of R3 .
Problem 17
Part (a): Let
A=
10
01
which are both invertible. Now A + B =
and B =
1 0
0 1
,
00
, which is not. Thus the set of invertible
00
matrices is not a subspace.
Part (b): Let
A=
13
26
which are both singular. Now A + B =
and B =
63
21
,
76
, which is not singular, showing that the set
46
of invertible matrices is not a subspace.
Problem 18
Part (a): True, since if A and B are symmetric then (A + B )T = AT + B T = A + B is
symmetric. Also (cA)T = cAT = cA is symmetric.
Part (b): True, since if A and B are skew symmetric then (A + B )T = AT + B T = A B =
(A + b) and A + B is skew symmetric. Also if A is skew symmetric then cA is also since
(cA)T = cAT = cA.
0 1
13
, which is
which is unsymmetric and B =
00
25
12
should be unsymmetric but its not. Thus the set
also unsymmetric then A + B =
25
of unsymmetric matrices is not closed under addition and therefore is not a subspace.
Part (c): False since if A =
Problem 19
12
If A = 0 0 , then the column space is given by
00
12
x1 + 2x2
0 0 x1 =
,
0
x2
00
0
12
which is a line in the x-axis (i.e. all combinations of elements on the x-axis. If B = 0 2
00
x1
10
2x2 or the entire xy plane. If C = 2 0 then Cx is
then the column space of B is
0
00
x1
2x2 or a line in the xy plane.
given by
0
Problem 20
Part (a): Consider the augmented matrix
1
4
2
2
8
4
1 4 2
Let E21 be given by
E21
Then we nd that
1
4
2
8
4
E21 2
1 4 2
so that b2 = 2b1 and b1 = b3 .
b1
b2
b3
1 00
= 2 1 0 ,
1 01
b1
142
b2 = 0 0 0
000
b3
b1
b2 2b1 ,
b3 + b + 1
Part (b):
Let E21 and E31 be given by
E21
Then we see that
1
4
2
9
1 4
1 00
= 2 1 0
0 01
1
4
2
9
E31 E21
1 4
x1
x2
b1
= b2
b3
and E31
100
= 0 1 0 ,
101
b1
14
= 0 1
b2
b3
00
which requires that b1 + b3 = 0, or b1 = b3 .
b1
b2 2b1 ,
b3 + b + 1
Problem 21
A combination of the columns of B and C are also a combination of the columns of A. Those
two matrices have the same column span.
Problem 22
For the rst system
111
x1
b1
0 1 1 x2 = b2 ,
001
x3
b3
we see that for any values of b the system will have a solution. For the second system
111
b1
0 1 1
b2
000
b3
we see that we must have b3 = 0. For the
1
0
0
which is equivalent to
so we must have b2 = b3 .
third system
b1
11
01
b2
01
b3
111
0 0 1
000
b1
b2 ,
b3 b2
Problem 23
10
0
Unless b is a combination of the previous columns of A. If A = 0 1 with b = 0
00
1
10
2
0 1 with b = 0 the column space does not
has a large column space. But if A =
00
0
change. Because b can be written as a linear combination of the columns of A and therefore
adds no new information to the column space.
Problem 24
The column space of AB is contained in the and possibly equals the column space of A. If
01
01
10
which is of a smaller dimension than
, then AB =
and B =
A=
00
00
01
the original column space of A.
Problem 25
If z = x + y is a solution to Az = b + b . If b and b are in the column space of A then so is
b + b .
Problem 26
Any A that is a ve by ve invertible matrix has R5 as its column space. Since Ax = b
always has a solution then A is invertible.
Problem 27
12
1
0
then x1 =
and x2 =
are each not in the
12
0
1
1
column space but x1 + x2 =
is in the column space. Thus the set of vectors not in the
1
column space is not a subspace.
Part (a): False. Let A =
Part (b): True.
Part (c): True.
Part (d): False, the matrix I can add a full set of pivots (linearly independent rows). Let
10
00
, then A has a column space consisting of the zero vector
, with I =
A=
01
00
and
1 0
AI =
,
0 1
has all of R2 as its column space.
Problem 28
112
1 0 0
012
or
112
1 0 1
011
Section 3.2
Problem 1
Fpr the matrix (a) i.e
let E21 be given by
12246
1 2 3 6 9
00123
E21
so that
Now let E33 be given by
12246
E21 A = 0 0 1 2 3 .
00123
E21
So that
1 00
= 1 1 0 ,
0 01
100
= 0 1 0 .
0 1 1
12246
E33 E21 A = 0 0 1 2 3 .
00000
Which has pivot variables x1 and x3 and free variables x2 , x4 and x5 . For the matrix (b)
242
A= 0 4 4
088
let E32 be given by
E32
so that
100
= 0 1 0 ,
0 2 1
242
E32 A = 0 4 4 = U .
000
Then the free variables are x3 and the pivot variables are x1 and x2 .
Problem 2
Since the ordinary echelon form for the matrix in (a) is
12246
U = 0 0 1 2 3 ,
00000
we nd a special solution that corresponds to each free vector by assigning ones to each free
variable in turn and then solving for the pivot variables. For example, since the free variables
are x2 , x4 , and x5 we begin by letting x2 = 1, x4 = 0, and x5 = 0. Then our system becomes
x1
12246 1
0 0 1 2 3 x3 = 0
00000 0
0
or
12
0 1
00
x1
x3
2
= 0
0
which has a solution x3 = 0 and x1 = 2. So our special solution in this case is given by
2
1
0 .
0
0
For the next special solution let x2 = 0, x4 = 1, and x5 = 0. Then our special solution solves
x1
12246 0
0 0 1 2 3 x3 = 0
00000 1
0
or
12
0 1
00
x1
x3
=
4
2
Which requires x3 = 2 and x1 + 2(2) = 4 or x1 = 0. Then our second special solution
is given by
0
0
2 .
1
0
Our nal special solution is obtained by setting x2 = 0, x4 = 0, and x5 = 1. Then our
system is
x1
12246 0
0 0 1 2 3 x3 = 0
00000 0
1
which reduces to solving
12
0 1
00
x1
x3
=
6
3
So that x3 = 3 and x1 = 6 2(3) = 0 is given by
0
0
3 .
0
1
Lets check our calculations. Create a matrix N with columns consisting of the three special
solutions found above. We have
2 0
0
1
0
0
N = 0 2 3 ,
2 1
0
0
0
1
And then the product of A times N should be zero. We see that
2 0
0
000
0
0
12246 1
AN = 0 0 1 2 3 0 2 3 = 0 0 0 ,
000
0
0 0 0 0 0 2 1
0
0
1
as it should. For the matrix in part (b) we have that
242
U = 0 4 4
000
then the pivot variables are x1 and x2 while the free variables are x3 . Setting x3 = 1 we
obtain the system
24
x1
2
=
,
04
x2
4
so that x2 = 1 and x1 =
2(4)(1)
2
= 1, which gives a special solution of
1
1 .
1
Problem 3
From Problem 2 we have three special solutions
0
2
0
1
v1 = 0 , v2 = 2
1
0
0
0
,
v3 =
0
0
3
0
1
,
then any solution to Ax = 0 can be expressed as a linear combination of these special
solutions. The nullsapce of A contains the vector x = 0 only when there are no free variables
or there exist n pivot variables.
Problem 4
The reduced echelon form R has ones in the pivot columns of U . For Problem 1 (a) we have
12246
U = 0 0 1 2 3 ,
00000
1 2 0
0 1 0 , so that
then let E13 =
001
12000
E13 U = 0 0 1 2 3 R
00000
The nullspace of R is equal to the nullspace of U since row opperations dont change the
nullspace. For Problem 1 (b) our matrix U is given by
242
U = 0 4 4
000
1 1 0
0 1 0 , so that
so let E12 =
001
2 0 2
E12 U = 0 4 4 .
00 0
1/2 0 0
0 1/4 0 , then
Now let D =
0
01
1 0 1
DE12 U = 0 1 1 .
00 0
Problem 5
For Part (a) we have that
A=
then letting E21 =
10
2 1
10
21
,
we get that
E21 A =
Then since E211 =
1 3 5
2 6 10
1 3 5
0 00
.
we have that
A = E211 U =
10
21
1 3 5
0 00
.
Where we can dene the rst matrix on the right hand side of the above to be L. For
Part (b) we have that
1 3 5
,
A=
2 6 7
then letting E21 be the same as before we see that
E21 A =
1 3 5
0 0 3
.
so that a decoposition of A is given by
A = E211 U =
10
21
1 3 5
0 0 3
.
Problem 6
1 3 5
so we see that x1 is a pivot variable and
0 00
x2 and x3 are free variables. Then two special solutions can be computed by setting x2 = 1,
x3 = 0 and x2 = 0, x3 = 1 and solving for x1 . In the rst case we have x1 + 3 = 0 or x1 = 3
giving a special vector of
3
1.
v1 =
0
For Part (a) since we have that U =
In the second case we have x1 + 5 = 0 giving x1 = 5, so that the second special vector is
given by
5
v2 = 0 .
1
Thus all special solutions to Ax = 0 are contained in the set
3
5
c1 1 + c2 0 .
0
1
1 3 5
so we see that x1 and x3 are pivot
0 0 3
variable while x2 is a free variables. To solve for the vector in the nullspace set x2 = 1 and
solve for x1 and x3 . This gives
x1
1 3 5
1 = 0,
0 0 3
x3
For Part (b) since we have that U =
or the system
1 5
0 3
x1
x3
=
3
0
.
This gives x3 = 0 and x1 = 3. So we have a special vector given by
3
1.
0
For an mxn matrix the number of free variables plus the number of pivot variables equals n.
Problem 7
For Part (a) the nullspace of A are all points (x, y, z ) such that
3 c1 + 5 c2
x
= y ,
c1
c2
z
or the plane x = 3y + 5z . This is a plane in the xyz space. This space can also be described
as all possible linear combinations of the two vectors
3
5
1 and 0 .
0
1
3
For Part (b) the nullspace of A are all points that are multiples of the vector 1 which is
0
3
a line in R . Equating this vector to a point (x, y, z ) we see that our line is given by x = 3c,
y = c, and z = 0 or equivalently x = 3y and z = 0.
Problem 8
1 3 5
. Let D =
0 00
For Part (a) since we have that U =
DU =
1 3 5
00
0
1 0
01
then we have that
,
which is in reduced row echelon form. The identity matrix in this case is simply the scalar
1 giving
1
3 5
DU =
0
0
0
where we have put a box around the identity in this case. For Part (b) since we have that
1 0
1 3 5
so that dening D =
U=
we then have that
1
0 3
0 0 3
DU =
The let E13 =
15
01
1 3 5
00
1
.
and we then get that
E13 DU =
1 3 0
001
,
for our reduced row echelon form. Our box around the identinty in the matrix R is around
the pivot rows and pivot columns and is given by
1
0
3
0
0
1
Problem 9
Part (a): False. This depends on what the reduced echelon matrix looks like. Consider
11
11
A=
. Then the reduced echelon matrix R is
, which has x2 as a free
11
00
variable.
Part (b): True. An invertible matrix is dened as one that has a complete set of pivots i.e.
no free variables.
Part (c): True. Since the number of free variables plus the number of pivot variables equals
n in the case of no free variables we have the maximal number of pivot variables n.
Part (d): True. If m n, then by Part (c) the number of pivot variables must be less
than n and this is equivalent to less than m. If m < n then we have fewer equations than
unknowns and when our linear system is reduced to echelon form we have a maximal set of
pivot variables. We can have at most m, corresponding to the block identity in the reduced
row echelon form in the mxm position. The remaining n m variables must be free.
Problem 10
Part (a): This is not possible since going from A to U involves zeroing elements below
the diagonal only. Thus if an element is nonzero above the diagonal it will stay so for all
elimination steps.
Part (b): The real requirement to nd a
matrix A is that A have three linearly independent
1
2
3
100
columns/rows. Let A = 1 1 3 , then with E = 1 1 0 we nd that
1 2 2
101
123
EA = 0 1 0 .
001
1 2 3
Continuing this process let E = 0 1
0 then
00
1
100
E EA = 0 1 0 = I .
001
Part (c): This is not possible and the reason is as follows.
11
of its pivot variables. What about the matrix A =
22
Then
111
10
A=
U=
000
2 1
R must have zeros above each
1
which has no zero entries.
2
,
which also equals R.
1
Part (d): If A = U = 2R, then R = 2 A = 1 U so let
2
R=
so take A =
10
01
=
1
2
1
1
= A= U.
2
2
20
02
20
.
02
Problem 11
Part (a): Consider
0
0
0
0
1
0
0
0
x
0
0
0
x
1
0
0
x
0
1
0
x
x
x
0
x
x
x
0
Part (b): Consider
1
0
0
0
x
0
0
0
0
1
0
0
x
x
0
0
x
x
0
0
0
0
1
0
0
0
1
1
0
0
0
0
0
0
0
0
0
0
0
0
1
0
0
0
x
0
0
0
x
1
0
0
x
x
0
0
1
0
0
0
x
0
0
0
x
1
0
0
x
x
1
0
x
x
x
1
x
x
x
x
Part (c): Consider
Problem 12
Part (a): Consider
0
0
R=
0
0
this is so that the pivot variables are
x5 , and x6 we we have
1
0
R=
0
0
x
x
,
x
x
x2 , x4 , x5 , and x6 . For the free variables to be x2 , x4 ,
x
0
0
0
0
1
0
0
x
x
0
0
x
x
0
0
x
x
0
0
0
0
1
0
0
0
,
0
1
1
0
0
0
x
0
0
0
0
1
0
0
0
0
1
0
x
x
x
0
x
x
x
0
x
x
.
x
0
Part (b): Consider
0
0
R=
0
0
Problem 13
x4 is certainly a free variable and the special solution is x = (0, 0, 0, 1, 0).
Problem 14
Then x5 is a free variable. The special solution is x = (1, 0, 0, 0, 1).
Problem 15
If an mxn matrix has r pivots the number of special solutions is n r . The nullspace contains
only zero when r = n. The column space is Rm when r = m.
Problem 16
When the matrix has ve pivots. The column space is R5 when the matrix has ve pivots.
Since m = n then Problem 15 demonstrates that the rank must equal m = n.
Problem 17
x
If A = 1 3 1 and x = y , the free variables are y and z . Let y = 1 and z = 0
z
3
1 . The second special solution is given by
then x = 3 giving the rst special solution of
0
setting y = 0 and z = 1, then x 1 = 0 or x = 1, so we have a second special solution of
1
0 .
1
Problem 18
If x 3y z = 12, then expressing the vector (x, y, z ) in iterms of y and z we nd
x
12
3
1
y = 0 +y 1 +z 0 .
z
0
0
1
Problem 19
For x in the nullspace of B means that Bx = 0 thus ABx = A0 = 0 and thus x is in
the nullspace of AB . The nullspace of B is contained in the nullspace of AB . An obvious
example when the nullspace of AB is larger than the nullspace of B is when
B=
10
10
,
0
. If A =
1
which has a nullspace given by the span of the vector
00
00
then AB =
00
, and has a nullspace given by the span of
00
1
0
0
1
and
,
which is larger than the nullspace of B .
Problem 20
If A is invertible then the nullspace of AB equals the nullspace of B . If v is an element of
the nullspace of AB then ABv = 0 of Bv = 0 by multiplying both sides by A1 . Thus v is
an element of the nullspace of B .
Problem 21
We see that x3 and x4 are free variables. To determine the special solutions we consider the
two assignments (x3 , x4 ) = (1, 0), and (x3 , x4 ) = (0, 1). Under the rst we have
10
01
x1
x2
=
2
2
which give
1 0 2
0 1 2
x1
x2 = 0 .
1
In the same way under the second assignment we have
x1
1 0 3
x2 = 0 .
01 1
x4
when we combine these two results we nd that
1 0 2 3
0 1 2 1
so that A is given by
A=
x1
x2
x3 = 0 ,
x4
1 0 2 3
0 1 2 1
.
Problem 22
If x4 = 1 and the other variables are
10
0 1
00
or
solved for we have
0
x1
4
x2 = 3 (1)
0
1
x3
2
x1
1 0 0 4
0 1 0 3 x2 = 0
x3
0 0 1 2
x4
so that A is given by
1 0 0 4
A = 0 1 0 3 .
0 0 1 2
Problem 23
We have three equations with a rank of two which means that the nullity must be one. Let
10a
1
1 3 b for some a, b, and c. Then if 1 is to be in the nullity of A we must
A=
51c
2
have
1
10a
1
1 + 2a
A 1 = 1 3 b 1 = 1 + 3 + 2b = 0 .
2
51c
2
5 + 1 + 2c
Which can be made true if we take a = 1 , b =
2
case is
10
A= 1 3
51
2, and c = 3. Thus our matrix A in this
1/2
2 .
3
Problem 24
The number of equations equals three and the rank is two. We are requiring that the nullspace
be of dimension two (i.e. spanned by two linearly independent vectors), thus m = 3 and
n = 4. But the dimension of the vectors in the null space is three which is not equal to four.
Thus it is not possible to nd a matrix with such properties.
Problem 25
1 1 0
0
We ask will the matrix A = 1 0 1 0 , work? Then if the column space contains
10
0 1
(1, 1, 1) then m = 3. If the nullspace is (1, 1, 1, 1) then n = 4. Reducing A we see that
1 1 0
0
1 0 1 0
1 0 0 1
A 0 1 1 0 0 1 1 0 0 1 0 1 .
01
0 1
0 0 1 1
0 0 1 1
So if Av = 0, then
x
100 1
0 1 0 1 y = 0
z
0 0 1 1
w
Implying that x w = 0, y w = 0, and z w = 0, thus our vector v is given by
1
x
1
y
v=
z = w 1 ,
1
w
and our matrix A does indeed work.
Problem 26
A key to solving this problem is to recognize that if the column space of A is also its nullspace
then AA = 0. This is because AA represents A acting on each column of A and this produces
zero since the column space is the nullspace. Thus we need a matrix A such that A2 = 0. If
ab
, the requirement of A2 = 0 means that
A=
cd
a2 + bc ab + bd
ac + cd cb + d2
=
00
00
.
This gives four equations for the unknowns a,b,c, and d. To nd one solution let a = 1 then
d = 1 by considering the (1, 2) element. Our matrix equation then becomes
1 + bc
0
0
cb + 1
=
00
00
.
Now let 1 + bc = 0, which we can satisfy if we take b = 1 and c = 1. Thus with all of these
unknowns specied we have that our A is given by
A=
1
1
1 1
.
r n-r=3-r
1
2
2
1
3
0
Table 1: All possible combinations of the dimension of the column space and the row space
for a three by three matrix.
We can check this result. It is clear that As row space is spanned by
is given by computing the R matrix
R=
giving n =
11
00
1
1
and its nullity
,
1
.
1
Problem 27
In a three by three matrix we have m = 3 and n = 3. If we say that the column space has
dimension r the nullity must then have dimension n r . Now r can be either 1, 2, or 3. If
we consider each possibility in tern we have Table 1, from which we see that we never have
the column space equal to the row space.
Problem 28
If AB = 0 then the column space of B is contained in the nullity of A. For example the
product AB can be written by recognizing this as the action of A on the columns of B . For
example
AB = A b1 |b2 | |bn = Ab1 |Ab2 | |Abn = 0 ,
which means that Abi = 0 for each i. Let A =
span of
1
. Next consider B =
1
1 1
1 1
which has nullity given by the
12
. From which we see that AB = 0.
12
Problem 29
Almost sure to be the identity. With a random
end with
10
0 1
R=
0 0
00
four by three matrix one is most likely to
0
0
.
1
0
Problem 30
Part (a): Let A =
has
1
1
1
1
1 1
then A has
1
1
as its nullspace, but AT =
as its nullspace.
111
, then x2
002
10
1
T
1 0 0
A=
12
1
Part (b): Let A =
is a free variable. Now
0
10
10
0 0 2 0 1 ,
2
00
00
which has no free variables. A similar case happens with
111
A= 0 0 2 ,
000
Then A has x2 as a free variable and AT has x3 as a free variable.
Part (c): let A be given by
Then
111
A= 0 0 0 .
020
111
111
0 0 0 0 2 0 .
020
000
Which has x1 and x2 as pivot columns. While
100
AT = 1 0 2 ,
100
has x1 and x3 as pivot columns.
1 1
1 1
Problem 31
If A = [II ], then the nullspace for A is
is
I
I
I
I
. If B =
II
, then the nullspace for B
00
. If C = I , then the nullspace for C is 0.
Problem 32
x = (2, 1, 0, 1) is four dimensional so n = 4. The nullspace is a single vector so n r = 1 or
4 r = 1 giving that r = 3 so we have three pivots appear.
Problem 33
23
We must have RN = 0. If N = 1 0 , then let R = 1 2 3 . The nullity has
01
dimension of two and n = therefore using n r = 2, we see that r = 1. Thus we have only
3
0
one nonzero in R. If N = 0 the nullity is of dimension one and n = 3 so from n r = 1
1
we conclude that r = 2. Therefore we have two nonzero rows in R.
100
010
R=
.
If N = [], we assume that this means that the nullity is the zero vector only. Thus the nullity
is of dimension zero and n = 3 still so n r = 0 means that r = 3 and have three nonzero
rows in R
100
R= 0 1 0 .
001
Problem 34
Part (a):
R=
10
00
,
11
00
,
10
01
,
01
00
00
00
,
Part (b):
100
,
010
,
001
,
.
and
000
,111
,
and
110
,
101
,
011
.
They are all in reduced row echelon form.
Section 3.3 (The Rank and the Row Reduced Form)
Problem 1
Part (a): True
Part (b): False
Part (c): True
Part (d): False
Problem 5
AB
, then B is the r xr identity matrix, C = D = 0 and A is a r by n r
CD
matrix of zeros, since if it was not we would make pivot variables from them. The nullspace
I
is given by N =
.
0
If R =
Problem 13
Using the expression proved in Problem 12 in this section we have that
rank(AB ) rank(A) .
By replacing A with B T , and B with AT in the above we have that
rank(B T AT ) rank(AT ) .
Now since transposes dont aect the value of the rank i.e. rank(AT ) = rank(A), by the
above we have that
rank(B T AT ) = rank((AB )T ) = rank(AB ) rank(AT ) = rank(A)
proving the desired equivalence.
Problem 14
From problem 12 in this section we have that rank(AB ) rank(A) but AB = I so
rank(AB ) = I = n
therefore we have that n rank(A), so equality must hold or rank(A) = n. A then is
invertible and B must be its two sided inverse i.e. BA = I .
Problem 15
From problem 12 in this section we know that rank(AB ) rank(A) 2, since A is 2x3.
This means that BA cannot equal the identity matrix I , which has rank 3. An example of
such matrices are
10
101
A=
and B = 0 1
010
00
Then BA is
10
BA = 0 1
00
101
010
101
= 0 1 0 =I.
000
Problem 16
Part (a): Since R is the same for both A and B we have
A = E1 1 R
B = E2 1 R
for two elementary elimination matrices E1 and E2 . Now the nullspace of A is equivalent to
the nullspace of R (they are related by an invertible matrix E1 ), thus A and R have the same
nullspace. This can be seen to be true by the following argument. If x is in the nullspace of
A then
Ax = 0 = E1 1 Rx
so multiplying by E1 on the left we have
Rx = E1 0 = 0
proving that x is in the nullspace of R. In the same way if x is in the nullspace of R it must
be in the nullspace of A. Therefore
nullspace(A) = nullspace(B )
The fact that E1 A = R and E2 A = R imply that A and B have the same row space. This
is because E1 and E2 perform invertible row operations and as such dont aect the span of
the rows. Since
E1 A = R = E2 B
each matrix A and B has the same row space.
Part (b): Since E1 A = R = E2 B we have that A = E1 1 E2 B and A equals an invertible
matrix times B .
Problem 17
We rst nd the rank
1
1
A=
1
of the matrix A,
10
110
110
110
1 4 0 0 4 0 0 1 0 0 1 ,
18
008
001
000
from which we can see that A has rank 2. The elimination matrices used in this process were
1 00
10
0
100
E21 = 1 1 0 D = 0 1/4 0 E33 = 0 1 0
1 0 1
0 0 1/ 8
0 1 1
so
110
E33 DE21 A = R = 0 0 1
000
Then A can be reconstructed as
100
100
100
A = E211 D 1 E331 R = 1 1 0 0 4 0 0 1 0 R
101
008
011
100
100
1 4 0 0 1 0 R
=
108
011
100
1 4 0 R = E 1 R
=
188
Then A can be written by taking the rst r = 2 columns of E 1 and the rst r = 2 rows of
R giving
10
1 4 1 1 0
001
18
Our results we can check as follows
1
1 1 1 0
=
1
110
= 1 1 0 +
110
0
+ 4 0 0
8
000
1
= 1
004
008
1
1
10
1 4
18
The above is the sum of two rank one matrices. Now for B = [A A], concatenating the matrix
A in this way does not change the rank. Thus the (COL)((ROW )T decomposition would
take the rst r = 2 columns of E 1 with the rst r = 2 rows of R . When we concatenate
matrices like this we nd the reduced row echelon form for B to be that for A concatenated
i.e.
RB = [R R] ,
and the elimination matrix is the same. Thus
1
1
1
our two columns of E 1 are the same
0
4
8
and our two rows of RB are the concatenation of the two rows in R or
110110
110110
As before one can verify that
10
[A A] = 1 4
18
110110
110110
Section 3.4 (The Complete Solution to Ax = b)
Problem 1
Let our augmented matrix A be,
then with
1 3 31
A= 2 6 9 5
1 3 3 5
E21
1 00
= 2 1 0
1 01
we have
1331
E21 A = 0 0 3 3
0066
continuing by dividing by the appropriate pivots and eliminating the elements below and
above each pivot we have
1 3 0 5
1 3 0 5
1331
E21 A = 0 0 3 3 0 0 0 0 0 0 1 0
0066
001 1
000 0
From this expression we recognize the pivot variables of x1 and x3 . The particular solution
is given by x1 = 5, x2 = 0, and x3 = 1. A homogeneous solution, is given by setting the
free variable x2 , equal to one and solving for the pivot variables x1 , and x3 . When x2 = 1
we have the system
x1
x3
10
01
=
5
1
3
0
=
8
0
,
so x1 = 8 and x3 = 0. Thus our total solution is given by
5
8
x = 0 + x2 1
1
0
Problem 2
Our system is given by
x
1
1312
2 6 4 8 y = 3
z
1
0024
t
Let our augmented system be
1312
2 6 4 8
[A|b] =
0021
1312
0 0 1 2
0012
1300
0 0 1 2
0000
1
1
0
1
4
0
1
1/2
1/2
1/2
1/2 .
0
3121
0 2 4 1
0241
1312 1
0 0 1 2 1/2
0000 0
Which we see has rank 2. Thus since n = 4 the dimension of the null space is 2. The pivot
variables are x1 and x3 , and the free variables are x2 and x4 . A particular solution can be
obtained by setting x2 = x4 = 0 and solving for x1 and x3 . Performing this we have the
system
10
x1
1/2
=
01
x3
1/2
so our particular solution is given by
1/2
0
xp =
1/2 .
0
Now we have two special solutions to nd for Ax = 0.
Problem 10
Part (a): False. The combination c1 xp + c2 xn is not a solution unless c1 = 1. E.g.
A(c1 xp + c2 xn ) = c1 Axp + c2 Axn = c1 b = b
Part (b): False. The system Ax = b has an innite number of particular solutions (if
A is invertible then there is only one solution). For a general A this particular solution
corresponds to a point on the space obtained by assigning values to the free variables.
Normally, the zero vector is assigned to the free variables to obtain one particular solution.
Any other arbitrary vector maybe assigned in its place.
Part (c): False. Let our solution be constrained to lie on the line passing through the points
(0, 1) and (1, 0), given by the equation x y = 1. As such consider the system
1 1
2 2
x
y
=
1
2
,
this matrix has the row reduced echelon form of
R=
1 1
00
,
thus x is a pivot variable and y is a free variable. Setting the value of y = 0 gives the
particular solution x = 1, which has norm ||xp || = 1. A point on this line exists that is
closer to the origin, however, consider
||xp ||2 = x2 + y 2 = x2 + (x + 1)2
or the norm of all points on the given line. To minimize this take the derivative with respect
to x and set this expression equal to zero,
||xp ||2
= 2x + 2(x + 1) = 0 .
dx
1
Which has a solution given by x = 1 and y = 2 . Computing the norm at this point we
2
have
1
11
||xp ||2 = + = < 1 ,
44
2
which is smaller than what was calculated before. Thus showing that selecting the free
variables set to zero does not necessary give a minimum norm solution.
Part (d): False. The point xn = 0 is always in the nullspace. It happens that if A is
invertible x = 0 is the only element of the nullspace.
Section 3.6 (Dimensions of the Four Subspaces)
Problem 3 (from ER nd basis for the four subspaces)
Since we are given A in the decomposition ER we can begin by reading the rank of A from R
which we see is two since R has two independent rows. We also see that the pivot variables
are x2 and x4 while the free variables are x1 , x3 , and x5 . Thus a basis for the column space
is given by taking two linearly independent column vectors from A. For example, we can
take
1
3
1 and 4 ,
0
1
as a basis for the column space. A basis for the row space is given by two linearly independent
rows. Two easy rows to take are the rst and the second. Thus we can take
0
0
0
1
2 and 0 ,
1
3
2
4
as a basis for the row space. A basis for the nullspace is given by nding the special solution
when the free variables are sequentially assigned ones and then solving for the pivot variables.
For example our rst element of the nullspace is given by letting (x1 , x3 , x5 ) = (1, 0, 0), and
solving for (x2 , x4 ). We nd x2 = 0 and x4 = 0 giving the rst element in the nullspace of
1
0
0.
0
0
Our second element of the nullspace is given by letting (x1 , x3 , x5 ) = (0, 1, 0), and solving
for (x2 , x4 ). We nd x2 = 2 and x4 = 0 giving the second element in the nullspace of
0
2
1 .
0
0
Finally, our third element of the nullspace is given by letting (x1 , x3 , x5 ) = (0, 0, 1), and
solving for (x2 , x4 ). We nd x2 = 0 and x4 = 1 giving the third element in the nullspace of
0
0
0 .
1
1
These three vectors taken together comprise a basis for the nullspace. A basis for the left
nullspace can be obtained by the last = 3 minusr = 2 (or one) rows of E 1 . Since
m
100
1
00
E = 1 1 0 , we have that E 1 = 1 1 0 from which we see that the last row
011
0 1 1
1
of E is given by
1
1 .
1
We can check that this element is indeed in the left nullspace of A by computing v T A. We
nd that
01234
1 1 1 0 1 2 4 6 = 0 0 0 0 0 ,
00012
as it should.
Problem 4
Part (a): The matrix
10
1 0,
01
has the two given vectors as a column space and since the row space is R2 both
2
.
5
1
2
and
Part (b): The rank is one (r = 1) and the dimension of the nullspace is one. Since the rank
plus the dimension of the nullspace must be n we see that n = 1 + 1 = 2. The number of
components in both the column space vectors and the nullspace vector is three, which is not
equal to two, we see that this is not possible.
Part (c): The dimension of the nullspace n r equals one plus the dimension of the left
nullspace or 1 + (m r ) which must be held constant. We see that we need a matrix with
a rank of one, m = 1, and n = 2. Lets try the matrix
A=
12
.
Which has m = 1, r = 1, and n = 2 as required. The dimenion of the nullity is 2 1 = 1
and the dimension of the left nullspace is 1 1 = 0 as required, thus everything is satised.
Part (d): Consider
1
3
31
a
=
b
1
Thus a = 1 and b = 3 so the matrix A =
3 + 3a 1 + 3b
3
1
1 1
3
= 0.
satises the required conditions.
Part (e): If the row space equals the column space then m = n. Then since the dimension
of the nullspace is n r and the dimension of the left nullspace is also n r then these
two spaces have equal dimension and dont contain linearly independent rows (equivalently
columns).
Problem 5
Let V =
110
. For B to have V as its nullspace we must have
210
1
2
0 = 0 and B 1 = 0 .
B
1
0
Which imposes two constraints on B . We can let B = 1 a b
requires that
1
1 = 1+a+b = 0,
B
1
and the second constraint requires that
2
B 1 = 2+a = 0,
0
then the rst condition
or a = 2, which when used in the rst constraint gives that b = (1 + a) = 1. Thus our
matrix B is given by
1
2 .
1
Problem 6
Now A has rank two, m = 3, and n = 4. The dimension of its column space is two. The
dimension of its row space is two, the dimension of its nullspace is n r = 2. The dimension
of its left nullspace is m r = 3 2 = 1. To nd basis for each of these spaces we simply
need to nd enough linearly independent vectors. For the column space we can take the
vectors
3
3
0 and 0 .
1
0
For the row space pick
0
3
3
3
For the left nullspace pick
0
1
.
0
1
and
0
1.
0
For B we have r = 1, m = 3, and n = 1. The dimension of its column space is one. The
dimension of its row space is one, the dimension of its nullspace is n r = 0. The dimension
of its left nullspace is m r = 2. For the column space we can take a basis given by the
span of
1
4.
5
For the row space pick
1
.
For the left nullspace pick the empty set (or only the zero vector). For the left nullspace pick
4
5
1 and 0 .
0
1
Problem 7
For A we have m = n = r = 3 then the dimension of the column space is three and has a
basis given by
1
0
0
0, 1, 0.
0
0
1
The dimension of the row space is also three and has the same basis. The dimension of the
nullspace is zero and contains on the zero vector. The dimension of the left nullspace is zero
and contains only the zero vector.
For b we have m = 3, n = 6, and r = 3 then the dimension of the column space is three and
has the same basis as above. The dimension of the row space is still three and has a basis
given by
0
0
1
0
1
0
0
, 0, 1.
0
0
1
0
1
0
1
0
0
The dimension of the nullspace is 6 3 = 3 and a basis can be obtained from
0
0
1
0
1
0
1
0
0
1 , 0 , 0 .
0
1
0
1
0
0
The dimension of the left nullspace is m r = 3 3 = 0 and contains only the zero vector.
Problem 8
For A we have m = 3, n = 3 + 2 = 5, and r = 3. Thus
dim((C )(A))
dim((C )(AT ))
dim((N )(A))
dim((N )(AT ))
=
=
=
=
3
3
nr = 53 =2
m r = 0.
For B we have m = 3 + 2 = 5, n = 3 + 3 = 6, and r = 3. Thus
dim((C )(A))
dim((C )(AT ))
dim((N )(A))
dim((N )(AT ))
=
=
=
=
3
3
nr =3
m r = 5 3 = 2.
For C we have m = 3, n = 2, and r = 0. Thus
dim((C )(A))
dim((C )(AT ))
dim((N )(A))
dim((N )(AT ))
=
=
=
=
0
0
nr =2
m r = 3.
Problem 9
Part (a): First lets consider the equivalence of the ranks. The rank of A alone is equivalent
A
because we can simply subtract each row of A from the correto the rank of B
A
sponding newly introduced row in the concatenated matrix B . Eectively, this is applying
the elementary transformation matrix
E=
I0
I I
A
to produced
A
we can again multiply by E above obtaining
to the concatenated matrix
EC =
I0
I 0
,
A
. Now for the matrix C
0
AA
AA
=
AA
00
AA
AA
.
RR
00
where R is the reduced row echelon matrix for A. Since this has the same rank as R the
composite matrix has the same rank as the original. If A is m by n then B is 2m by n and
A and B have the same row space and the same nullity.
Continuing to perform row operations on the top half of this matrix we can obtain
Part (b): If A is m by n then B is 2m by n and C is 2m by 2n. Then B and C have the
same column space and left nullspace.
Problem 10
If a matrix with m = 3 and n = 3 with random entries it is likely that the matrix will be
non-singular so its rank will be three and
dim((C )(A))
dim((C )(AT ))
dim((N )(A))
dim((N )(AT ))
=
=
=
=
3
3
0
0.
If A is three by ve then m = 3 and n = 5 it is more likely that dim((C )(A)) = 3 and
dim((C )(AT )) = 3, while dim((N )(A)) = n r = 2, and dim((N )(AT )) = m r = 3 3 = 0.
Problem 11
Part (a): If there exits a right hand side with no solution then when we perform elementary
row operations on A we are left with a row of zeros in R (or U ) that does not have the
corresponding zero elements in Eb. Thus r < m (since we must have a row of zeros). As
always r m.
Part (b): Because letting y be composed of r zeros stacked atop vectors with ones in each
component i.e. in the case r = 2 and m = 4 consider the vectors
0
0
0
0
y1 = and y2 = .
0
1
1
0
T
T
Then y1 R = 0 and y2 R = 0 so that y T (EA) = 0 or equivalently (E T y )T A = 0. Therefore
T
E y is a nonzero vector in the left nullspace. Alternatively if the left nullspace is nonempty
it must have a nonzero vector. Since the left nullspace dimension is given by m r which
we know is greater than zero we have the existence of a non-zero element.
Problem 12
Consider the matrix A which I construct by matrix multiplication as
11
221
101
A= 0 2
= 2 4 0 .
120
10
101
If (1, 0, 1) and (1, 2, 0) are a basis for the row space then dim(AT ) = 2 = r . To also be a
basis for the nullspace means that n r = 2 implying that n = 4. But these are vectors in
R3 resulting in a contradiction.
Problem 13
Part (a): False. Consider the matrix
A=
Then the row space is spanned by
1
0
10
20
.
and the column space by
1
2
which are dierent.
Part (b): True. A is a trivial linear transformation of A and as such cannot alter the
subspaces.
Part (c): If A and B share the same four spaces then E1 A = R and E2 B = R and we see
that A and B are related by a linear transformation i.e. A = E1 1 E2 1 B . As an example pick
A=
10
02
and B =
20
03
.
Then the subspaces are the same but A is not a multiple of B .
Problem 14
The rank of A is three and a basis for
10
6 1
98
and
and lastly
the column space is given by
0
1
1
0 0 = 6 ,
1
0
9
100
2
1
0
2
6 1 0 1 = 2 6 + 1 = 13 ,
981
0
9
8
26
100
3
6 1 0 2 =
981
1
Equivalently the three by three block composing the rst three pivots of U is invertible so
that an additional basis can be taken from the standard basis. A basis for the row space of
dimension three is given by
(1, 2, 3, 4) ,
(0, 1, 2, 3) ,
(0, 0, 1, 2) .
Problem 15
The row space and the left nullspace will not change. If v = (1, 2, 3, 4) is in the column space
of the original matrix the vector in the column space of the new matrix is (2, 1, 3, 4).
Problem 16
If v = (1, 2, 3) was a row
of
1
x
.
.
.
x
of A then
23
x x
. .
. .
..
xx
when we multiply by v this row
14
1 2 + 22 + 32
1
x
x
2 =
= .
.
.
.
.
.
3
x
x
would give the product
,
which cannot equal zero.
Problem 17
010
For the matrix A given by A = 0 0 1 , our matrix is rank two. The column span is
000
x
0
x
all vectors y , the row space is all vectors y , the nullspace is all vectors 0 , and
0
0
z
0
110
nally, the left nullspace is all vectors 0 . For the matrix I + A = 0 1 1 . The
z
001
x
rank is three and the row space is given by all vectors y , the column space is all vectors
z
x
y , and the left nullspace and the nullspace both contain only the zero vector.
z
Problem 18
We have
1 2 3 b1
12
3
b1
A b = 4 5 6 b2 0 3 6 b2 4b1 ,
7 8 9 b3
0 6 12 b3 7b1
1 00
4 1 0 . This matrix then reduces to
using the elimination matrix E1 =
7 0 1
12
3
b1
12
3
b1
0 3 6
= 0 3 6
b2 4b1
b2 4b1 ,
0 0 3 b3 7b1 2(b2 4b1 )
0 0 3 b3 2b2 + b1
100
using the elimination matrix E2 = 0 1 0 . The combination of the rows that produce
0 2 1
the zero row is given by one times row one, minus two times the second row, one times the
third row. Thus the vector
1
2
1
is in the null space of AT . A vector in the nullspace is given by setting x3 = 1 and solving
for x1 and x2 . This gives the equation x1 + 2(2) + 3(1) = 0 or x1 = 4 3 = 1. The vector
then is
1
2
1
which is the same vector space as the left nullspace.
Problem 19
Part (a): Reducing our
1
3
4
matrix to upper
2 b1
4 b2
6 b3
triangular form we have
12
b1
0 2 b2 3b1
0 2 b3 4b1
12
b1
b2 3b1
0 2
0 0 b3 4b1 b2 + 3b1
12
b1
b2 3b1 .
= 0 2
0 0 b3 b2 b1
Thus the vector (1, 1, 1) is in the left nullspace which has a dimension given by m r =
3 2 = 1.
Part (b): Reducing our
1
2
2
2
matrix to upper triangular form we have
12
b1
2 b1
3 b2
0 1 b2 2b1
0 0 b3 2b1
4 b3
0 1 b4 2b1
5 b4
12
b1
0 1
b2 2b1
0 0
b3 2b1
0 0 b4 2b1 + b2 2b1
12
b1
0 1
b2 2b1
.
=
0 0
b3 2b1
0 0 b4 + b2 4b1
Thus the vectors in the left nullspace are given by
4
2
0
and 1 ,
0
1
1
0
which has a dimension of m r = 4 2 = 2.
Problem 20
Part (a): We must have Ux = 0 which has two pivot variables x1 and x3 and free variables
x2 and x4 . To nd the nullspace we set (x2 , x4 ) = (1, 0) and solve for x1 and x3 . Thus we
get 4x1 + 2 = 0 or x1 = 1 which gives a vector in the nullspace of
2
1
2
1
3 .
0
Now setting (x2 , x4 ) = (0, 1) and solving for x1 and x3 we need to solve 4x1 + 2(0) + 0 + 1 = 0
or x3 = 3 which gives a vector in the nullspace of
1
4
0
3 .
1
Part (b): The number of independent solutions of AT y are given by m r = 3 2 = 1
Part (c): The column space is spanned by
100
4
4
2 1 0 0 = 8 ,
341
0
12
and
100
0
0
2 1 0 1 = 1 .
341
0
4
Problem 21
Part (a): The vectors u and w .
Part (b): The vectors v and z .
Part (c): u and w are multiples of each other or are linearly dependent or v and z are
multiples of each other or are linearly dependent.
Part (d): u = z = (1, 0, 0) and v = w = (0, 0, 1). Then
0
000
uv T = 1 0 0 0 = 0 0 0
1
100
and
wz T =
So that
001
001
1
0 = 0 0 0
000
0
001
A = uv T + wz T = 0 0 0 ,
100
which has rank two.
Problem 22
Consider A decomposed as
A=
=
=
=
12
2 2
41
1
2 1
4
100
2 0 0
400
122
2 2 2
411
100
011
2
0 0 + 2 0 1 1
1
022
+ 0 2 2
011
Problem 23
A basis for the row space is (3, 0, 3) and (1, 1, 2) which are independent. A basis for the
column space is given by (1, 4, 2) and (2, 5, 7) which are also independent. A is not invertible
because it is the product of two rank two matrices and therefore rank(AB ) rank(B ) = 2.
To be invertible we must have rank(AB ) = 3 which it is not.
Problem 24
d is in the span of its rows. The solution is unique when the left nullspace contains only the
zero vector.
Problem 25
Part (a): A and AT have the same number of pivots. This is because they have the same
rank they must have the same number of pivots.
Part (b): False. Let A =
10
, then y T =
20
y T AT =
2 1
12
00
2 1
=
is in the left nullspace of A but
2 4
= 0,
and therefore is not in the left nullspace of AT .
Part (c): False. Pick an invertible matrix say of size m by m then the row and column
spaces are the entirety of Rm . It is easy to imagine an invertible matrix such that A = AT .
12
.
For example let A =
34
Part (d): True, since if AT = A then the rows of A are the negative columns of A and
therefore have exactly the same span.
Problem 26
The rows of C are combinations of the rows of B . The rank of C cannot be greater than the
rank of B , so the rows of C T are the rows of AT , so the rank of C T (which equals the rank
of C ) cannot be larger than the rank of AT (which equals the rank of A).
Problem 27
To be of rank one the two rows must be multiples of each other and the two columns must
be multiples of each other. To make the rows multiples of each other assume row two is a
c
multiple (say k ) of row one i.e. ka = c and kb = d. Thus we have k = a and therefore
a
c
and a basis for the
d = a b. A basis for the row space is then given by the vector
b
nullspace is given by
b
b
a
.
a
1
Problem 28
The rank of B is two and has a basis of the row space given by the rst two rows in its
representation, The reduced row echelon matrix looks like
10101010
0 1 0 1 0 1 0 1
0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0
00000000
which is obtained by EA = R where E
1
0
1
0
E=
1
0
1
0
is given by
0
1
0
1
0
1
0
1
0
0
1
0
0
0
0
0
0
0
0
1
0
0
0
0
0
0
0
0
1
0
0
0
0
0
0
0
0
1
0
0
0
0
0
0
0
0
1
0
0
0
0
0
0
0
0
1
Chapter 4 (Orthogonality)
Section 4.1 (Orthogonality of the Four Subspaces)
Problem 1
For this problem A is 2x3 so m = 2 and n = 3 and r = 1. Then the row and column space
has dimension 1. The nullspace of A has size n r = 3 1 = 2. The left nullspace of A has
size m r = 2 1 = 1.
Problem 2
For this problem m = 3, n = 2, and r = 2. So the dimension of the nullspace of A is given
by 2 2 = 0, and the dimension of the left nullspace of A is given by 3 2 = 1. The two
components of x are xr which is all of R2 and xn which is the zero vector.
Problem 3
Part (a): From the given formulation we have that m = 3 and n = 3, obtained from the
size (number of elements) of the column and nullspace vectors respectively. Then n r =
3 r = 1, we have a r = 2. This matrix seems possible and to obtain it, consider a matrix
A as
1
2 3
A = 2 3 1 ,
3 5 2
which will have the requested properties.
Part (b): From the denition of the vectors in the row space we have m = 3, and r = 2
since there are only two vectors in the row space. Then the size of the nullspace imply that
n r = n 2 = 1, so n = 3. Having the dimensions worked out we remember that for all
matrices, the row space must be orthogonal to the nullspace. Checking for consistency in
this example we compute these inner products. First we have
1
1 =0
1 2 3
1
which holds true but the second requirement
1
2 3 5 1 = 4 ,
1
is not equal to zero, so the required matrix is not possible.
Part (c): To see if this might be possible let x be in the nullspace of A. Then to also be
perpendicular to the column space requires AT x = 0. So A and AT must have the same
nullspace. This will trivially be true if A is symmetric. Also we know that A cannot be
invertible since the nullspace for A and AT would then be trivial, consisting of only the zero
vector. So we can try for a potential A the following
42
21
A=
Then N(A) = N(AT ) is given by the span of the vector
1
2
,
which by construction is perpendicular to every column in the column space of A.
T
Part (d): This is not possible since from the statements given the vector 1 1 1
must
be an element of the left nullspace of our matrix A and as such is orthogonal to every element
T
of the column space of A. If the column space of A contains the vector 1 2 3
then
checking orthogonality we see that
1
1 1 1 2 =6
3
and the two vector are not orthogonal.
Part (e): The fact that the columns of add to the zero column means that the vector of all
ones must be in the nullspace of our matrix. We can see if a two by two matrix of this form
exists. We rst investigate if we can construct a 2x2 example matrix that has the desired
properties. The rst condition given is that
ab
cd
1
1
=0
or in equations
a+b = 0
c+d = 0
The second condition is that
11
ab
cd
=
11
or
a+c b+d
=
11
.
(3)
So our total system of requirements on our unknown 2x2 system A is given by
a+b
c+d
a+c
b+d
which in matrix form is given by
1
0
1
0
Performing
110
0 0 1
1 0 1
010
1
0
0
1
0
1
1
0
=
=
=
=
0
0
1
1
a
0
1 b
0 c
d
1
0
0
= .
1
1
row reduction on the augmented matrix we
10
1 1 000
00
0 0
0 0 1 1 0
1 0
0 1 1 0 1 0 1
0 1
00
0 1 011
11
have
1
1
1
1
0
1
0
1
10
0
0 1
0
0 0
1
00
2
1
1
1
1
0
0
1
1
1
1
.
0
2
Since the last two equations contradict each other, one can conclude that this is not possible.
Another way to see this same result is to notice that a row of all ones will be in the nullspace
but also in the row space. Since the only vector in both these spaces must be the zero vector,
we have a contradiction, showing that no such matrix exists.
Problem 4 (can the row space contain the nullspace)
It is not possible for the row space to contain the nullspace. To show this let x = 0 be a
member of both, then from the second fundamental theorem of linear algebra (that the row
space and the nullspace are orthogonal) we have xT x = 0, which is not true unless x = 0.
Problem 5
Part (a): We have that y is perpendicular to b, since b is in the column space of A and y
is in the left nullspace.
Part (b): If Ax = b has no solution, then b is not in the column space of A and therefore
ybT = 0 and y is not perpendicular to b.
Problem 6
If x = xr + xn , then Ax = Axr + Axn = Axr + 0 = Axr . So x is in the column space of A
because Axr is a linear combination of the columns of A.
Problem 7
For Ax to be in the nullspace of AT , it must be in the left nullspace of A. But Ax is in
the column space of A and these two spaces are orthogonal. Because Ax is in both spaces it
must be the zero vector.
Problem 8
Part (a): For any matrix A, the column space of is perpendicular to its left nullspace. By
the symmetry of A the left nullspace of A is the same as its nullspace.
Part (b): If Ax = 0 and Ax = 5z , then z T AT = 5z T or z T Ax = 5z T x. Since Ax = 0, we
have that 5z T x = 0 or z T x = 0. In terms of subspaces, x is in the nullspace and the left
nullspace of A, while z is in the column space of A and therefore since the column space and
the left nullspace are perpendicular we must have that x and z perpendicular.
Problem 9
The matrix
A=
12
36
,
has rank one. A row space given by the span of [1, 2]T , a column space given by the span
of [1, 3]T , a nullspace given by [2, 1]T , and a left nullspace given by the span of [3, 1]T .
With these vectors Figure 4.2 from the book would look like that seen in Figure XXX. We
can verify the mapping properties of the matrix A by selecting a nonzero component along
the two orthogonal spaces spanning the domain of A (its row space and its nullspace). For
example, take xn = [1, 2]T , and xr = [2, 1]T , two be vectors in the nullspace and row space
of A respectively then dene x xn + xr = [3, 1]T . We compute that
Ax =
12
36
3
1
=
5
15
and as required
Axn =
12
36
2
1
=
0
0
The matrix
B=
and Axr =
10
30
12
36
1
2
=
5
15
.
,
has rank one. A row space given by the span of [1, 0]T , a column space given by the span of
[1, 3]T , a nullspace given by [0, 1]T , and nally a left nullspace given by the span of [3, 1]T .
With these vectors Figure 4.2 from the book would look like that seen in Figure XXX. We
can verify the mapping properties of the matrix B by selecting a nonzero component along
the two orthogonal spaces spanning the domain of B (its row space and its nullspace). For
example, take xn = [0, 2]T , and xr = [1, 0]T , be two vectors in the nullspace and row space
of B respectively then dene x xn + xr = [1, 2]T . We compute that
10
30
Bx =
1
2
1
3
=
and as required (the component in the direction of the nullspace contributes nothing)
Bxn =
0
2
10
30
0
0
=
and Bxr =
10
30
1
2
=
1
3
.
Problem 10 (row and nullspaces)
The matrix
1 1
A= 0 0 ,
00
has rank two. A row space given by the span of [1, 1]T , a column space given by the span
of [1, 0, 0]T , a nullspace given by [1, 1]T , and a left nullspace given by the span of [0, 1, 0]T
and [0, 0, 1]T . With these vectors Figure 4.2 from the book would look like that seen in
Figure XXX. We can verify the mapping properties of the matrix A by considering the
vector x provided. Since x has components along the two orthogonal spaces spanning the
domain of A (its row space and its nullspace) we have, since xn = [1, 1]T , and xr = [1, 1]T .
We compute that
2
1 1
0 0 2 = 0
Ax =
0
0
00
and as required
1 1
Axn = 0 0
00
1
1
=
0
0
1 1
and Axr = 0 0
00
1
1
2
= 0 .
0
Problem 11
Let y N (AT ), then AT y = 0, now y T Ax = (y T Ax)T , since y T Ax is a scalar and taking the
transpose of a scalar does nothing. But we have that (y T Ax)T = xT AT y = xT 0 = 0, which
proves that y is perpendicular to Ax.
Problem 12
The Fredholm alternative is the statement that exactly one of these two problems has a
solution
Ax = b
AT y = 0 such that bT y = 0
In words this theorem can be stated that either b is in the column space of A or that there
exists a vector in the left nullspace of A that is not orthogonal to b. To nd an example
where the second situation holds let
A=
10
20
and b =
2
1
Then Ax = b has no solution (since b is not in the column space of A). We can also show
this by considering the augmented matrix [A b] which is
102
201
10 2
0 0 3
,
since the last row is not all zeros, Ax = b has no solution. For the second part of the
Fredholm alternative, we desire to nd a y such that AT y = 0 and bT y = 0. Now AT y is
given by
0
y1
12
=
0
y2
00
Then we have that the vector y can be any multiple of the vector [2 1]T . Computing bT y
we have bT y = 2(2) + 1(1) = 3 = 0, and therefore the vector y = [2, 1]T is a solution to
the second Fredholms alternative.
Problem 13
If S is the subspace with only the zero vector then S = R3 . If S = span{(1, 1, 1)} then S
is all vectors y such that
1
T
1 =0
y
1
or y1 + y2 + y3 = 0. Equivalently the nullspace of the matrix A dened as
A=
111
which has a nullspace given by the span of y1 and y2
1
1
y1 = 1 and y2 = 0
0
1
If S is spanned by the two vectors [2, 0, 0]T and [0, 0, 3]T , then S consists of all vectors y
such that
2
0
y T 0 = 0 and y T 0 = 0
0
3
So 2y1 = 0 and 3y3 = 0 which imply that y1 = y3 = 0, giving S = span{[0, 1, 0]T }.
Problem 14
S is the nullspace of
A=
151
222
Therefore S is a subspace of A even if S is not.
Problem 15
L is the plane perpendicular to this line. Then (L ) is a line perpendicular to L , so
(L ) is the same line as the original.
Problem 16
V contains only the zero vector. Then (V ) contains all of R4 , and (V ) is the same as
V.
Problem 17
Suppose S is spanned by the vectors [1, 2, 2, 3]T and [1, 3, 3, 2]T , then S is spanned by the
nullspace of the matrix A given by
A=
1223
1332
122 3
0 1 1 1
100 5
0 1 1 1
.
Which has a nullspace given by selecting a basis for the free variables x3 and x4 and then
solving for the pivot variables x1 and x2 . Using the basis [1, 0]T and [0, 1]T , if x3 = 1, x4 = 0,
then x1 = 0 and x2 = 1, while if x3 = 0 and x4 = 1 then x1 = 5 and x2 = 1 and in vector
form is spanned by
5
0
1
and 1 .
0
1
1
0
Problem 18
If P is the plane given then A = 1 1 1 1 has this plane as its nullspace. Then P are
composed of the the elements of the left nullspace of A i.e. the nullspace of AT . Since
1
1
0
1
AT =
0
1
0
1
Thus the nullspace of AT equivalently P is
0
1
,
0
0
given by the span of the vectors
0
0
0 0
,
1 0
1
0
Problem 19
We are asked to prove that if S V then S V . To do this, let y V . Then for every
element x V , we have xT y = 0. But we can also say that for every element x S it is also
in V by the fact that S is a subspace of V and therefore xT y = 0 so y S . Thus we have
V S .
Problem 20
The rst column of A1 is orthogonal to the span of the second through the last.
Problem 21 (mutually perpendicular column vectors)
AT A would be I .
Problem 22
AT A must be a diagonal matrix since it represents every column of A times every row of A.
When the two columns are dierent the result is zero. When they are the same the norm
(squared) of that column results.
Problem 23
The lines 3x + y = b1 and 6x + 2y = b2 are parallel. They are the same line if 2b1 = b2 . Then
[b1 , b2 ]T is perpendicular to the left nullspace of
A=
or
31
62
2
. Note we can check that this vector is an element of the left nullspace by computing
1
2 1
b1
b2
= 2b1 + b2 = 2b1 + 2b2 = 0
The nullspace of the matrix is the line 3x + y = 0. One vector in this nullspace is [1, 3]T .
Problem 24
Part (a): As discussed in the book if two subspaces are orthogonal then they can only meet
at the origin. But for the two planes given we have many intersections. To nd them we
solve the system given by
x
11 1
y = 0,
1 1 1
z
then the point (x, y, z ) will be on both planes. Performing row reduction we obtain
x
110
y =0
001
z
so
we see that z = 0 and x + y = 0, giving the fact that any vector that is a multiple of
1
1 is in both planes and these two spaces cannot be orthogonal.
0
Part (b): The two lines specied are described as the spans of the two vectors
2
1
4 and 3
5
2
respectively. For their subspaces to be
orthogonal, the subspace generating vectors must be
1
T
3 = 2 12 + 10 = 0 and they are orthogonal.
orthogonal. In this case 2 4 5
2
We still need to show that they are not orthogonal components. To do so it suces to nd
a vector orthogonal to one space that is not in the other space. Consider 2 4 5 , which
as a nullspace given by setting the free variables equal to a basis and solving for the pivot
variables. Since the free variables are x2 and x3 we have a rst vector in the nullspace given
by setting x2 = 1,x3 = 0, which implies that x1 = 2. Also setting x2 = 0, x3 = 1, we have
that x1 = 5 , giving two vector of
2
2
5/2
1 and 0
0
1
2
2
1 it is orthogonal to 4 and thus is in its orthogonal
Now consider the vector
0
5
1
complement. This vector however is not in the span of 3 . Thus the two spaces are
2
not the orthogonal complement of each other.
Part (c): Consider the subspaces spanned by the vectors
0
, and
1
1
, respectively.
1
They meet only at the origin but are not orthogonal.
Problem 25
Let
123
A= 2 4 5 ,
367
then A has [1 , 2 , 3]T in both its row space and its nullspace. Let B be dened by
1 1 1
B = 2 2 2 ,
3 3 3
then B has [1 , 2 , 3]T in the column space of B and
1
0
2 = 0 .
B
3
0
Now v could not be both in the row space of A and in the nullspace of A. Also v could not
both be in the column space of A and in the left nullspace of A. It could however be in the
row space and the left nullspace or in the nullspace and the left nullspace.
Problem 26
A basis for the left nullspace of A.
Section 4.2 (Projections)
Problem 1 (simple projections)
Part (a): The coecient of projection x is given by
aT b
1+2+2
5
=
=
Ta
a
1+1+1
3
x=
so the projection is then
T
p=a
ab
aT a
and the error e is given by
1
5
= 1
3
1
1
2
1
1
e= bp= 2 1 = 1 .
3
2
1
1
To check that e is perpendicular to a we compute eT a = 1 (2 + 1 + 1) = 0.
3
Part (b): The projection coecient is given by
x=
aT b
1 9 1
=
= 1 .
Ta
a
1+9+1
so the projection p is then
1
p = xa = a = 3 .
1
The error e = b p = 0 is certainly orthogonal to a.
Problem 2 (drawing projections)
Part (a): Our projection is given by
p = xa =
aT b
a = cos()
aT a
1
0
=
cos()
0
Part (b): From Figure XXX of b onto a is zero. Algebraically we have
p = xa =
aT b
a=
aT a
11
2
1
1
=
0
0
Problem 3 (computing a projection matrix)
T
Part (a): The projection matrix P equals P = aa a , which in this case is
aT
1
1 1 1 1
111
1
1
P=
= 1 1 1.
3
3
111
For this projection matrix note that
333
111
1
1
P2 = 3 3 3 = 1 1 1 = P .
9
3
333
111
The requested product P b is
111
1
5
1
1
Pb = 1 1 1 2 = 5 .
3
3
111
2
5
T
aa
Part (b): The projection matrix P equals P = aT a , which in
1
3 1 3 1
1
1
1
3
=
P=
1+9+1
11
1
For this projection
1
1
2
P= 2 3
11
1
matrix note
31
1
3
93
31
1
this case is
31
9 3.
31
that P 2 is given by
131
31
11 33 11
1
1
3 9 3 =P.
9 3 = 2 33 99 33 =
11
11
131
31
11 33 11
The requested product P b is then given by
131
1
11
1
1
1
3 9 3 3 =
33 = 3 .
Pb =
11
11
131
1
11
1
Problem 4 (more calculations with projection matrices)
Part (a): Our rst projection matrix is given by P1 =
P1 =
1
0
10
=
aaT
aT a
10
00
which in this case is
2
Calculating P1 we have that
2
P1 =
10
00
= P1 ,
as required.
Part (b): Our second projection matrix is given by P2 =
P2 =
1
2
1
1
1
4
2 2
2 2
1 1
=
1
2
aaT
aT a
which in this case is
1 1
1 1
2
Calculating P2 we have that
2
P2 =
=
1
2
1 1
1 1
= P2 ,
as required. In each case, P 2 should equal P because the action of the second application of
our projection will not change the vector produced by the action of the rst application of
our projection matrix.
Problem 5 (more calculations with projection matrices)
We compute for the rst project matrix P1 that
1
T
1
aa
2 1 2 2
P1 = T =
aa
(1 + 4 + 4)
2
and compute the second projection matrix P2 by
2
T
1
aa
2 2 2 1
P2 = T =
aa
(4 + 4 + 1)
1
1 2 2
1
4 ,
= 2 4
9
2 4
4
4
4 2
1
4 2 .
= 4
9
2 2 +1
With these two we nd that the product P1 P2 is then given by
4
4 2
1 2 2
1
2 4
4 4
4 2
P1 P2 =
81
2 4
4
2 2 +1
48+4
4 8 + 4 2 + 4 2
1
8 + 16 8 8 + 16 8 4 8 + 4 = 0 .
=
81
8 + 16 8 8 + 16 8 4 8 + 4
An algebraic way to see this same result is to consider the multiplication of P1 and P2 in
terms of the individual vectors i.e.
a1 aT a2 aT
1
2
T
T
a1 a1 a2 a2
1
1
a1 aT a2 aT
=T
1
2
T
a1 a1 a2 a2
1
1
a1 (aT a2 )aT = 0 ,
=T
1
2
T
a1 a1 a2 a2
P1 P2 =
since for the vectors given we can easily compute that aT a2 = 0. Conceptually this result
1
is expected since the vectors a1 and a2 are perpendicular and when we project a given
vector onto a1 we produce a vector that will still be perpendicular to a2 . Projecting this
perpendicular vector onto a2 will result in a zero vector.
Problem 6
From Problem 5 we have that P1
1
1
2
P1 =
9
2
and P2 given by
given by
2 2
4
4
4
4
4
4 2
1
4 2
P2 = 4
9
2 2 1
and nally P3 given by
P3 =
a3 aT
3
T
a3 a3
Then we have that
=
4
1
2
=
9
4
1
1
0 = 1 0
so P1
9
0
0
4
1
1
so P2 0 = 4
9
2
0
2
1
1 2 1 2
4+1+4
2
2 4
4
1
1
1 2 so P3 0 = 2 .
9
2 4
4
0
1+4+4
1
1
p1 + p2 + p3 = 2 + 4 2 = 0 .
9
2 2 + 4
0
We are projecting onto three orthogonal axis a1 , a2 , and a3 , since aT a1 = 2 2 + 4 = 0,
3
aT a2 = 4 2 2 = 0, and aT a2 = 2 + 4 2 = 0.
3
1
Problem 7
From Problem 6 above we have that P3 is given by
4 2 4
1
P3 = 2 1 2
9
4 2 4
So adding all three projection matrices we nd that
1 + 4 + 4 2 + 4 2 2 2 + 4
100
1
422 = 0 1 0 ,
P1 + P2 + P3 = 2 + 4 2 4 + 4 + 1
9
2 2 + 4 4 2 2
4+1+4
001
as expected.
Problem 8
We have
aT b
1
1
= 1 so p1 = x1 a1 =
T
0
a1 a1
T
ab
3
31
= T2 =
so p2 = x2 a2 =
a2 a2
5
52
x1 =
x2
This gives
p1 + p2 =
1
0
+
3
5
1
2
=
2
5
4
3
Problem 9
The projection onto the plane a1 and a2 is the full R2 so the projection matrix is the identity
I . Since A is a two by two matrix with linearly independent columns AT A is invertible. This
product is given by
11
10
11
AT A =
=
12
02
15
so that (AT A)1 is given by
(AT A)1 =
1
4
5 1
1 1
.
The product A(AT A)1 AT can be computed. We have
A(AT A)1 AT =
1
4
1
=
4
=
11
02
11
02
40
04
1
4
5 1
1 1
10
12
4 2
02
=I,
as claimed.
Problem 10
When we project b onto a the coecients are given by x =
would have coecients and a projection given by
1
aT a1
2
=
T
a2 a2
5
11
p = xa2 =
52
x=
.
aT b
,
aT a
so to project a1 onto a2 we
The projection matrix is given by P1 =
P1 =
1
5
a2 aT
2
aT a2
2
1
2
and equals
12
1
5
=
12
24
.
Then to project this vector back onto a1 we obtain a coecient and a projection given by
pT a1
11
1
=
=
T
a1 a1
51
5
11
p = xa1 =
.
50
x=
The projection matrix is given by P2 =
a1 aT
1
aT a1
1
and equals
P2 =
10
00
.
1
5
12
24
=
So that P2 P1 is given by
10
00
P2 P1 =
1
5
12
00
.
Which is not a projection matrix since it would have to be written proportional to a row
which it cant be.
Problem 11
Remembering our projection theorems AT Ax = AT b and p = Ax we can evaluate the various
parts of this problem.
Part (a): We nd that AT A is given by
100
110
AT A =
and AT b is given by
11
0 1 =
00
2
3 =
4
100
110
AT b =
11
12
2
5
,
.
With this information the system for the coecients x i.e. AT Ax = AT b is given by
11
12
x1
x2
=
2
5
which has a solution given by
x1
x2
=
1
1
2 1
1 1
2
5
=
1
3
.
so that p = Ax is given by
11
p = Ax = 0 1
01
1
3
2
= 3 .
0
With this projection vector we can compute its error. We
2
2
e=bp = 3 3 =
4
0
nd that e = b p is given by
0
0 .
1
Part (b): We have for AT A the following
AT A =
11
1 1 =
01
110
111
also we nd that AT b is given by
110
111
AT b =
4
4 =
6
22
23
8
14
.
.
So that our system of normal equations AT Ax = AT b, becomes
22
23
x1
x2
8
14
=
.
This system has a solution given by
x1
x2
=
1
2
3 2
2 2
8
14
=
2
6
With these coecients our projection vector p becomes
4
11
1 1 2 = 4 .
p = Ax =
6
6
01
and our error vector e = b p is then given by
4
4
4 4 = 0.
e=bp =
6
6
.
Problem 12
The projection matrix is given by P1 = A(AT A)1 AT . Computing P1 we nd that
P1
11
1
= 0 1
1
00
11
2
= 0 1
1
00
10
1
= 1 1
1
00
00
10
1
1
00
10
2
We can check that P1 = P1 as required by
100
1
2
0 1 0 0
P1 =
000
0
1
11
0 1
00
100
110
100
110
100
= 0 1 0 .
000
projection matrices. We have
00
100
1 0 = 0 1 0 = P1 .
00
000
Now consider P1 b from which we have
100
2
2
P1 b = 0 1 0 3 = 3 .
000
4
0
For the second part we again have P2 = A(AT A)1 AT , which is given by
11
1
3 2
110
P2 = 1 1
111
2 2 2
01
11
110
1
1
1 1 2
1 1
= 1 1 0.
=
00 2
2
2
01
002
2
Then P2 is given by
110
110
220
110
1
1
1
2
P2 = 1 1 0 1 1 0 = 2 2 0 = 1 1 0 = P2 .
4
4
2
002
002
004
002
Now consider P1 b from which we
1
1
1
P2 b =
2
0
have
10
4
8
4
4 = 1 8 = 4 .
10
2
02
6
12
6
Problem 13
100
0 1 0
With A =
0 0 1
000
T
computing A A. We
, we will compute the projection matrix A(AT A)1 AT . We begin by
nd that
Then
1
1000
0
AT A = 0 1 0 0
0
0010
0
0
1
0
0
0
100
0
= 0 1 0 .
1
001
0
1
0
0
1000
0
0 1 0 0 =
0
1
0010
0
0
1
2
So P is four by four and we have that P b = .
3
0
1
0
A(AT A)1 AT =
0
0
0
1
0
0
0
1
0
0
0
0
1
0
0
0
.
0
0
Problem 14
Since b is in the span of the columns of A the projection will be b itself. Also P = I since
for vectors not in the column space of A their projection is not themselves. As an example
let
01
A= 1 2 ,
20
Then the projection matrix is given by A(AT A)1 AT . Computing AT A we nd
01
012
52
1 2 =
.
AT A =
120
25
20
And then the inverse is given by
(AT A)1 =
1
21
5 2
2 5
.
Which gives for the projection matrix the following
01
1
5 2
P = 1 2
21 2 5
20
01
1
2 1 10
1 2
=
5 8 4
21
20
5 8 4
1
8 17 2 .
=
21
4 2 20
012
120
So that p = P b is the given by
5 8 4
0
0
0
1
2 = 1 42 = 2 = b .
8 17 2
p = Pb =
21
21
4 2 20
4
84
4
Problem 15
The column space of 2A is the same as that of A, but x is not the same for A and 2A since
pA = Ax and p2A = 2Ax while pA = p2A since the column space of A and 2A are the same
so the projections must be the same. Thus we have that
xA = 22A .
x
This can be seen by writing the equation for xA and x2A in terms of A. For example the
equation for xA is given by
AT AxA = AT b .
While that for x2A is given by
4AT Ax2A = 2AT b .
This latter equation is equivalent to AT A(22A ) = AT b. Comparing this with the rst
x
equation we see that xA = 22A .
x
Problem 16
11
We desire to solve for x in AT Ax = AT b. With A = 2 0 we have that
1 1
11
1 2 1
60
2 0 =
AT A =
.
10 1
02
1 1
So that x is then given by
x = (AT A)1 AT b
=
1
6
0
0
1
2
1 2 1
10 1
2
1 =
1
1
6
0
0
1
2
3
3
=
1
2
3
2
.
Problem 17 (I P is an idempotent matrix)
We have by expanding (and using the fact that P 2 = P ) that
(I P )2 = (I P )(I P ) = I P P + P 2 = I 2P + P = I P .
So when P projects onto the column space of A, I P projects onto the orthogonal complement of the column space of A. Or in other words I P projects onto the the left nullspace
of A.
Problem 18 (developing an intuitive notion of projections)
Part (a): I P is the projection onto the vector spanned by [1, 1]T .
Part (b): I P is the projection onto the plane perpendicular to this line, i.e. x + y + z = 0.
The projection matrix is derived from the column of
1
A= 1
1
which has x + y + z = 0 as its left nullspace.
Problem 19 (computing the projection onto a given plane)
Consider the plane given by x y 2z = 0, by setting the free variables equal to a basis
(i.e. y = 1; z = 0 and y = 0; z = 1) we derive the following two vectors in the nullspace
1
2
1 and 0 .
0
1
These are two vectors in the plane which we make into columns of A as
12
A= 1 0
01
with this denition we can compute AT A as
110
201
(AT A)1 =
AT A =
12
1 0 =
01
1
6
Then (AT A)1 is given by
5 2
2 2
22
25
.
,
and our projection matrix is then given by P = A(AT A)1 AT or
12
1
110
5 2
1 0
A(AT A)1 AT =
201
2 2
6
01
1
2
1
110
5 2
=
201
6
2 2
51
2
1
1 5 2 .
=
6
2 2 2
Problem 20 (computing the projection onto the same plane ... dierently)
A vector perpendicular to the plane x y 2z = 0 is the vector
1
e = 1
2
x
T
y = 0 for every x, y , and z in the plane. The projection onto this vector
since then e
z
is given by
Q=
eeT
eT e
1
1
1 1 1 2
=
1+1+4
2
1 1 2
1
1 1
2 .
=
6
2 2
4
Using this result the projection onto the given plane is given by I Q or
61
1
2
51
2
1
1
1
6 1 2 = 1 5 2 ,
6
6
2
2 6 4
2 2 2
which is the same as computed earlier in Problem 19.
Problem 21 (projection matrices are idempotent)
If P = A(AT A)1 AT then
P 2 = (A(AT A)1 AT )(A(AT A)1 AT ) = A(AT A)1 AT = P .
Now P b is in the column space of A and therefore its projection is itself.
Problem 22 (proving symmetry of the projection matrix)
Given the denition of the projection matrix P = A(AT A)1 AT , we can compute its transpose directly as
P T = (A(AT A)1 AT )T = A(AT A)T AT = A((AT A)T )1 AT = A(AT A)1 AT .
which is the same denition as P proving that P is a symmetric matrix.
Problem 23
When A is invertible the span of its columns is equal to the entire space from which we are
leaving i.e. Rn , so the projection matrix should be the identity I . Therefore, since b is in Rn
its projection into Rn must be itself. The error of this projection is then zero.
Problem 24
the nullspace of AT is perpendicular to the column space C (A), by the second fundamental
theorem of linear algebra. If AT b = 0, the projection of b onto C (A) will be zero. From the
expression for the projection matrix we can see that this is true because
P b = A(AT A)1 AT b = A(AT A)1 0 = 0 .
Problem 25
The projection P b ll the subspace S so S is the basis of P .
Problem 26
Since A2 = A, we have that A(A I ) = 0. But since the rank of A is m, A is invertible we
can therefore multiply both sides by A1 to obtain A I = 0 or A = I .
Problem 27
The vector Ax is in the nullspace of AT . But Ax is always in the column space of A. To be
in both spaces (since they are perpendicular) we must have Ax = 0.
Problem 28
From the information given P x is the second column of P . Then its length squared is given
by (P x)T (P x) = xT P T P x = xT P 2 x = xT P x = p22 , or the (2, 2) element in P .
Section 4.3 (Least Squares Approximations)
Problem 1 (basic least squares concepts)
If our mathematical model of the relationship between b and t is a line given by b = C + Dt,
then the four equations through the given points are given by
0
8
8
20
=
=
=
=
C +D0
C +D1
C +D3
C +D4
If the measurements change to what is given in the text then we have
1
5
13
17
=
=
=
=
C +D0
C +D1
C +D3
C +D4
Which has as an analytic solution given by C = 1 and D = 4.
Problem 2 (using the normal equations to solve a least squares problem)
For the b and the given points our matrix A is given by
10
1 1
and b =
A=
1 3
14
0
8
8
20
The normal equations are given by AT Ax = AT b, or
10
1111
1 1 1 1 1 1
1 3 = 0 1 3 4
0134
14
or
48
8 26
C
D
=
which has as its solution [C, D]T = [1, 4]T . So the
1
5
Ax =
13
17
With this solution by direct calculation the
1
0
8 1
e=
8 1
1
20
0
8
8
20
36
112
four heights with this x are given by
.
error vector e = b Ax is given by
1
0
3
1 1
=
5
3 4
3
4
The smallest possible value of E = 1 + 9 + 25 + 9 = 44.
Problem 3
1
5
From problem 2 we have p =
13 , so that e =
17
T
consider e A which is given by
1
1
eT A = 1 3 5 3
1
1
So the shortest distance is given by ||e|| = E = 44.
1
3
b p is given by e =
5 . Now
3
0
1
=
3
4
00
Problem 4 (the calculus solution to the least squares problem)
We dene E = ||Ax b||2 as
E = (C + D 0 0)2 + (C + D 1 8)2 + (C + D 3 8)2
+ (C + D 4 20)2
so that taking derivatives of E we have
E
C
= 2(C + D 0 0) + 2(C + D 1 8)
+ 2(C + D 3 8) + 2(C + D 4 20)
E
= 2(C + D 0 0) 0 + 2(C + D 1 8) 1
D
+ 2(C + D 3 8) 3 + 2(C + D 4 20) 4 .
where the strange notation used in taking the derivative above is to emphases the relationship
between this procedure and the one obtained by using linear algebra. Setting each equation
equal to zero and then dividing by two we have the following
(C + D 0) + (C + D 1) +
(C + D 3) + (C + D 4) = 0 + 8 + 8 + 20 = 36
(C + D 0) 0 + (C + D 1) 1 +
(C + D 3) 3 + (C + D 4) 4 = 0 0 + 8 1 + 8 3 + 20 4 = 112 .
Grouping the unknowns C and D we have the following system
48
8 26
C
D
=
36
112
Problem 5
The best horizontal line is given by the function y = C .
given by
0
1
8
1
Ax = c =
8
1
20
1
By least squares the coecient A is
Which has normal equations given by AT Ax = AT b or 4C =
gives an error of
1
0
8 1
9 =
e = b Ax =
8 1
1
20
Problem 6
We have x =
aT b
aT a
=
8+8+20
4
= 9. Then
9
9
p = xa =
9
9
16 + 20 = 36, or C = 9. This
9
1
1
11
and
09
89
e= bp=
89
20 9
1
1
so that eT a = 9 1 1 +11 = 0 as expected. Our error norm is given by
1
1
||e|| = ||b p|| = 81 + 1 + 1 + 121 = 204.
Problem 7
For the case when b = Dt our linear system is given by Ax = b with x = [D ] and
0
0
8
1
A = and b =
8 .
4
20
4
With these denitions we have that AT A = [1 + 9 + 16] = [26], and AT b = [0 + 8 + 24 + 80] =
[112], so that
112
56
x=
=
,
26
13
then Figure 1.9 (a) would look like
Problem 8
We have that
x=
0 + 8 + 24 + 80
56
aT b
=
=
.
Ta
a
1 + 9 + 16
13
so that p is given by
0
56 1
.
p=
13 3
4
In problems 1-4 the best line had coecients (C, D) = (1, 4), while in the combined problems
5-6 and 7-8 we found C and D given by (C, D) = (9, 56 ). This is because (1, 1, 1, 1) and
13
(0, 1, 3, 4) are not perpendicular.
Problem 9
Our matrix and right hand side in
1
1
A=
1
1
So the normal equations are
1
T
0
A A=
0
and AT b is given by
this case is given by
00
1 1
and b =
39
4 16
given by
1
111
1
1 3 4
1
1 9 16
1
0
8
.
8
20
00
4 8 26
1 1
8 26 92 .
=
3 9
26 92 338
4 16
0
36
1111
8
AT b = 0 1 3 4
8 = 112 .
400
0 1 9 16
20
In gure 4.9 (b) we are computing the best t to the span of three vectors where best is
measured in the least squared sense.
Problem 10
For the A given
100 0
1 1 1 1
A=
1 3 9 27 .
1 4 16 64
The solution to the equation Ax = b is given by performing
augmented matrix [A; b] as follows
100
100 0 0
0 1 1
1 1 1 1 8
[A; b] =
1 3 9 27 8 0 3 9
0 4 16
1 4 16 64 20
100
100 0
0
0 1 1
0 1 1 1
8
0 0 6 24 16 0 0 6
000
0 0 12 60 12
Given
0
1 47
,
x=
3 28
5
Gaussian elimination on the
00
1 8
27 8
64 20
0
0
1
8
.
24 16
84 XXX
then p = b and e = 0.
Problem 11
Part (a): The best line is 1 + 4t so that 1 + 4t = 1 + 4(2) = 9 =
b
Part (b): The rst normal equation is given by Equation 9 in the text and is given by
ti D =
mC +
i
bi ,
by dividing by m gives the requested expression.
Problem 12
Part (a): For this problem we have at ax = at b given by
mx =
bi ,
i
so x is then given by
x=
1
m
bi ,
i
or the mean of the bi
Part (b): We have
Then ||e|| =
m
i=1 (bi
x )2
e = b x
1
1
.
.
.
1
=
b1 x
b2 x
.
.
.
bm x
Part (c): If b = (1, 2, 6)T , then x = 1 (1 + 2 + 6) = 3 and p = (3, 3, 3)T , so the error e is
3
given by
1
3
2
e = 2 3 = 1 .
6
3
3
We can check pT e = 3(2 1 + 3) = 0 as it should. Computing our projection matrix P we
have
1
111
T
aa
1
1
P= T = 1 1 1 1 = 1 1 1.
aa
3
3
1
111
Problem 13
We will interpret this question as follows. For each instance the residual will be one of the
values listed (1, 1, 1). Considering b Ax = (1, 1, 1) we have by multiplying by
(AT A)1 AT the following
(AT A)1 AT (b Ax) = (AT A)1 AT b (AT A)1 AT Ax = x x .
If the residual can equal any of the following vectors
1
1
1
1
1
1
1
1
1 , 1 , 1 , 1 , 1 , 1 , 1 , 1 .
1
1
1
1
1
1
1
1
We rst note that the average of all of these vectors is equal to zero. In the same way the
action of (AT A)1 AT on each of these vectors would produce (each of the following should
be multiplied by 1/3)
3 , 3 , 1 , 1 , 1 , 1 , 1 , 1 ,
which when summed gives zero.
Problem 14
Consider (b Ax)(b Ax)T and multiply by (AT A)1 AT on the left and A(AT A)1 on the
right, to obtain
(AT A)1 AT (b Ax)(b Ax)T A(AT A)1 .
Now since B T C = (C T B )T the above becomes remembering the denition of x
( x)( A(AT A)1
x
T
(b Ax))T = ( x)((AT A)1 AT (b Ax))T
x
= ( x)( x)T .
x
x
so that if the average of (b Ax)(b Ax)T is T I we have that the average of ( x)( x)T
x
x
T
1 T
2
T
1
2
T
1 T
T
1
2
T
1
is (A A) A ( I )A(A A) , to obtain (A A) A A(A A) = (A A) .
Problem 15
The expected error ( x)2 is 2 (AT A)1 =
x
O (1/m)).
2
,
m
so the variance drops signicantly (as
Problem 16
We have
1
99
1
b100 +
x99 =
100
100
100
bi .
i
Problem 17
Our equations are given by
7 = C + D (1)
7 = C + D (1)
21 = C + D (2) .
Which as a system of linear equations matrix are given by
1 1
7
1 1 C = 7 .
D
12
21
The least squares solution is given by AT Ax = AT b which in this case simplify as follows
7
1 1
1 11
C
1 11
7 or
=
1 1
1 1 2
D
1 1 2
21
12
32
26
C
D
=
35
42
.
Which gives for [C, D]T the following
C
D
=
9
4
,
so the linear line is b = 9 + 4t.
Problem 18
We have p given by
1 1
5
9
p = Ax = 1 1
= 13
4
12
17
that gives the values on the closest line. The error vector e is then given by
7
5
2
e = b p = 7 13 = 6 .
21
17
4
Problem 19
1 1
2
Our matrix A is still given by A = 1 1 , but now let b = 6 , so that x =
12
4
T
1 T
(A A) A b = 0. Each column of A is perpendicular to the error in the least squares
solution and as such has AT b = 0. Thus the projection is zero.
Problem 20
5
When b = 13 , we have
17
x = (AT A)1 AT b
= (AT A)1
35
42
5
13
17
1 11
1 1 2
= (AT A)1
.
Or inserting the value of (AT A)1 we have
x=
1
14
6 2
2 3
35
42
=
9
4
.
Thus the closest line is given by b = 9 + 4t and the error is given by
5
5
e = b Ax = 13 13 = 0 .
17
17
Now e = 0 because this b is in the column space of A.
Problem 21 (the subspace containing the components of projections)
The error vector e must be perpendicular to the column space of A and therefore is in the
left nullspace of A. The projection vector p must be in the column space of A, the projected
basis x must be in the row space of A. The nullspace of A is the zero vector assuming that
the columns of A are linearly independent which is generally true for least squares problems
if m > n.
Problem 22
With A given by
A=
1 2
1 1
1 0
1 1
12
we should form AT Ax = AT b and solve for x. Note that for this problem we have that
ti = 0 and our line has coecients given by
C=
D=
1
m
i
1
bi = 5 = 1
5
b1 T1 + . . . + bm Tm
4(2) + 2(1) + 1(0) + 0(1) + 0(2)
= ... .
=
2
2
2
T1 + T2 + . . . + Tm
4+1+0+1+4
Then the least squares line is C + Dt.
Problem 23
With P = (x, x, x) and Q = (y, 3y, 1) then
||P Q||2 = (x y )2 + (x 3y )2 + (x + 1)2 .
Then to nd the minimum of this we set the x and y derivatives equal to zero
||P Q||2
=0
x
||P Q||2
= 0,
y
and solve for the unknowns x and y .
Problem 24
Now e is orthogonal to anything in the column space of A so that would be p = Ax, so
T
e p = 0. We have for our error e the following
||e||2 = (b p)T (b p) = eT (b p) = eT b = (b p)T b = bT b bT p .
Problem 25
Since ||Ax b||2 can be expressed as
||Ax b||2 = (Ax b)T (Ax b)
= (Ax)T (Ax) (Ax)T b bT (Ax) + bT b
= ||Ax||2 2bT (Ax) + ||b||2 .
So the derivatives of ||Ax b||2 will be zero when
2AT Ax 2AT b = 0 ,
or
AT Ax = AT b .
These equations we recognized as the normal equations.
Section 4.4 (Orthogonal Bases and Gram-Schmidt)
Problem 1
1
= 1 = 0, and the second vector does
1
not have norm equal to one so these vectors are only independent.
10
Part (a): We check the dot product
Part (b): We check the dot product
0.4
0.3
0.6 0.8
= 0.24 0.24 = 0, so they are
othogonal. The norm of each is given by
0.36 + 0.64 = 1
||v1 || =
0.16 + 0.09 = 0.25 = 0.5 .
||v2 || =
Part (c): Here we have that
T
v1 v2 = cos() sin() + sin() cos() = 0 ,
and ||v1 || = ||v2 || = 1 so the vectors are orthonormal.
Problem 2
We have
2
1
q1 = 2
3
1
1
1
and q2 = 2
3
2
so that the matrix obtained by concatonating q1
2/3
Q = 2/3
1/3
Then QT Q is given by
QT Q =
and q2 as column is given by
1/3
2/ 3
2/ 3
10
01
and the symmetric product QQT is given by
5 2 4
1
QQT = 2 8 2
9
4 2 5
Problem 3
Part (a): Here AT A would be the three by three identity matrix times 42 = 16.
Part (b): Here AT A would be
12 0 0
100
0 22 0 = 0 4 0
0 0 32
009
Problem 4
10
Part (a): Let Q = 0 1 , then
00
1
QQT = 0
0
Part (b): Let v1 =
1
0
and v2 =
QQT is given by
0
1
0
100
010
100
= 0 1 0 .
000
0
.
0
Part (c): Let the basis be composed of
1/2
1/2 1/2 1/2
1/2 1/2 1/2
1/ 2
1/2 1/2 1/2 1/2
1/2 1/2 1/2 1/2
Problem 5
All vectors that lie in the plane must be in the nullspace of
A=
112
,
which has a basis given by the span of v1 and v2 given by
1
2
v1 = 1 and v2 = 0 .
0
1
These two vectors are not orthogonal. Now let w1 be given by
1
1
w1 = 1
2
0
1
T
T
and W2 = v2 (v2 w1 )w1 . Now as v2 w1 = 2 2 = 2 and ||w1||2 = 1, we have the ratio
above given by
1
1
T
1
(v2 w1 )
w1 = 2 1 = 1 .
||w1 ||2
2
0
0
So with this subcalculation we have W2 given
2
W2 = 0
1
by
1
1
1 = 1 .
0
1
Therefore when we normalize we get w2 equal to
1
1
1
1
1 = 1
w2 =
||w2 ||
3
1
1
Problem 6
To show that a matrix Q is orthogonal we must show that QT Q = I . For the requested matrix
Q1 Q2 consider the product (Q1 Q2 )T (Q1 Q2 ). Since this is equal to QT QT Q1 Q2 = QT Q2 = I ,
21
2
showing that Q1 Q2 is orthogonal.
Problem 7
The projection matrix P is given by P = Q(QT Q)1 QT = QI 1 QT = QQT , so the projection
onto b will be
T
q1 b
qT b
2
T
T
T
p = P b = QQT b = Q . = (q1 b)q1 + (q2 b)q2 + . . . + (qm b)qm
.
.
T
qm b
Problem 8
Part (a): For Q given by
we have
0.8 0.6
Q = 0.6 0.8
0
0
0.8 0.6
QQT = 0.6 0.8
0
0
0.8 0.6 0
0.6 0.8 0
100
= 0 1 0 .
000
Then our projection matrix is given by
100
P = 0 1 0
000
so that P 2 is then
100
100
100
P2 = 0 1 0 0 1 0 = 0 1 0 = P .
000
000
000
Part (b): Since (QQT )(QQT ) = QQT QQT = QQT , we have that P = QQT = (QQT )(QQT )
so that P which equals QQT is the projection matrix onto the columns of the the matrix Q.
Problem 9 (orthonormal vectors are linearly independent)
Part (a): Assuming that c1 q1 + c2 q2 + c3 q3 = 0 and taking the dot product of both sides with
T
q1 gives c1 q1 q1 = 0 implying that c1 = 0. The same thing holds when we take the dot product
with q2 and q3 showing that all ci s must be zero and the qi s are linearly independent.
Part (b): Dening Q = [q1 q2 q3 ], then to prove linearly dependence we are looking for an
x = 0 such that Qx = 0. From Qx = 0 multiply on the left by QT to get QT Qx = 0. Since
QT Q = I by the orthogonality of the qi s we have that x = 0 showing that no nonzero x
exists and the qi s are linearly independent.
Problem 10
Part (a): To be in both planes we are looking for a variable
Let v1 =
13457
T
A=
x
y
has
1 6
3 6
4 8
5 0
78
so that normalized we have
1
1
3
3
1
4 = 1 4
v1 =
10
1 + 9 + 16 + 25 + 49 5
5
7
7
then
v2 =
6
6
8
0
8
13457
Normalizing we then have
6
6
8
0
8
1
3
4
5
7
7
3
4
5
1
1
102
1
v2 =
49 + 9 + 16 + 25 + 1
=
6
6
8
0
8
7
3
4
5
1
1
=
10
1
3
4
5
7
=
7
3
4
5
1
T
T
Part (b): The vector closes to [1 , 0 , 0 , 0 , 0]T is given by p = q1 (q1 b) + q2 (q2 b) or
1
7
25
3
3
9
1
1 1
4
4 7 = 1 12 .
10
10 + 10
10
50
5
5
20
7
1
0
Problem 11
T
T
This is (q1 b)q1 + (q2 b)q2 .
Problem 12
Part (a): If the ai s are orthogonal then Ax = b is [a1 a2 a3 ] x = b, and multiplying by AT
(which is the inverse of A) gives AT Ax = AT b or
T
a1 b
aT b
x=
2
aT b
3
Part (b): If the as are orthogonal then
T
T
T
a1
a1 a1 aT a2 aT a3
a1 a1
0
0
1
1
0
aT a2
AT A = aT a1 a2 a3 = aT a1 aT a2 aT a3 = 0
2
2
2
2
2
T
T
T
T
T
a3
a3 a1 a3 a2 a3 a3
0
0
a3 a3
aT b
1
so from AT Ax = AT b = aT b we obtain
2
aT b
3
x=
aT b
1
aT a1
1
aT b
2
aT a2
2
aT b
3
aT a3
3
Part (c): If the as are independent then x1 is the rst row of A1 times b.
Problem 13
We would let
A=a
B = b
aT b
a=
aT a
4
0
4
2
1
1
=
4
0
2
2
=
2
2
We need to subtract two times a to make the result orthogonal to a.
Problem 14
We have
a
1
1
=
||a||
21
1
B
=
=
||B ||
4+4
q1 =
q2
Then we have
T
with q1 b =
1
2
4
0
1
1
=
2
1 / 2 1 /
1/ 2 1/ 2
4
,
2
which implies that the above matrix decomposition is
given by
14
10
1
=
2
=
14
10
11
2
2
=
1/2 1/
2
1/ 2 1/ 2
T
2 qb
1
0 22
2 4/ 2
0 22
.
We can check this result by multiplying the above matrices together. Performing the multiplication of the two matrices on the right together we have
1/2 1/
2
2 4/ 2
14
1 4/ 2 + 2
,
=
=
10
1 4/ 2 2
1/ 2 1/ 2
0 22
verifying the decomposition.
Problem 15
Part (a): With the matrix A given by
1
1
A = 2 1
2 4
1
1
1
we will let a = 2 , so that q1 = 1 2 . Now let b = 1 , then B is given by
3
2
2
4
aT b
B = b T a
a
a
1
1
2
2 (1 2 8) 2 = 1
=
(1 + 4 + 4)
4
2
2
Then q2 is the normalized version of B and is given by
2
2
1
B
1 = 1 1
=
q2 =
||B ||
3
4+1+4 2
2
1
0 , that is linearly independent from a
Now to compute q3 we pick a third vector say
0
and b, we then have
1
cT b
cT a
C = 0 T a T b
aa
bb
0
1
1
5
1
1
1
1
1 = 1 .
= 0 2
9
18
6
2
4
0
0
Which gives for q3 the following
5
1
q3 = 1
26
0
Part (b): q3 must be orthogonal to the columns and therefore is in the left nullspace.
Part (c): We have that p is given by
2/3
2/3
T
1/3 1/3 + 1 2 7
127
p=
2/3
2/3
2
1
3
= 2 1 + (1) 2 = 0 ,
2
2
6
T
1/3
1/3
2/3 2/3
2/3
2/3
is one method, another would be by solving the normal equations AT Ax = AT b which in
this case turn out to be
1
1
1 2 2
11
2 1 = 9
.
AT A =
1 1 4
12
2 4
and AT b is given by
1 + 4 14
1 2 + 28
AT b =
Then x is given by
11
9 (2 1)
x=
15
27
21
11
15
27
=
=
1
3
19
14
Problem 16
Find the projection of b onto a. We have that our coecient x is given by
x=
bT a
4 + 10
2
=
=.
Ta
a
16 + 25 + 4 + 4
7
To nd orthonormal vectors let
4
5 1
1
=
q1 =
16 + 25 + 4 + 4 2 7
2
and dene B to be
1
2 bT a
B = T
0 a a
0
1
4
5 2
=
2 0
0
2
Normalizing this vector we then have
4
14 5 1
49 2 = 7
2
1
4
1
1
=
1 + 3(16) 4 4 3
4
q2 =
4
5
.
2
2
1
4
4
4
Problem 17
We have
1
1
1+3+5
ba
1 = 3 1 .
p= T a=
aa
3
1
1
T
1
4
.
4
4
with an error given by
Normalizing we have
1
3
2
e=bp = 3 3 = 0 .
5
3
2
1
1
q1 = 1
31
2
1
1
and q2 = 0 = 0 .
2
2
1
Problem 18
If A = QR then AT A = (RT QT )(QR) = RT R which we recognize as a lower triangular
matrix times a upper triangular matrix. Therefore Gram-Schmidt on A corresponds to
elimination on AT A. If A is as given in this problem then
AT A =
which reduces as
AT A
39
9 35
3
9
0 35 27
=
,
39
08
.
Which has pivots equal to ||a||2 and ||e||2 respectively.
Problem 19
Part (a): True, since the inverse of an orthogonal matrix is its transpose.
Part (b): Yes, if Q has orthonormal columns then
||Qx||2 = (Qx)T (Qx) = xT QT Qx = xT x = ||x||2 .
Problem 20
1
1
1 1 1
1
Let q1 = 4 = 2
1
1
1
1
so that q2 =
B
||B ||
. Then B is given by
2
0 (2 + 1 + 3)
B=
1
4
3
2
1
0 1 1
=
1 2 1
3
1
5
1 1
.
=
2 1
5
1
1
1
1
or
5
1
1
= 1
q2 =
25 + 1 + 1 + 25 1
52
5
5
1
.
1
5
The projecting b onto the column space of A is equivalent to computing
T
T
p = (q1 b)q1 + (q2 b)q2
=
(4 3 + 3)
2
7
1 3
.
=
2 1
3
1
2
1
1 (20 + 3 + 3)
+
1
52
1
So that the error vector e = b p is given by
8 + 7
1 6 + 3
e= bp =
2 6+1
3
5
1
1
52 1
5
1
1 3
=
2 7 ,
3
and then computing the inner product of e with each column of A we nd (using Matlab
notation that)
1
(1 3 + 7 3) = 0 and
2
1
(2 + 0 + 7 9) = 0 ,
eT A(:, 2) =
2
eT A(:, 1) =
as required.
Problem 21
1
If A = 1 so that q1 =
2
1
1
1 , we have B given by
6
2
1
T
1 A v A
B=
AT A
0
1
(1 1)
A
= 1
AT A
0
1
= 1 .
0
The next vector C is given by removing the projections along A and B . We nd
AT v
BT v
C = v T A T B
A A BB
1
1
1
1
9
1
= 0 1 1 = 1 .
6
2
4
2
0
1
Problem 22
One could do this by performing elimination on AT A as in Problem 18 or just simply performing Gram-Schmidt on the columns of the matrix A. We have
1
0 and q1 = A .
A=
0
With v =
203
T
we have that
1
3
2
2
vA
B =v T A= 0 0 = 0 ,
AA
1
0
0
3
T
0
so that q2 = 0 . then in v =
1
So that A is given by
456
T
we have a third orthogonal vector C as
BT v
AT v
C = v T A T B
A A BB
1
0
0
4
5 4 0 6 0 = 5 .
=
1
0
1
0
6
100
124
A = 0 0 1 0 3 6 .
010
005
Problem 23
Part (a): We desire to compute a basis for the subspace for the plane given by
x1 + x2 + x3 x4 = 0 .
Consider the matrix A dened as A = 1 1 1 1 , then since we want to consider the
nullspace of A we will assign ones to each free variables in succession and zeros to the other
variables and then solve for the pivot variables. This will give us a basis for the nullspace.
We nd
x2 = 1, x3 = 0, x4 = 0 x =
T
1 1 0 0
x2 = 0, x3 = 1, x4 = 0 x =
1 0 1 0
x2 = 0, x3 = 0, x4 = 1 x =
1001
T
T
.
Part (b): The orthogonal complement to S are all vectors that are orthogonal to each
T
component of the nullspace of A. This is the vector 1 1 1 1 .
T
Part (c): If b = 1 1 1 1 , then to decompose b into b1 and b2 consider the unit vector
of the vector that spans the orthogonal complement i.e.
1
1 1
,
q2 =
2 1
1
then b2 given by
1
1 1 1 1
T
=
b2 = (q2 b)q2 = (2)
2 2 1 2
1
1
1
.
1
1
Then
1
1 1
b1 = b b2 =
1 2
1
1
1 1
=
1 2
1
1
1
.
1
3
Problem 24
ab
. We begin by computing q1 . We nd
cd
We would like to perform A = QR when A =
q1 =
1
+ c2
a2
a
c
.
and then B is given by
b
d
B=
bd
1
a2 + c2
a
c
1
a2 + c2
a
c
ab + dc a
b
2
d
a + c2 c
ad bc c
.
=
a
a2 + c2
=
which is orthogonal to
a
, and has a unit vector given by
c
1
c
.
2
a
+c
So the matrix Q in the QR decomposition of A is given by
a2
Q=
1
a2 + c2
a c
ca
.
Then R is given by (using Matlab notation)
R=
T
T
q1 A(:, 1) q1 A(:, 2)
T
0
q2 A(:, 2)
=
1
a2 + c2
To the decomposition of A is then given by
A=
1
2 + c2
a
a c
ca
1
2 + c2
a
a2 + c2 ab + cd
0
cb + ad
a2 + c2 ab + cd
0
cb + ad
If (a, b, c, d) = (2, 1, 1, 1) then we obtain
A=
1
5
2 1
12
1
5
53
01
,
1
2
22
00
,
while if (a, b, c, d) = (1, 1, 1, 1) we obtain
A=
1
2
1 1
11
From which we see that the (2, 2) element of R in this case is zero.
.
.
Problem 25
Equation 8 is given by
C =c
AT c
BT c
A T B
AT A
BB
The rst equation in 12 is given by
m
rkj =
aik aij ,
i=1
is the expression for the dot product between the k th column of Q and the j th column of
A. Then aij = aij qik rkj subtracts the projection onto the basis functions.
Problem 26
a and b may not be orthogonal so by subtracting projections along non-orthogonal vectors
one would be double counting.
Problem 27
See the Matlab code chap4 sect 4 4 prob 27.m.
Problem 28
Equation 11 involves m multiplications from the summation and m divisions for the calcuaik
lations of qik = rkk giving a total of O (2m) calculations. Each of these multiplications are
performed multiple times. Thus we have
n
n
2m +
k =1
n
2m = 2mn +
j =k +1
k =1
2m(n k 1 + 1)
n
= 2mn + 2m
= 2mn + 2m
k =1
n1
(n k )
k
k =1
= 2mn + 2m
= mn2 + mn ,
which is the required number of ops.
n(n 1)
2
Problem 29
Part (a): We desire to check that QT Q = I , when computing this product we have
1 1 1 1
1 1 1 1
1 1 1 1 1 1 1 1
QT Q = c2
1 1 1 1 1 1 1 1
1 1 1 1
1 1 1 1
4000
0 4 0 0
= c2
0 0 4 0 =I,
0004
by picking c = 1 .
2
Part (b): We know that Q dened by
1 1 1 1
1 1 1 1
Q = c
1 1 1 1 ,
1 1 1 1
which will be orthogonal if c =
1
2
as in Part (a).
Problem 30
T
Projecting onto the rst column of Q we have a coecient given by q1 b = 1 (2) = 1, so
2
that we have a projection of
1
1 1
.
p=
2 1
1
To project onto the rst two columns of the matrix A we give
T
q1 b = 1
1
T
(2) = 1 .
q2 b =
2
So that p is now given by
1
1 1 1
p=
2 1 2
1
0
1
1 0
=
1 1
1
1
.
Problem 31
Now Q = I 2uuT is a reection matrix. If u = [0, 1]T then
uuT =
0
1
01
=
00
01
so that Q is given by
Q=I
If r =
x
y
00
02
=
10
0 1
.
x
. If u = (0, 1/ 2, 1/ 2) then
y
0
00
0
uuT = 1/2 0 1/ 2 1/ 2 = 0 1/2 1/2
0 1/ 2 1/ 2
1/ 2
then Qr =
so that Q is given by
10
0
Q = I 2uuT = 0 0 1 .
0 1 0
x
x
y then Qr = z .
If r =
z
y
Problem 32
Part (a): From the denition of Q we have
Qu = u 2uuT u = u 2u = u .
Part (b): If uT v = 0 then we have
Qv = v 2uuT v = v .
Problem 33
What is special about the columns of W is that they
then its transpose i.e.
1
1
1
1
1
W 1 = W T =
22
2
0
0
are orthonormal. The inverse of W is
1
1
1 1
.
0
0
22
Chapter 5 (Determinants)
Section 5.1 (The Properties of Determinants)
Problem 1 (examples of properties of the determinant)
If det(A) = 2, and A is 4 by 4 we then have
det(2A) = 24 det(A) = 24 2 = 32
det(A) = (1)4 det(A) = 2
det(A2 ) = det(A)2 = 4
1
1
det(A1 ) =
=
det(A)
2
Problem 2 (more examples with the determinant)
If det(A) = 3, and A is 3 by 3 we then have
3
1
1
3
det( A) =
det(A) =
2
2
8
3
det(A) = (1) det(A) = (3) = 3
det(A2 ) = det(A)2 = 9
1
1
=
det(A1 ) =
det(A)
3
Problem 3 (true/false propositions with determinants)
Part (a): False. If we dene A as
A=
12
34
,
then det(A) = 2 and we have I + A given by
I +A=
22
35
,
so det(I + A) = 10 6 = 4, while 1 + det(A) = 1 2 = 1, which are not equal.
Part (b): True
Part (c): True
Part (d): False, let A = I then
4 A = 4I =
40
04
,
so det(4A) = 16 = 4det(A) = 4 det(I ) = 4.
Problem 4 (row exchanges of the identity)
If
001
J3 = 0 1 0
100
then J3 is obtained from I by exchanging rows one and three from the three by three identity
matrix. If J4 is given by
0001
0 0 1 0
J4 =
0 1 0 0
1000
then J4 is obtained from the four by four identity matrix by exchanging the second and third
rows and the rst and fourth rows.
Problem 5 (more row exchanges of the identity)
We will propose an inductive argument to express the number of row exchanges needed
to permute the reverse identity matrix Jn to the identity matrix In . From problem 4, we
have the number of row exchanges needed when n = 3 and n = 4 is given by one and two
respectively. For n = 5 the reverse identity matrix is given by
00001
0 0 0 1 0
J5 = 0 0 1 0 0
0 1 0 0 0
10000
and can be converted into the identity matrix with two exchanges; by exchanging rows one
and ve, and rows two and four. So we have that the determinant of J5 is given by (1)2 = 1.
For n = 6 the identity and the reverse identity are given by
000001
100000
0 0 0 0 1 0
0 1 0 0 0 0
0 0 0 1 0 0
0 0 1 0 0 0
I6 =
0 0 0 1 0 0 and J6 = 0 0 1 0 0 0
0 1 0 0 0 0
0 0 0 0 1 0
100000
000001
n
3
4
5
6
7
number of row exchanges
1
2
2
3
3
Table 2: The number of row exchanges needed to convert the identity matrix into the reverse
identity matrix.
From which we can see that the reverse identity in this case has three row exchanges; row
one and six, row two and ve, row three and four. So we have that the determinant of J6
is given by (1)3 = 1. For n = 7 we will have three row exchanges to obtain the reverse
identity matrix, so the determinant of J7 will be given by (1)3 = 1. A summary of our
results thus far can be given in Table 2. From Table 2, the general rule seems to be that
the number of exchanges required for transforming the n by n identity matrix to the n by
n reverse identity matrix involves oor( n ) row exchanges. So to produce the J101 matrix
2
we have oor( 101 ) = 50 row exchanges from the 101 101 identity matrix. From this the
2
determinant of J101 is given by (1)50 = 1.
Problem 6 (a row of all zeros gives a zero determinant)
If a matrix has a row of all zeros, we can replace that row with a row of non-zeros times a
multiplier which is zero i.e. in the notation of the book take t = 0. Then part of rule number
three, says that the determinant of this matrix is equal to t times the determinant of the
matrix with the non-zero row. Since 0 times anything gives zero, the original determinant
must be zero.
Problem 7 (determinants of orthogonal matrices)
An orthogonal matrix has the property that QT Q = I . Taking the determinant of both
sides of this equation we obtain |Q||QT | = 1. Since |Q| = |QT | we have that |Q|2 = 1, or
|Q| = 1. Also from the above we have that for orthogonal matrices Q1 = QT . By taking
determinants of both sides we have that |Q1 | = |QT | = |Q|.
Problem 8 (determinants of rotations and reections)
If Q is a two-dimensional rotation, then
Q=
cos() sin()
sin() cos()
Then |Q| = cos()2 + sin()2 = +1. For a reection Q is given by
Q=
1 2 cos()2
2 cos() sin()
2 cos() sin()
1 2 sin()2
so that
|Q| = (1 2 cos()2 )(1 2 sin()2 ) 4 cos()2 sin()2
= 1 2(cos()2 + sin()2 ) = 1 2 = 1
Problem 9
If A = QR, then AT = RT QT so the |AT | = |RT ||QT |, and since R is upper triangular
|RT | = |R| since both expressions are the product of the diagonal elements in each matrix.
Also from the problem above we have that QT = Q for an orthonormal matrix thus
|AT | = |RT ||QT | = |R||Q| = |Q||R| = |QR| = |A| .
Problem 10
If the entries of every row of A add to zero, then from the determinant rule that |AT | = |A|,
and the fact that by subtracting a multiple of one row from another leaves the determinant
unchanged we see that by subtracting a multiple of a column from another column leaves
the determinant unchanged. Thus by repeatedly adding a multiple (one) of each column to
each other (say accumulating the sum in the rst column) we will obtain a column of zeros
and therefore show that the determinant is zero.
If every row of A adds to one we can prove that det(A I ) = 0 by recognizing that because
of this fact every row of A I adds to zero and therefore the determinant must be zero by
the previous part of this problem. This does not imply that det(A) = 1 since if we let
A=
20
1 1
has every row adding to one but det(A) = 2 = 1.
Problem 11
If CD = DC , then the determinant of the left hand side is given by |CD| = |C ||D | and
the determinant of the right hand side is given by | DC | = (1)n |DC | = (1)n |D ||C |.
This shows that (1 (1)n )|D ||C | = 0, so |D | = 0, or |C | = 0, or 1 (1)n = 0, i.e. n is
even.
Problem 12
The correct calculation is given by the following
1
d b
ad bc c a
1
d b
det
=
2
c a
(ad bc)
1
1
=
(ad cb) =
.
2
(ad bc)
ad bc
det(A1 ) = det
Problem 13
We have by applying row operations to the rst
1
1 230
0
2 6 6 1
det
1 0 0 3 = det 0
0
0 205
1
0
= det
0
0
The second example is given by
2
2 1 0
0
0
1 2 1 0
det
0 1 2 1 = det 0
0
0
0 1 2
2
0
= det
0
0
2
0
= det
0
0
example the following
230
2 0 1
0 3 3
205
230
2 0 1
= 1 2 3 4 = 24 .
0 3 2
004
1 0
0
3/2 1 0
1 2 1
0 1 2
1 0
0
3/2 1 0
0 4/3 1
0 1 2
1 0
0
3/2 1 0
= 2 3 4 5 = 4.
0 4/3 1
234
0
0 5/ 4
Problem 14
We have using row operations to simplify the determinant
1
a
a2
1 a a2
det 1 b b2 = det 0 b a b2 a2 .
0 c a c2 a2
1 c c2
Continuing in this fashion when we eliminate the element b a we obtain a (3, 3) element
of the above give by
(c2 a2 )
(c a) 2
(b a2 ) = c2 cb ca + ab = c(c a) + b(a c) = (c b)(c a)
(b a)
so our determinant above becomes equal to
1
a
a2
= (b a)(c b)(c a) ,
b2 a2
det 0 b a
0
0
(c b)(c a)
as expected.
Problem 15
For the matrix A we know that its determinant must equal zero since it will be a three by
three matrix but of rank one and therefore will not be invertible. Because it is not invertible
its determinant must be zero. Another way to see this is to recognize that this matrix can
be easily reduced (via elementary row operations) to a matrix with a row of zeros.
For the matrix K we see that K T = K , so that |K T | = |K | from Proposition 10 from this
section of the book. We also know that | K | = (1)3 |K | since K is a three by three matrix.
Thus the determinant of K must satisfy |K | = (1)3 |K | = |K |, which when solved for for
|K |, gives |K | = 0.
Problem 16
From the problem above we have shown that for a matrix K that is skew symmetric with m
odd we have that |K | = 0. If m can be even giving a non zero determinant. For a four by
four example consider the matrix K dened by
0
1
11
1 0
1 1
K=
1 1 0 1
1 1 1 0
then we would have |K | equal to (using elementary row operations)
1 0
1
1
1 0
11
0
1
1
1
1
1 1
= (1)det 0
(1)det
0 1 1 0
1 1 0 1
0 1 2 1
1 1 1 0
1 0 1 1
0 1 1 1
= (1)det
0 0 0 1
0 0 1 0
1 0 1 1
0 1 1 1
= (1)2 det
0 0 1 0 = (1) 1 (1) = 1 .
0001
Where the last equality is obtained by exchanging rows three and four.
Problem 17
The determinant of the rst matrix (denoted A in this solution manual) the solution is
formally,
101 201 301
det(A) = det 102 202 302 ,
103 203 303
which by subtracting the second row from the third gives
101 201 301
det(A) = det 102 202 302 ,
1
1
1
continuing we now subtract the rst row from the second to obtain
101 201 301
det(A) = det 1
1
1 ,
1
1
1
from which since our matrix has two identical rows requires that its determinant must be
zero.
For the second matrix (denoted by B in this solution manual) we have for the expression for
the determinant the following
1 t t2
det(A) = det t 1 t .
t2 t 1
Now by multiplying the rst row by t and subtracting from the second and multiplying the
rst row by t2 and subtracting from the third we have
1
t
t2
det(A) det = 0 1 t2 t t3 .
0 t t3 1 t4
Continuing using elementary row operations we have
1
t
t2
.
t t3
det(A) = det 0 1 t2
4
3
0
0
1 t t(t t )
The (3, 3) element of this matrix simplies to 1 t2 , which gives for the determinant of B
the product of the diagonal elements or
1 (1 t2 ) (1 t2 ) .
This expression will vanish if t = 1.
Problem 18
For the rst U given by
123
U = 0 4 5
006
from which we have |U | = 1 4 6 = 24. From this we have that |U 1 | =
|U 2 | = |U | |U | = |U |2 = 242 = 416.
1
|U |
=
1
,
24
and
For the second U given by
U=
we have |U | = ad, |U 1 | =
1
|U |
=
1
ad
ab
0d
and |U 2 | = |U |2 = a2 d2 .
Problem 19 (multiple row operations in a single step)
One cannot do multiple row operations at one time and get the same value of the determinant.
The correct manipulations are given by
det(A) = det
= det
= det
= det
ab
cd
a
b
c la d lb
a L(c la) b L(d lb)
c la
d lb
a Lc + Lla b Ld + Llb
c la
d lb
.
The proposed matrix in the book is missing the terms Lla and Llb. Another way to show
that the two determinants are not equal is to compute the second one directly. Which is
given by
(a Lc)(d lb) (b Ld)(c la) =
=
=
=
ad alb Lcd + Llcb (bc lba Ldc + Llad)
ad bc + Llcb Llad
ad bc Ll(ad cb)
(ad bc)(1 Ll)
Problem 20
Following the instructions given and the matrix A we see that
det(A) = det
ab
cd
= det
a
b
c+a d+b
= det
c
d
c+a d+b
= det
c d
a
b
= (1)det
cd
ab
= (1)det(B )
where in the transformations above we have used two rules. The rst is that subtracting
a multiple of one row from another row does not change the determinant and the second
being that factoring a multiplier of a row out of the matrix multiples the determinant by an
appropriate factor.
Problem 21
We have |A| = 4 1 = 3, |A1 | =
|A I | = 0 we must have
1
(4
32
1
1) = 3 , and |A I | = (2 )2 1. Thus for
(2 ) = 1
or = 1 or = 3. If = 1 then A I is given by
AI =
11
11
If = 3 then A I is given by
A 3I =
1 1
1 1
.
Problem 22
If A is given by
A=
41
23
.
so we have that |A| = 12 2 = 10 and A2 is given by
A2 =
18 7
14 11
.
with a determinant given by |A2 | = 100, now A1 is given by
1
10
A1 =
so that |A1 | =
1
.
10
3 1
2 4
.
We now compute A I which gives
A I =
4
1
2
3
.
so that |A I | = (4 )(3 ) 2. Now by setting this equal to zero and solving for
we have that |A I | = 0 is equivalent to ( 2)( 5) = 0 giving that = 2 or = 5.
Problem 23
Since |L| = 1, we have that |U | = 3(2)(1) = 6, so |A| = |L| |U | = 6. Then since
A = LU we have that A1 = U 1 L1 , so
|A1 | = |U 1 ||L1 | =
11
1
= .
|U | |L|
6
Since U 1 L1 A = I we have the obvious identity that |U 1 L1 A| = 1.
Problem 24
If Aij = i j , then the A matrix is m by
1
2
A= .
.
.
m
m and is given by the outer product
1 2 ... m
.
Which is a rank one matrix and therefor has a determinant equal to zero, since it is not
invertible. Multiple rows are multiples of a single row.
Problem 25
We are asked to prove that if Aij = i + j then det(A) = 0. Lets consider the case when A is
m by m and consider the rst second and third rows of A. These rows are given by
1 + 1 1 + 2 1 + 3 1 + 4 ... 1 + m
2 + 1 2 + 2 2 + 3 2 + 4 ... 2 + m
3 + 1 3 + 2 3 + 3 3 + 4 ... 3 + m
Now the determinant is unchanged if we subtract the second row from the rst. Doing this
gives for the rst three rows the following
1 + 1 1 + 2 1 + 3 1 + 4 ... 1 + m
2 + 1 2 + 2 2 + 3 2 + 4 ... 2 + m
1
1
1
1
...
1
Now subtracting the rst row from the second row gives
1 + 1 1 + 2 1 + 3 1 + 4 ... 1 + m
1
1
1
1
...
1
1
1
1
1
...
1
Since this matrix has two repeated rows, the determinant must be zero.
Problem 26
For A we have
0a0
c00
c00
det(A) = det 0 0 b = (1)det 0 0 b = (1)2 det 0 a 0 = abc .
c00
0a0
00b
For B we have
0
0
det(B ) = det
0
d
a
0
0
0
0
b
0
0
d
0
= (1)3 det
0
0
d
0
0
0
= (1)det
0
c
0
0
000
a 0 0
= abcd .
0 b 0
00c
0
0
0
a
0
b
0
0
0
0
= (1)2 det
c
0
d
0
0
0
0
a
0
0
0
0
0
b
0
0
c
0
Finally for C we have
aaa
a
a
a
det(C ) = det a b b = 0 b a b a
abc
0 ba ca
a
a
a
ba
= 0 ba
0
0
c a (b a)
a
a
a
= 0 b a b a = a(b a)(c b) .
0
0
cb
Problem 27
Part (a): True. We know from a previous problem that rank(AB ) rank(A) and since
rank(A) < m, the product must have ran(AB ) rank(A) < m, and therefore AB cannot
be invertible.
Part (b): True. Since elementary row operations change A into U and the determinant of
U is the product of the pivots.
20
and B =
02
B ) = 1, but det(A) det(B ) = 4 1 = 3.
Part (c): False. Let A =
10
, then A B =
01
10
, so det(A
01
Part (d): True. If the product of A and B is dened in that way.
Problem 28
If f (A) = ln(det(A)), then for a two by two system our f is given by f (A) = ln(ad bc).
Dening = ad bc, we have that
f
a
f
b
f
c
f
d
so that
f
a
f
b
f
c
f
d
=
1
=
d
=
c
b
a
=
=
d b
c a
= A1 .
Section 5.2 (Permutations and Cofactors)
Problem 1 (practice computing determinants)
For the matrix A using the formula |A| =
a1 a2 an , we have
10
11
01
+3
2
11
10
10
= 1(1) 2(1) + 3(1) = 1 + 2 + 3 = 4 = 0
|A| = 1
Since the determinant is not zero the columns are independent. For the matrix B we have
44
44
44
+3
2
56
57
67
= 1(28 24) 2(28 20) + 3(24 20) = 4 16 + 12 = 0 .
|B | = 1
Since the determinant is zero the columns are not independent.
Problem 2 (more practice computing determinants)
For the matrix A using the formula |A| =
a1 a2 an , we have
11
01
+0
1
01
11
= 1 1 = 2 = 0 ,
|A| = 1
Since the determinant is not zero the columns are independent. For the matrix B we have
45
46
56
+3
2
78
79
89
= (45 48) 2(36 42) + 3(32 35) = 3 + 12 9 = 0 .
|B | = 1
Since the determinant is zero the columns are not independent.
Problem 3
We have that
|A| = x
0x
0x
= 0,
since an entire column is zero. The rank of A is at most two, since the second column has
no pivot.
Problem 4
Part (a): Since the rank of A is at most two, there can only be two linearly independent
rows. As such this matrix must have a zero determinant.
Part (b): Formula 7 in the book is det(A) =
det(P )a1 a2 an . In this expression
every term will be zero because when we select columns we eventually have to select a zero in
the three by three block in the lower left of the matrix A. These zeros in the multiplication
is what makes every term zero.
Problem 5
For A we can expand the determinant about the rst row giving
011
111
|A| = 1 1 0 1 1 1 1 0
100
001
11
11
11
01
+1
1 1
1
10
00
01
01
= 1(1) 1(1) = 1 + 1 = 0 .
=1
We can also compute |A| by expanding about the last row of A given by
100
001
|A| = 1 1 1 1 + 1 0 1 1
110
101
11
11
+1
10
10
= 1(1) 1 = 1 1 = 0 .
= 1(1)
For the matrix B we can compute the determinant in the same way as with A. Expanding
about the rst row gives
034
345
|B | = 1 4 0 3 2 5 4 0
200
001
,
followed by expanding each of the remaining determinants along the bottom row gives
|B | = 1
34
34
2(2)
40
40
= 16 4(16) = 48 .
Problem 6
By creating a matrix with no zeros we have
matrix could be
1
1
A=
1
1
certainly used the smallest number. One such
111
1 1 1
,
1 1 1
111
then certainly det(A) = 0. To create a matrix with as many zeros as possible and still
maintain det(A) = 0, consider the diagonal matrix
a000
0 b 0 0
A=
0 0 c 0,
000d
with a, b, c, d all nonzero. This matrix is certainly not singular but by setting any of a, b,
c, or d equal to zero a singular matrix results.
Problem 7
Part (a): Our expression for the determinant is given by |A| = a1 a2 an . Assuming
our matrix has elements a11 = a22 = a33 = 0, we can reason which of the 3! terms in the
determinant sum will be zero as follows. Obviously all permutations with a11 in them i.e.
(1, 2, 3), and (1, 3, 2) will have a zero in them. Additionally, all permutations with a22 in
them i.e. (1, 2, 3), (3, 2, 1) will be zero. The term a33 = 0 will cause the two permutations
(1, 2, 3) and (2, 1, 3) to be zero. Since the permutation (1, 2, 3) is counted three times in total
we have four zero elements in the determinant sum.
Problem 8
To have det(P ) = +1 we must have an even number of row exchanges. Now the total number
of ve by ve permutation matrices is given 5! = 120. Half of this number are permutation
matrices with an odd number of row exchanges and the other half have an even number of
row exchanges so 60 have det(P ) = 1. Now
01000
0 0 1 0 0
P = 0 0 0 1 0 ,
0 0 0 0 1
10000
will require four exchanges to obtain the identity using row exchanges. Specically, exchanging the rst and the last row, then the second and the last row, and nally the third and
1
1
1
1
1
1
2
2
2
2
2
2
2
2
3
3
4
4
1
1
3
3
4
4
3
4
2
4
2
3
3
4
1
4
1
3
4
3
4
2
3
2
4
3
4
1
3
1
+
+
+
+
+
+
3
3
3
3
3
3
4
4
4
4
4
4
1
1
2
2
4
4
1
1
2
2
3
3
2
4
1
4
1
2
2
3
3
1
2
1
4
2
4
1
2
1
3
2
1
3
1
2
+
+
+
+
+
+
-
Table 3: An enumeration of the possible 4! permutations with + denoting a even permutation
and denoting an odd permutation.
the last row we have that
01
0 0
J = 0 0
0 0
10
10
0 1
0 0
0 0
00
J transforms under these row operations
1
10000
000
0
0 0 1 0 0
1 0 0
0 1 0 0 0 0 1 0 0
0
0 0 0 0 1
0 0 1
0
01000
000
10000
000
0 1 0 0 0
0 0 0
1 0 0 0 0 1 0 0 .
0 0 0 1 0
0 0 1
00001
010
as follows
0
1
0
0
0
0
0
0
0
1
0
0
1
0
0
0
0
0
1
0
Problem 9
Since we have that det(A) = 0 then say a1 a2 an = 0, for some specication of the
variables (, , , ). Construct the permutation that takes (, , , ) = (1, 2, 3, , n),
i.e. the inverse permutation. Then in this case AP will have a1 in position (1, 1), a a2 in
position (2, 2), a a3 in position (3, 3), etc ending with an in position (n, n). This is because
AP permutes the columns of A and will move a1 to (1, 1), etc.
Problem 10
Part (a): A systematic wave to do this problems would be to enumerate all of the possible
permutations and separate them into positive and negative permutations. Consider the
Table 3 for this enumeration.
Part (b): An odd permutation times an odd permutation is an even permutation.
Problem 11
For A given by
A=
21
36
,
we have that c11 = 6, c12 = 3, c21 = 1, and c22 = 2 so our C is given by
6 3
1 2
C=
For the matrix B given by
we have c11 =
0, c22 =
13
70
56
00
123
C= 4 5 6
700
= 21, c23
and nally c33 =
46
= 42, c13 =
70
2
12
= 14, c31 =
=
5
70
= 0, c12 =
12
45
45
= 35, c21 =
70
1
3
= 3, c32 =
4
6
23
=
00
3
= 6,
6
= 3. Thus the cofactor matrix C is given by
0
42 35
C = 0 21 14 .
3 6
3
The determinant of B is given by (expanding about the third row)
det(B ) = 7
23
56
= 21 .
Problem 12 (the second derivative matrix)
For A given by Strangs favorite matrix
2 1 0
A = 1 2 1
0 1 2
1 1
2 1
= 3, c12 =
0
2
1 2
2 1
20
= 2,
= 4, c23 =
0 1
02
we compute for the various cofactors the following: c11 = +
2, c13 =
1 2
0 1
= 1, c21 =
1 0
1 2
= 2, c22 =
=
2
1 0
= 1, c32 =
1
2 1
cofactor matrix C is given by
32
2 4
C=
12
0
1
c31 =
since C is symmetric. We then
3
CT A = 2
1
= 2, and nally c33 =
1
2
3
2 1
1 2
= 3. Thus the
321
so C T = 2 4 2
123
have for C T A the following
21
2 1 0
400
4 2 1 2 1 = 0 4 0
23
0 1 2
004
or four times the identity matrix. Note that
det(A) = 2
1
CT ,
det(A)
so we see that A1 =
1 1
2 1
+1
0
2
1 2
= 4.
as we know must be true.
Problem 13
As suggested in the text expanding |B4 | using cofactors in the last row of B4 we have
1 1 0
1 1 0
0
|B4 | = 2 1 2 1 + 1 1 2
0 1 1
0 1 2
= 2|B3 | + (1)
= 2|B3 | |B1 | .
1 1
1 2
Continuing our expansion we have that |B2 | = 2 1 = 1 and that
|B3 | =
So we see that |B4 | = 1.
1 1 0
1 2 1
0 1 2
=1
2 1
1 1
+1
1 2
0
2
= 1.
Problem 14
Part (a): We see that
C1 = | 0| = 0
01
= 1
C2 =
10
C3 =
010
101
010
= (1)
C4 =
0
1
0
0
0
0
1
0
1
0
1
0
0
1
0
1
10
10
=0
110
= (1) 0 0 1
010
= (1)2
11
01
= 1.
Part (b): We desire to compute the determinant of a matrix Cn of size n n with all ones
on the super and sub-diagonal as
| Cn | =
0
1
0
0
.
.
.
1
0
1
0
0
0
1
0
..
.
0
0
0
1
0
0
0
0
.
.
.
0
0
1
0
1
1
0
0
1
1
0
00
0 0 ...
.
By expanding this determinant about the rst row we have that
1
0
0
|Cn | = (1) 0
.
.
.
1
0
1
0
0
1
0
1
0
0
1
0
..
.
0
0
0
1
0
0
0
0
.
.
.
0
1
0
0
1
1
0
00
0 0 ...
,
which by further expanding about the rst column gives
0
1
0
|Cn | = (1)(1) 0
.
.
.
1
0
1
0
0
1
0
1
00
0 0 ...
0
0
1
0
..
.
0
0
0
1
0
0
0
0
.
.
.
0
1
0
0
1
1
0
= (1)|Cn2 | ,
since we have removed two rows from the original Cn matrix. So since |C1| = 0 we see
from the above that |C3 |, |C5|, |C7|, are all zero. Now |C2 | will determine all even terms
i.e. |C4 |, |C6 |, |C8|, . We therefore have |C4| = 1, |C8| = 1, |C12 | = 1, and |C6 | =
1, |C10 | = 1, |C14 | = 1, , so |C10 | = 1.
Problem 15
In Problem 14 (above) we have shown the desired relationships.
Problem 16
Part (a): We see that computing a few determinants that
|E1 | = 1
|E2 | = 0
|E3 | = 1
11
11
1
01
11
= 0 1(1) = 1 .
To derive a recursive relationship consider dene |En | as
|En | =
1
1
0
0
.
.
.
1
1
1
0
.
.
.
0
1
1
1
.
.
.
0
0
1
1
.
.
.
0
0
0
1
.
.
.
..
.
.
1
0
0
0
.
.
.
1
1
1
0
.
.
.
Now expand about the rst row and we have that
0
1
1
1
.
.
.
0
0
1
1
.
.
.
0
0
0
1
.
.
.
1
..
.
1
1
= |En1 | 0
0
.
.
.
1
1
1
0
.
.
.
0
1
1
1
.
.
.
0
0
1
1
.
.
.
1
1
|En | = +1 0
0
.
.
.
as we were reqested to show.
1
1
1
0
.
.
.
0
0
0
1
.
.
.
..
.
0
1
1
1
.
.
.
0
0
1
1
.
.
.
0
0
0
1
.
.
.
..
.
= |En1 | |En2 | ,
Part (b): With E1 = 1 and E2 = 0 we can iterate the above equation to nd that
E3
E4
E5
E6
E7
E8
E9
=
=
=
=
=
=
=
E2 E1
E3 E2
E4 E3
E5 E4
E6 E5
E7 E6
E8 E7
= 1
= 1 0 = 1
= 1 (1) = 0
= 0 (1) = 1
= 10 =1
= 11 =0
= 0 1 = 1 .
From these the pattern looks like
E2,5,8, = 0 or E3n+2 = 0 for n = 0, 1, 2,
and
E3,4,9,10,15,16, = 1 ,
or E3+6n = 1 and E4+6n = 1 for n = 0, 1, 2, . Finally we hypothesis that
E6,7,12,13,18,19, = 1 ,
or E6n = 1 and E1+6n = 1 for n = 0, 1, 2, . Then E100 can be written as E166+4 so looking
at these patterns we see that E6n+4 = 1 so E100 = 1.
Problem 17
We dene Fn to be
Fn =
1 1 0
0
0
1 1 1 0
0
01
1 1 0
00
1
1 1
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
so that expanding about the rst row we nd Fn to be
Fn
1 1 0
0
0
1 1 1 0
0
1 1 0
=10 1
00
1
1 1
.
.
.
..
.
.
.
..
.
.
.
..
= Fn1 + Fn2
+1
..
.
1 1 0
0
0
0 1 1 0
0
01
1 1 0
00
1
1 1
.
.
.
..
.
.
.
..
.
.
.
..
..
.
Problem 18
Thus linearity gives |Bn | = |An | |An1 | = (n + 1) (n 1 + 1) = 1, where we have used
the discussion in this section to evaluate |An | and |An1|.
Problem 19
The 4 4 Vandermonde determinant containings x3 and not x4 because a third degree
polynomical requires four points to t to. Thus a n n Vandermonde determinant will have
xn1 in it. This determinant is zero if x = a, b or c. The cofactor of x3 is given by
1 a a2
1 b b2
1 c c2
=1
b b2
c c2
a
1 b2
1 c2
+ a2
1b
1c
= bc2 cb2 a(c2 b2 ) + a2 (c b)
= bc(c b) a(c b)(c + b) + a2 (c b)
= (c b)(c a)(b a) .
Thus since V4 is a polynomial with roots a,b, and c and the coefactor of x3 represents the
leading coecient of the x3 term in the total determinant. Thus
V4 = (c b)(c a)(b a)(x b)(x a)(x c) .
Problem 20
0
1
We have that G4 is dened by G4 =
1
0
| G4 | =
0
1
1
1
1
0
1
1
1
1
0
1
1
0
= (1)
0
0
1
0
= (1)
0
0
1
1
1
0
1
0
1
1
1
0
= (1)
1
1
01
1
11
1
1 1 0
1 0 1
01
1
11
1
0 2 1
0 0 3/2
We nd
det(G2 ) =
and
det(G3 ) =
011
101
110
1
1
0
1
= (1)
1
1
. Then |G4 | is given by
1
0
0
1
1
1
1
1
0
1
1
1
1
0
1
0
= (1)
0
0
01
1
11
1
1 2 1
0 1 2
= (1)(1)(1)(2)(3/2) = 3 .
01
10
= 1 .
10
11
+1
11
10
= (1)(1) + 1(1) = 2 .
So by the induction hypothesis we have that det(Gn ) = (1)n1 (n 1).
Problem 21
Part (a): The rst statement is true since by applying elementary row opperations to the
AB
matrix
the pivots obtained will be determined from the matrices A and D only.
0D
Since the determinant is the product of the pivots it is equal to the products of the pivots
from A and D .
Part (b): Let our large block matrix be
1
0
1
1
03
12
1 1
22
which has submatrices given by A =
10
,B=
01
2
3
.
2
1
32
,C=
23
11
, and D =
12
1 2
. These have individual determinats given by |A| = 1, |B | = 5, |C | = 1, and
21
|D | = 5. The determinant of the large block matrix is given by 15, while the product of
|A||D | = 1 (5) = 5 = 15. In addition, the expression
|A||D | |C ||B | = 1(5) 1(5) = 10 ,
which is not equal to the true determinant 15 either.
Part (c): Computing AD CB we have
10
01
1 2
21
11
12
32
23
=
1 2
21
55
78
=
6 3
5 7
,
which has a determinant given by 42 15 = 27, which is not equal to the true value either.
Problem 22
Part (a): Assuming that the index k refers to how many rows and columns the matrix
Lk /Uk ) subsumes i.e. L1 /U1 are 1 1, L2 /U2 are 2 2, etc. Then since the matrix L is
lower triangular and constructed to have ones on its diagonal |Lk | = 1, for k = 1, 2, 3. The
determinant of Uk wil then be |U1 | = 2, |U2 | = 2 3 = 6, and |U3 | = 2(3)(1) = 6. In the
same way |Ak | = |Uk |.
Part (b): If A1 , A2 and A3 have determinants given by 2, 3 and 1 the pivots are given by
p1 = 2 , p2 =
3
1
1
= .
, p3 =
3
2
3
22
Problem 23
Taking the determinant of the left hand side of the and using the determinant rule that
row opperations dont change the the value of the determinant or the fact that the matrix
I
0
is lower triangular with ones on the diagonal we have that
CA1 I
LHS =
AB
CD
=
AB
CD
I
0
1
CA
I
A
B
0 D CA1 B
=
= |A||D CA1 B | ,
which is valid if A1 exists. The above equals
|AD ACA1 B | ,
by distributing |A| into the determinant |D CA1 B |. If AC = CA then this is equivalent
to
|AD CAA1 B | = |AD CB | .
Problem 24
Now
det(M ) = det
Noiw since det
AB A
0I
I0
B I
I0
B I
AB A
0I
= det
det
I0
B I
.
= 1 so that the above is give by
det
AB A
0I
= det(AB ) ,
From Problem 21. If A is a single row and B is a single column then AB is a scalar and
equals its own determinant. So we have that det(M ) = AB . For a 3 3 let A = 1 2
1
and B =
, so that M is given by
1
0 12
1 1 0 =
M=
1 0 1
1
1
12
0
0
1 2
1 0
01
Chapter 6 (Eigenvalues and Eigenvectors)
Section 6.1 (Introduction to Eigenvalues)
Problem 1
For the matrix A given
A=
0.8 0.3
0.2 0.7
we have 1 = 1 and x1 = (0.6, 0.4) and 2 = 1/2 and x2 = (1, 1). For the square of A i.e.
A2 given by
0.7 0.45
A2 =
0.3 0.55
we have 1 = 1 and x1 = (0.6, 0.4) and 2 = (1/2)2 and x2 = (1, 1). For A (given by)
0.6 0.6
0.4 0.4
A =
we have 1 = 1 and x1 = (0.6, 0.4) and 2 = 0 and x2 = (1, 1). To show why A2 is halfway
between A and A consider the common eigenvalues of all of them i.e.
x1 =
0.6
0.4
1
1
and x2 =
.
These two vectors are linearly independent and thus span R2 , that is they are a basis for R2 .
Consider the action of A2 and 1 (A + A ) on this particular basis of R2 . We have that
2
A2 x1 = 1x1 = x1
1
1
(A + A )x1 =
(1 + 1)x1 = x1
2
2
and
1
x2
4
11
1
=
( + 0)x2 = x2
22
4
A2 x2 =
1
(A + A )x2
2
1
Thus the action of A2 and 2 (A + A ) is the same on a basis of R2 and therefore the two
matrices must be identical.
Part (a): If we exchange two rows of A we obtain
A=
0.2 0.7
0.8 0.3
,
which has eigenvalues given by
0.2
0.7
0.8
0.3
=0
which when expanded can be factored into ( 1)(2 + 1) = 0 and therefore has solutions
given by = 1, and = 1/2. These are not the same as the eigenvalues of the original
matrix A which were 1, and 1/2.
Part (b): A zero eigenvalue means that A is not invertible. This property would not be
changed by elimination.
Problem 2
For the matrix A given
14
23
A=
we have eigenvalues given by the solutions of
1
4
2
3
= 0,
which when expanded gives ( 5)( + 1) = 0, so the two eigenvalues are given by = 5
and = 1. The eigenvectors for A are given by the nullspace for (rst for = 5)
4 4
2 2
x1
x2
=0
v1 =
1
1
.
In a similar way for = 1 we have
24
24
x1
x2
=0
v2 =
2
1
.
The eigenvalues of A + I are the eigenvalues of A plus 1 or = 6 and = 1. The
eigenvectors of A + I are the same as the eigenvectors of A.
Problem 3
For A dened by
A=
02
23
,
the eigenvalues are given by solving
2
2 3
= 0,
which simplies to ( 4)( 1) = 0, so = 4 and = 1. The eigenvectors of A are given
by the nullspaces of the following matrices (for = 4 rst and then = 1)
4 2
2 1
x1
x2
=0
v1 =
1
2
,
and
x1
x2
12
24
=0
v2 =
2
1
,
The eigenvalues of A1 are the inverses of the eigenvalues of A. When A has eigenvalues 1
and 2 its inverse has eigenvalues 1/1 and 1/2 . The eigenvectors of A1 are given by the
nullspace of the following operators (for = 1/4 rst and then for = 1)
3
4
1
2
and
1
4
3
4 + 1
1
2
1
2
1
4
1
2
1
=
=
1
4
1
2
1
2
1
1
2
1
4
1
2
1
v1 =
v2 =
2
1
1
2
,
,
These eigenvectors are the same as the eigenvectors of A. That A and A1 have the same
eigenvectors can be seen from the simple expression Ax = x, which when we divide both
sides by and multiply by A1 gives
1
x = A1 x ,
1
showing that x is an eigenvector of A1 with eigenvalue .
Problem 4
For A given by
A=
1 3
20
we have eigenvalues given by the solutions to
1 3
2
=0
or 2 + + 6 = 0, which factors into ( + 3)( 2) = 0, giving the two values of = 3 or
= 2. The eigenvectors are then given by the nullspaces of the following operators.
23
23
and
3 3
2 2
or x =
or x =
3
2
1
1
From these, the eigenvalues of A2 are given by (3)2 = 9 and 22 = 4, with the same
eigenvectors as A. This is because when A has eigenvalues i , A2 will have eigenvalues 2 .
i
Problem 5
For A we have eigenvalues given by
1
0
1
1
= 0 (1 )2 = 0 = 1 .
For B we have eigenvalues given by
1
1
0
1
= 0 (1 )2 = 0 = 1 .
For the matrix A + B we have eigenvalues given by
2
1
1
2
= 0 (2 )2 1 = 0 = 1 , 3 .
So the eigenvalues of A + B are not equal to the eigenvalues of A plus the eigenvalues of B .
This would be true if A and B has the same eigenvectors which will happen if and only if A
and B commute, i.e. AB = BA. Checking this fact for the matrices given here we have
AB =
10
11
11
01
=
11
12
BA =
11
01
10
11
=
21
11
while
which are not equal so consequently A and B cant have the same eigenvectors.
Problem 6
From Problem 5 the eigenvalues of A and B are 1. The eigenvalues of the product AB are
given by
1
1
= (1 )(2 ) 1 = 0 ,
|AB I | =
1
2
which has roots given by
3 5
=
.
2
The eigenvalues of BA are given by
|BA I | =
2
1
1
1
= (1 )(2 ) 1 = 0 ,
which has the same roots as before and therefore BA has the same eigenvalues as AB . We
note that the eigenvalues of AB /BA are not equal to the product of the eigenvalues of A
and B . For this to be true A and B would need to have the same eigenvectors which they
must not.
Problem 7
The eigenvalues of U are on its diagonal. They are also the pivots of A. The eigenvalues
of L are on its diagonal, they are all ones. The eigenvalues of A are not the same as either
the eigenvalues of U or L or the product of the eigenvalues of U and L (which would be the
same as the product of the eigenvalues of U since the eigenvalues of L are all ones).
Problem 8
Part (a): If we know that x is an eigenvector one way to nd is to multiply by A and
factor out x.
Part (b): If we know that is an eigenvalue one way to nd x is to determine the nullspace
of A I .
Problem 9
Part (a): Multiply Ax = x by A on the left to obtain
A2 x = Ax = 2 x
1
Part (b): Multiply by A1 on both sides to get
1
x = A1 x
Part (c): Add Ix on both sides of Ax = x to get
(A + I )x = x + Ix = ( + 1)x ,
which shows that + 1 is an eigenvalue of A + I .
Problem 10
For A the eigenvalues are given by
|A I | =
0.6
0.2
0.4
0.8
= 0 (0.6 )(0.8 ) 0.08 = 0 .
which gives 2 1.4 + 0.4 = 0. To solve this we know that = 1 because A is a Markov
matrices. The other root can be found by using the quadratic equation or factoring out the
known root = 1 from the above quadratic. When that is done one nds that the second
root is given by = 2 = 0.4. The eigenvectors for = 1 are given by considering the
5
nullspace of the operator
0.4 0.2
,
AI =
0.4 0.2
which has a nullspace given by the span of
1
2
.
For = 0.4 we have A I given by
0.2 0.2
0.4 0.4
,
which has a nullspace given by the span of
1
1
.
= 0 and the same
For the matrix A our eigenvalues are given by 1 = 1 and = 2
5
eigenvectors as A. Now A is obtained from the diagonalization of A i.e. A = S S 1 .
Which given the specic matrices involved is
A=
1 1
21
1
1+2
10
02
5
11
2 1
1
3
11
2 1
So that A is given by
1 1
21
A =
1
3
1
=
3
10
00
10
20
=
11
2 1
11
22
1
3
2
3
=
1
3
2
3
.
.
So A100 is then given by A100 = S 100 S 1 or
100
A
=
=
1100
0
1 1
21
1
3
0
1
0
1 1
21
0
12
5
2 100
2
5
1
=
3
1+2
22
=
1
3
2
3
1
3
2
3
2 100
5
2 100
5
+
2
5
which we see is a slight perturbation of A
11
2 1
11
2 1
2 100
5
100
1
=
3
1
3
2 100
5
11
2 1
1
2+
100
2 100
5
2 100
5
2
3
2
3
1
3
1
3
,
Problem 11
Now P is a block diagonal matrix and as such has
the block matrices on its diagonal. Since = 1 is
0.2
matrix and the upper right block is given by
0.4
solving for the roots of
0.2
0.4
0.4
0.8
eigenvalues given by the eigenvalues of
the eigenvalue of the lower right block
0.4
, which has eigenvalues given by
0.8
= 0 (0.2 )(0.8 ) 0.16 = 0
Multiplying this polynomial out we obtain 2 = 0 or = 0 and = 1 as its roots. Now
the eigenvectors for = 1 are given by computing an appropriate null space. We nd
0.8 0.4 0
1 0.5 0
1 0.5 0
0.4 0.2 0 1 0.5 0 0
0
0,
0
0
0
0
0
0
0
0
0
0
1
0 , and another is given by 2 . For the eigenvalue
so one eigenvector is given by
1
0
given by = 0 we have
0.2 0.4 0
120
120
0.4 0.8 0 1 2 0 0 0 0 ,
0
01
001
001
2
so the nal eigenvector is given by 1 . For P 100 we have the same eigenvectors as for
0
P and the eigenvalues given by 0100 = 0 and 1100 = 1. Thus everything for P 100 is the same
as for P . If two eigenvectors sharethe same so do all linear combinations of the
then
0
1
0 and v2 = 2 share the same eigenvalue of = 1
eigenvectors. Thus since v1 =
1
0
1
so will their sum v1 + v2 = 2 , which has no zero components. We can check this by
1
computing
1
0.2 + 0.8
1
2 = 0.4 + 1.6 = 2
P
1
1
1
Problem 12
The rank one projection matrix is given by P = uuT , so P is given
1
113
1 1 1
11 1 3
1135 =
P =
6 3 6
36 3 3 9
5
5 5 15
Part (a): Now P u is given by
113 5
113 5
1 1 3 5 1 1 3 5
1
Pu =
36 3 3 9 15 3 3 9 15
5 5 15 25
5 5 15 25
1 + 1 + 9 + 25
36
1 1 + 1 + 9 + 25
1 36
=
= 3
6 3 + 3 + 27 + 75 63 108
5 + 5 + 45 + 125
180
Thus u is an eigenvector with eigenvalue one.
by
5
5
15
25
1
1
1
6 3
5
1
1 1
= .
6 3
5
Part (b): If v is perpendicular to u then uT v = v T u = 0 and P v = u(uT v ) = u 0 = 0 so v
is an eigenvector with eigenvalue = 0.
Part (c): To nd three independent eigenvectors of
need to nd three vectors perpendicular to u which
satisfy
x1
x2
1135
x3
x4
P all with eigenvalues equal to zero we
means that each of these vectors must
=0
or the three vectors that span the nullspace of A (where A is dened to be A = 1 1 3 5 .
Three vectors in the nullspace are given by assigning a basis to the variables x2 , x3 , and
x4 and computing x1 from these. We nd
x2 = 1 , x3 = 0 , x4 = 0 x1 = 1
x2 = 0 , x3 = 1 , x4 = 0 x1 = 3
x2 = 0 , x3 = 0 , x4 = 1 x1 = 5 .
Which gives the three vectors
1
1
0
0
3
0
,
1 ,
0
and
5
0
0 .
1
Problem 13
We nd that
det(Q I ) =
cos() sin()
sin()
cos()
= 0,
or when expanding the above determinant we nd the characteristic equation for Q is given
by
(cos() )2 + sin2 () = 0
or when expanding the quadratic we nd that
2 2 cos() + 1 = 0 ,
which using the quadratic equation gives for
4 cos2 () 4
2
= cos() cos2 () 1
= cos() i sin() .
=
2 cos()
To nd the eigenvectors we solve (Q I )x = 0, which is given by
Q I =
=
cos() (cos() i sin())
sin()
sin()
cos() (cos() i sin())
i sin()) sin()
sin()
i sin())
= sin()
i 1
1 i
which has eigenvectors given by v1,2 =
v1 =
i
1
.
i
1
i.e.
and v2 =
i
1
Problem 14
The matrix
010
P = 0 0 1 ,
100
will have eigenvalues given by the solution to
1
0
0 1
1
0
= 0.
.
2
This simplies to 3 + 1 = 0 and has solutions given by = e 3 ik for k = 0, 1, 2. This gives
1 = 1
3
1
2 = e = + i
2
2
4
3
1
3 = ei 3 = i
2
2
001
P = 0 1 0 ,
100
i 23
For the matrix
will have eigenvalues given by the solution to
0
1
0 1 0
1
0
= 0.
This simplies to 3 + 2 + 1 = 0, which has = 1 as a root. Long division gives a
factorization of ( 1)2 ( + 1) = 0.
Problem 15
Consider the polynomial det(A I ) factored into its n factors as suggested in the text, ie.
n
det(A I ) =
i=1
( i ) .
Evaluating this polynomial at = 0 we obtain
n
i .
det(A) =
i=1
Problem 16
If A has 1 = 3 and 2 = 4 then
det(A I ) = (1 )(2 )
= 1 2 ( 1 + 2 ) + 2 .
so let
det(A I ) = 12 7 + 2 .
The quadratic formula gives then
1 =
2 =
a+d+
a+d
(a + d)2 4(ad bc)
2
(a + d)2 4(ad bc)
.
2
Then 1 + 2 =
a + d = 1 + 2 .
2(a+d)
2
= a + d, which is the linear term in the determinant equation i.e.
Problem 17
We can always generate matrices with any specied eigenvalues by constructing them from
S
40
05
S 1 ,
with dierent choices for the eigenvector matrices S . For example pick eigenvectors given by
1
2
and
1
1
.
Then our matrix A is given by
A=
1
3
1
=
3
=
1 1
21
4 5
85
14 1
2 13
1
(1 + 2)
11
2 1
40
05
11
2 1
.
Note that other matrices can be generated in the same manner.
Problem 18
Part (a): the rank of A cannot be determined
let A be given by
00
A= 0 1
00
from the given information. For example,
0
0,
2
then A is diagonal and has eigenvalues as given and A has rank of two. Also consider A
given by A = S S 1 as
1 1 1
000
1/2 1/2 1/2
1
0
A = 1 0 1 0 1 0 1
210
002
1/2 3/2 1/2
0 2 1
= 1 3 1
1 1 0
This matrix has rank of three as can be seen by
0 2 1
1 3
1 3 1 0 2
1 1 0
1 1
13
0 2
0 2
which has rank three.
the following transformations
1
131
1 0 2 1
0
1 1 0
1
13 1
1 0 2 1 ,
1
0 0 2
Part (b): We nd |B T B | = |B T ||B | = |B |2 = (0 1 2)2 = 0.
Part (c): The eigenvalues of B T B are given by 02 , 11 , and 22 or 0, 1, and 4.
Part (d): The eigenvalues of B + I are the eigenvalues of B plus one, which gives 1, 2, and
3. The eigenvalues of (B + I )1 are the inverses of the eigenvalues of B + I and are given
1
by 1, 1 , and 3 .
2
Problem 19
Let our matrix A be given by
A=
01
cd
then the trace of A must equal 0 + d = d = 1 + 2 = 4 + 7 = 11, giving that d = 11.
Also the determinant of A must equal |A| = c = 1 2 = 28, so c = 28. Thus we have
determined A and it is
0
1
A=
.
28 11
Problem 20
Let A be given by
010
A= 0 0 1 .
abc
Then if the eigenvalues are 3, 0, and 3 we must have trace(A) = 0+0+c = c = 1 + 2 + 3 =
0 (or c = 0) and
01
= a = 1 2 3 = 0 .
det(A) =
a0
Now from what we know about A we can now conclude that
010
A= 0 0 1 .
0b0
Now computing the characteristic equation for A we have that
1
0
0 1
0
b
|A I | =
=
1
b
so we have that b = 9 and our matrix A is given
01
0 0
A=
09
= (2 b) = 3 b ,
by
0
1.
0
Problem 21
We have that det(A I ) = det(AT I ), since I T = I . Now let A =
10
11
and
11
, these are the examples from Problem 5 in this section. Then both A
01
and B have = 1 with algebraic multiplicity of two. The eigenvectors of A can be computed
by computing a basis for the nullspace of the operator A I . We have that
B = AT =
A I =
or the span of
00
10
,
0
. The eigenvectors of AT are given by a basis for the nullspace of AT I .
1
We nd that
AT I =
01
00
,
1
. Since these vectors are obviously not equivalent the eigenvectors of A
0
and AT are dierent.
or the span of
Problem 22
We have
0.6 0.8 0.1
M = 0.2 0.1 0.4 .
0.2 0.1 0.5
1
so that we nd M T 1 given by
1
1
0.6 0.2 0.2
1
1
M T 1 = 0.8 0.1 0.1 1 = 1 .
1
0.1 0.4 0.5
1
1
So we know that M T has an eigenvalue given by = 1, therefore M must have an eigenvalue
= 1. Since a three by three singular Markov matrix must have two eigenvalues equal to
1
zero and one and also must have trace(M ) = 2 we know that our third eigenvalue must
satisfy
1
0+1+= ,
2
1
showing that = 2 as the third eigenvalue. To assemble M construct it from its eigenvalues
by assigning random eigenvectors i.e. use the relationship M = S S 1 . Now we can simplify
T
things
some by working with M which has the same eigenvalues and where we know that
1
1 is the eigenvector corresponding to = 1. Thus
1
1 1 0
100
1
MT = 1 0 1 0 2 0 .
000
110
To compute the inverse of S we augment M T with
hand side to the identity. We nd
1
1 1 0 1 0 0
1 0 1 0 1 0 0
0 0 1001
0
1
0
0
1
0
0
1
0
0
Thus our inverse is given by
S 1
So that we nd that
MT
1
= 1
1
1
=
2
1
=
2
the identity matrix and reduce the left
1 0 1 0 0
1 1 1 1 0
2 0 1 0 1
01
0
10
1 1 1 1 0
0 2 1 2 1
01 0 1 0
1 1 1 1 0
0 1 1 1 1
2
2
1
01 1 0 2
2
1 0 1 0 1 .
2
2
0 1 1 1 1
2
2
101
1
= 1 0 1
2
1 2 1
101
1 0
100
1
0 1 0 1 0 1 0 1
2
2
1 2 1
000
10
1
120
101
1 0 1
100
1 1 0
1 2 1
2
1
1
3
02
03
2
4
4
1
1 0 1 = 2 0 1 ,
2
3
1
3
02
01
2
4
4
which is a valid Markov matrix.
Problem 23
ab
00
01
a general matrix where we would
, and A3 =
, A2 =
cd
10
00
like to determine a, b, c, and d. To do this, since 1 = 2 = 0 we have that from the trace
and determinant identities that
Let A1 =
0 = a + d a = d
0 = ad cd 0 = d2 cb d2 = cb .
We can nd a solution that satises this by letting a = 1, d = 1, so that cb = 1 and we
can take c = 1 and b = 1 obtaining
A3 =
1
1
1 1
Then checking that the eigenvalues of A3 are as they should be we nd that setting |A3 I | =
0 that
1
1
1 1
= (1 )(1 ) + 1
= 1 + 2 + 1 = 2
|A3 I | =
Now for each Ai we will check that A2 = 0. For A1 we have that
i
01
00
01
00
=
00
00
.
00
10
00
10
=
00
00
.
For A2 we have that
For A3 we have that
1
1
1 1
1
1
1 1
=
00
00
.
In general when a = d and d2 = cb then we have
d b
cd
d b
cd
=
d2 + bc db + bd
cd + dc cb + d2
=
00
00
.
Problem 24
We know since A is singular that at least one eigenvalue is zero. A corresponding eigenvector
is given by any vector x such that
x1
2 1 2 x2 = 0 .
x3
Two such vectors are
1
2
0
and
1
0 .
1
A third eigenvector/eigenvalue combination in the rank one case (like we have here) is
1
2.
1
This is because with this vector we have that
1
1
1
1
2 2 1 2 2 = 2 (2 + 2 + 2) = 6 2 .
Ax =
1
1
1
1
1
2 is an eigenvector with eigenvalue six.
So x =
1
Problem 25
Note that Ax = A( i ci xi ) = i ci Axi = i ci i xi , and Bx = i ci i xi by the same logic.
Since A and B have the same action on any vector x, they must represent the same linear
transformation thus A = B .
Problem 26
Consider the expression |A I | we have
BC
0D
I 0
0 I
=
B I
C
0
D I
= |B I | |D I | ,
since the lower left hand corner of A I is the zero matrix. We see that this expression
vanishes whenever |B I | = 0 or |D I | = 0 which happen when = 1, 2 or = 5, 7
respectively. Thus the eigenvalues of A are given by 1, 2, 5 and 7.
Problem 27
1
For our A since A = 1 1 1 1 1 we see that A is rank one with three eigenvalues
1
given by zero (counted according to multiplicity) and one eigenvalue given by
1
1 1 1 1 1 = 4.
1
For rank one metrics we can easily compute the eigenvectors since they are given by the null
vectors of the operator
1111 .
these are given by
1
1
0
and 0 ,
,
0
1
1
0
1
1
each with eigenvalue zero and the vector with eigenvalue four. For C we see that it
1
1
has a rank of two and thus is not invertible and so one eigenvalue is zero. Since the sum of
the rank plus the nullity of C must equal to four we know that the nullspace is of dimension
two. Two vectors that span this space are given by
0
1
0
and 1 .
0
1
1
0
1
1
0
0
The other vectors with eigenvalues of two are given by
0
1
1
0
and .
0
1
1
0
Problem 28
Since the eigenvalues of A were given by 0 with algebraic multiplicity 3 and 4 with algebraic
multiplicity 1, the eigenvalues of AI are -1 with algebraic multiplicity 3 and 3 with algebraic
multiplicity 1. If A is a 5x5 matrix of all ones, then A has eigenvalue 0 with multiplicity 4
and a single eigenvalue with value 5. A I will have 4 eigenvalues with value -1 and a single
eigenvalue with value 4. The determinant of B is given by (1)3 3 = 3. The determinant
of B with it is ve by ve is given by (1)4 4 = 4.
Problem 29
12
0 4
For A =
00
diagonal and are
3
5 (an upper triangular matrix) the eigenvalues can be read o of the
6
given by 1, 4, and 6. For B computing the characteristic equation we have
|B I | =
0
1
0 2 0
3
0
0 2
2 0
+1
3
0
0
= ((2 )) 3(2 )
= 3 + 22 + 3 6 .
=
From the expression for the determinant we see that = 2 must be a root of the above cubic
equation. Factoring our 2 from the above we see that the characteristic equation is equal
to ( 2)(2 + 3), so the other two roots are = 3. For C we recognize it as a rank
one matrix like
2
2 1 1 1 ,
C=
2
which has an eigenvalue/eigenvector combination given by
1
1
= 0 with 1 and 0
0
1
and
= 6 with
2
2
2
Problem 30
1
a+b
1
1
=
= (a + b)
, and we see that the vector
is an
1
c+d
1
1
eigenvector of A with eigenvalue a + b. Computing the characteristic equation of A i.e.
|A I | we nd that
Consider A
a
b
c
d
= (a )(d ) bc
= 2 (a + d) + (ad bc) .
|A I | =
Setting this to zero and solving using the quadratic equation we nd that
(a + d)2 4(ad bc)
2
2 + 2ad + d2 4ad + 4bd
(a + d ) a
=
2
2 2ad + d2 + 4bd
(a + d ) a
.
=
2
From our one relationship among a, b, c, and d replace a with a = c + d b to obtain
=
(a + d )
(c + d b)2 2(c + d b)d + d2 + 4bc
.
2
When we expand the terms in the under the radical in the above we nd that they simplify
to (c + b)2 , and our expression for then becomes
=
c + 2d b
=
c + 2d (c + b)
=
2
2c+2d
2
2d2b
2
=c+d
= db
The rst expression c + d is what we found before. The second eigenvalue is given by d b. A
much easier way to calculate this value is to recognize that tr(A) = 1 + 2 = a+ b+ 2 = a+ d,
so solving for 2 we nd that 2 = d b.
Problem 31
010
To exchange the rst two rows and columns of A let P = 1 0 0 . Considering the
001
nullspace of
1
1
10 2
1
1 5 10
A 11I = 3 5 3 3 5 3
4
8 7
4 8 7
1
1
1
1
1 5 10
1 5 10
0 22 33 0 1 3
5
10
4
0 44 33
0 1 3
5
5
4
1
1
1 0 4
1 1 10
5
0 1 3 0 1 3 ,
4
4
00
0
00 0
1
which has a nullspace given by 3 . For the matrix P AP we have
4
5 3
3
P AP 11I = 2 10 1 ,
8
4 7
which would be worked in the same way as earlier.
Problem 32
Part (a): A basis for the nullspace is given by the span of u. A basis for the column space
is is given by a span of {v, w }
1
Part (b): Let x = 1 v + 5 w , then
3
1
3
5
1
Ax = Av + Aw = v + w = v + w .
3
5
3
5
Then all solutions are given by
1
1
x = Cu + v + w .
3
5
Part (c): Ax = u will have a solution if and only if u is in the same column space as A.
This means that u Span{v, w }, or that
u = C1 v + C2 w .
This implies that u, v , and w are linearly independent in contradiction to the assumed
independence of u, v , and w .
Section 6.2 (Diagonalizing a Matrix)
Problem 1
To factor A = S S 1 we rst compute the eigenvalues and eigenvectors of A. The eigenvalues
are given by nding the roots of the characteristic equation |A I | = 0, which in this case
becomes
1
2
= (1 )(3 ) = 0 .
|A I | =
0
3
or = 1 or = 3. Then the eigenvectors associated with eigenvalue = 1 is given by the
1
02
. The eigenvector associated with
, which is
nullspace of A I or the matrix
0
02
1
2 2
. Thus
or
eigenvalue = 3 is given by the nullspace of the matrix A 3I or
1
00
the matrix whos columns are given by the eigenvectors is given by
S=
11
01
so that S 1 is given by
S 1 =
1 1
01
.
Thus A is given by
A=
11
01
10
03
1 1
01
.
This can easily be checked by multiplying the matrices above. For the matrix
A=
11
22
,
Computing its eigenvalues we have to consider
1
1
2
2
|A I | =
= 0.
Expanding the determinant of the above we have this equal to
( 3) = 0 ,
so we see that = 0 or = 3. The eigenvalue associated with = 0 is given by the nullspace
1
11
. The eigenvector associated with = 3 is given
which is
of A or the matrix
1
22
2 1
. This matrix has a nullspace given
by the nullspace of A 3I i.e. the matrix
2 1
1
. Thus the matrix S whos columns are the eigenvectors of A is given by
by the span of
2
S=
11
1 2
so S 1 =
1
3
2 1
11
.
Then we see that we can decompose A into the product S S 1 as
A = S S 1 =
11
1 2
00
03
2
3
1
3
1
3
1
3
,
which again can be checked by multiplying the matrices above together.
Problem 2
If A = S S 1 then
A3 = (S S 1)(S S 1 )(S S 1) = S 2 S 1 = S 3 S 1 ,
and
A1 = (S S 1 )1 = S 1 S 1 .
Problem 3
Then A can be assembled from its eigenvectors and eigenvalues by A = S S 1 . We have
11
1 1
S=
so S 1 =
, and then A is given by
01
01
A=
11
01
20
05
1 1
01
=
25
05
1 1
01
=
23
05
.
Problem 4
If A = S S 1 the the eigenvalue matrix for A is . The eigenvalue matrix for A + 2I is
given by + 2I . The eigenvector matrix for A + 2I is the same as that for A i.e. the matrix
S . These are shown by the manipulations
S ( + 2I )S 1 = S S 1 + 2SS 1 = A + 2I .
Problem 5
Part (a): False, A can still have an eigenvalue equal to zero.
Part (b): True, the matrix of eigenvectors S has an inverse.
Part (c): True, S has full rank and is therefore invertible.
Part (d): False, since S could have repeated eigenvalues and therefore possibly a non
complete set of eigenvectors.
Problem 6
Then A is a diagonal matrix since S = I = S 1 and A = S S 1 = . If the eigenvector
matrix S is triangular then S 1 is also triangular. Forming the product A = S S 1 we see
that left multiplying a triangular matrix S 1 onto is multiplication of the the rows of S 1
by the diagonal elements of the product S 1 is also triangular. Since S and S 1 are
both triangular their product is triangular and therefore A is triangular.
Problem 7
if A =
40
12
then A has eigenvectors given by
|A I | =
4
0
1
2
= (4 )(2 ) = 0 .
Which has solutions = 2 or = 4. The eigenvector associated with the eigenvalue = 2
0
20
. The eigenvector
which is
is given by the nullspace of A 2I or the matrix
1
10
00
. Which
associated with = 4 is given by the nullspace of A 4I i.e. the matrix
1 2
2
. Thus all matrices that diagonalize A are given
has a nullspace given by the span of
1
by
0 2
S=
so S 1 =
1
(2 )
2
0
21
=
1
2
1
0
.
The matrices that diagonalized A are the same ones that diagonalize A1 so the S and S 1
above apply to the diagonalization of A1 also.
Problem 8
We can assemble A from its eigenvectors using S S 1 . We nd
1
2
11
1 0
1 1
0 2
1
1 1
1 2
=
1 1
1 2
2
1 1 + 2 1 2
=
2 1 2 1 + 2
A = S S 1 =
1 1
1 1
Problem 9
If A =
11
10
then
A2 =
11
10
11
10
21
11
=
.
In addition, A3 is given by
A3 = AA2 =
11
10
21
11
=
32
21
,
A4 = AA3 =
11
10
32
21
=
53
32
.
and A4 is given by
Since F0 = 0, F1 = 1, F2 = 1, we have that if we dene the vector un as
un =
Then
un+1 =
Fn+2
Fn+1
=
Fn+1
Fn
Fn+1 + Fn
Fn+1
=
,
11
10
Fn+1
Fn
= Aun .
F1
1
=
and iterating un+1 = Aun we see that un = An u0 . If we want to
F0
0
compute F20 we extract the second component from u20 . Since u20 = A20 u0 , it will help to
have u0 written in terms of the eigenvectors of A. Doing this gives
With u0 =
u0 =
x1 x2
,
1 2
with x1 =
1
1
and x2 =
2
, so that u20 becomes
1
u20 =
20 x1 20 x2
1
2
.
1 2
11
10
Now is since for the Fibonacci matrix
we have
1 5
and 2 =
,
2
1+ 5
1 =
2
the value of F20 is given by
20
1
20
2
20
1
1+ 5
=
1 2
2
5
1 5
2
20
.
Problem 10
If Gk+2 = 1 (Gk + Gk+1 ) then dening
2
uk =
we have that
uk+1 =
Gk+2
Gk+1
Gk+1
Gk
,
1
(G k
2
+ Gk+1 )
Gk+1
=
=
1
2
1
2
10
.
so that we have A given by
1 / 2 1/ 2
1
0
A=
.
The eigenvalues and eigenvectors of A are given by
|A I | =
1/2 1/2
1
=
1
1
=0
2
2
Thus we have solving for that = 1 and = 1. The eigenvectors are given by the
2
nullspace of the operator A I . For = 1 this is the matrix
2
1 1/ 2
1 1/ 2
which has a nullspace given by the span of
,
1
. For = 1 the matrix A I is
2
1/2 1/2
1
1
,
1
.
1
which has a nullspace given by the span of
Part (b): Powers of A can be obtained by An = S n S 1 , with
S=
11
2 1
1
3
1 1
21
11
2 1
1
2
0
and S 1 =
.
We then compute that An is given by
n
A
1
3
1
3
n
1
2
2 1
2
1 1
21
1
1
n
=
n
1
2 + 2
1
2
n
2 1 + 2 1
2
2
n
n
+1
+1
A =
Part (c): If G0 = 0 and G1 = 1 then u0 =
1
. Thus G =
1
2
3
1
3
0
1
1 1
21
.
From which we see that
2
3
n
1
3
21
21
G1
G0
.
1
, so that u = A u0 =
0
=
2
the Gibonacci numbers approach 3 .
Problem 11
From the given pieces of the eigenvector decomposition A = S S 1 we recognize
1 2
11
S=
and S 1 =
so we have the decomposition of
11
10
=
1 2
11
1 0
0 2
1
1 2
k
=
1 2
11
k 0
1
0 k
2
1 2
1 1
.
1
1 2
1 2
1 1
.
From which we recognize that the requested multiplication is given by
S k S 1
1
0
,
1
1 2
Then powers of A are easy to compute. We nd that
11
10
1 2
1 1
1
1
1
1 2
1
k
1
=S
1 2 k
2
k +1
1
1 k+1
2
=
k k
1 2
1
2
= S k
.
1
3
2
2
=
Which has a second component given by Fk =
k k
1
2
.
1 2
Problem 12
The original equation for the s is the characteristic equation given by
2 1 = 0 ,
Since solutions to the quadratic equation we see that multiplying by k this equation can be
written as
k+2 k+1 k = 0 ,
or
k+2 = k+1 + k .
Then the linear combination of k and k must satisfy this. Thus
1
2
Fk =
k k
1
2
,
1 2
So Fk will satisfy this recurrence relation and has values F0 = 0 and F1 =
1 2
1 2
= 1.
Problem 13
Dening u0 =
F1
F0
=
1
2
= x1 + x2 , then
u20 = A20 u0 = A20 (x1 + x2 ) = 20 x1 + 20 x2
1
2
2
1
+ 20
= 20
2
1
1
1
So the second component of this vector is given by 20 + 20 . Thus
1
2
F20 =
1+ 5
2
20
+
1 5
2
20
.
Problem 14
Given Fn+2 = Fn + Fn+1 with initial conditions F0 = 0 and F1 = 1, we would like to
prove that F3n is an even number. One might be able to prove this by using the explicit
representation of the Fibonacci numbers but it will probably be easier to prove by induction.
Sine F3 = 2 we have a starting condition of an induction proof to be true. Then assuming
that F3k is an even number for k n we desire to show that it is even for F3(n+1) . Now
consider F3(n+1) we have using the Fibonacci recurrence that
F3(n+1) =
=
=
=
F3n+3
F3n+2 + F3n+1
F3n+1 + F3n + F3n+1
F3n + 2F3n+1 .
Thus since F3n is even (by the induction hypothesis and 2F3n+1 is even we see that F3(n+1)
is even. Thus our result is proven.
Problem 15
Part (a): True, = 0 and therefore A is invertible.
Part (b): This is possible but not denite. If the repeated eigenvalue has enough eigenvectors which is not in general true.
Part (c): It is possible if the = 2 eigenvalue does not have enough eigenvectors.
Problem 16
Part (a): False, the multiple eigenvector could correspond to a nonzero eigenvalue.
Part (b): This must be true of else if not we would have another distinct eigenvector.
Part (c): This is true. There are not enough eigenvectors to ll the eigenvector matrix S .
Problem 17
For the rst matrix A =
8b
c2
since det(A) = 1 2 = 25 we have that
16 bc = 25 ,
or that bc = 9. Pick b = 1 and c = 9 giving A =
|A I | =
8
1
9 2
81
. Then
92
= (8 )(2 ) + 9 = ( 5)2 .
An eigenvector for = 5 is given by the nullspace of the operator A 5I which is the matrix
31
1
or
. This matrix has only one eigenvector as requested. For the matrix
9 3
3
94
we must have Tr(A) = 10 = 1 + 2 = 10 (which is true) and det(A) = 9 4c = 25
c1
or c = 4. Thus our matrix A is given by
A=
94
4 1
,
then the characteristic equation for A is given by
|A I | =
(9 )(1 ) + 16
9
4
4 1
= ( 5)2 ,
as expected. We also have the eigenvectors for this matrix A given by the nullspace of A 5I ,
1
4
4
. Finally, for the matrix
or the vector
which in this case is the matrix
1
4 4
10 5
the determinant requirement gives
A=
5 d
10d + 25 = 25 ,
or d = 0 so A =
10 5
. Then the characteristic equation for A is given by
5 0
|A I | =
(2 10 + 25) = ( 5)2 ,
10 5
5
An the eigenvectors are given by the nullspace of A 5I or the matrix
vector
1
1
5
5
5 5
or the
Problem 18
The rank of A 3I is one and therefore since the rank plus the dimension of the nullspace
must equal two we see that the nullspace has a dimension of 2 1 = 1 and therefore there
does not exist a complete set of eigenvectors for the = 3 eigenvalue. If we changed the (1, 1)
or the (2, 2) element to 3.01 then the eigenvalues of A are given by 3 and 3.01 and since they
are dierent we are guaranteed to have independent eigenvectors and A is diagonalizable.
Problem 19
If every has a magnitude less than one. Since A is a Markov matrix it has eigenvalues
equal to one and therefore will not iterate to zero. For B it has eigenvalues given by solving
|B I | = 0 or
0.6
0.9
= (0.6 )2 0.09 = 0 ,
0.1
0.6
or = 0.3 or = 0.9. Since |i| < 1 we have Ak 0 as k .
Problem 20
For A in Problem 19 we know since it is a Markov matrix that one eigenvalue is equal to
one. Thus from the trace/determinant formulas its eigenvalues must satisfy
1 + 2 = 1.2 and 1 2 = 0.36 0.16 = 0.2 .
Thus we see that if 1 = 1 then 2 = 0.2. The eigenvector for 1 = 1 is given by the nullspace
0.4 0.4
1
of A I =
or the span of the vector
. For 2 = 0.2 the eigenvector
0.4 0.4
1
0.4 0.4
is given by the nullspace of the matrix A 0.2I =
or the span of the vector
0.4 0.4
1
. Thus our matrix of eigenvectors is given by
1
S=
11
1 1
,
with S 1 given by
S 1 =
1
1 1
1 1
1 1
1 / 2 1/ 2
1/2 1/2
=
,
so that we have our eigenvalue decomposition given by A = S S 1
A=
Thus since
k =
11
1 1
1k 0
0 0.2k
=
1 / 2 1/ 2
1/2 1/2
10
0 0.2
10
0 0.2k
10
00
.
ask ,
the limit of Ak as k is given by
11
1 1
10
00
1 / 2 1/ 2
1/2 1/2
=
11
1 1
1 / 2 1/ 2
0
0
=
1
2
11
11
which has the eigenvector corresponding to the = 1 eigenvalue in its columns.
,
Problem 21
The eigenvalues for B in Problem 19 are given by 1 = 0.3 and 2 = 0.9. For = 0.3 the
3
0.3 0.9
. For 2 = 0.9
or the span of
eigenvectors are given by the nullspace of
1
0.1 0.3
3
0.3 0.9
. Thus to
or the span of
the eigenvectors are given by the nullspace of
1
0.1 0.3
evaluate B 10 u0 we decompose u0 in a basis provided by the eigenvectors of B . Doing this in
matrix form we have
336
1 1 0
c1 c2 c3
1
1
1
c1 c2 c3
2
2
2
3 3
11
=
,
where I have concatenated the coecient vectors used to expand each u0 . For example
3
1
= c1
1
3
1
+ c1
2
3
1
.
Then this matrix of coecients is given by
c1 c2 c3
1
1
1
c1 c2 c3
2
2
2
=
or
1
(3 3)
1 3
1 3
336
1 1 0
3
1
0 1 1
10
1
= 1x1
3
1
=
= x1
6
0
= x1 + x2 .
Which could have been obtained by inspection. Thus since B 10 = S 10 S 1 , we have that
1 3
3 3
that
and S 1 = 1
since S =
6
1 3
11
B 10 =
3 3
11
0.310
0
0
0.910
=
30.310 3(0.9)10
(0.3)10
0.910
=
1
(0.3)10 + 1 0.910
2
2
1 (0.3)10 + 1 0.910
6
6
1/6 1/2
1 / 6 1/ 2
1/6 1/2
1 / 6 1/ 2
3
2 (0.3)10 + 3 (0.9)10
2
1
1
(0.3)10 + 2 (0.9)10
2
.
And more specically we nd that
B 10
B 10
3
1
3
1
= B 10 x2 = 10 x2 = (0.9)10 x2 = (0.9)10
2
= B 10 (x1 ) = B 10 x1 = 10 x1 = (0.3)10
1
3
1
3
1
,
= (0.3)10
3
1
,
and nally that
B 10
6
0
= B 10 (x1 + x2 ) = B 10 x1 + B 10 x2
= 10 x1 + 10 x2
1
2
3
+ (0.9)10
= (0.3)10
1
3
1
.
Problem 22
A has eigenvalues given by the roots of
2
1
1
2
= 0.
Expanding the determinant above we nd that the characteristic equation for A is given by
(2 )2 1 = 0 ,
which has = 1, and = 3 as solutions. For the eigenvalue 1 = 1 the corresponding
eigenvector is given by the nullspace of the matrix
11
11
,
1
. The eigenvalue 2 = 3 the corresponding eigenvector is
1
given by the nullspace of the matrix
or the span of the vector
1 1
1 1
or the span of the vector
S=
,
1
. Thus our matrix S and S 1 are given by
1
1
2
1 1
11
1
10
k
03
2
k
1 1 1
13
=
k
1 3
211
1
1 + 3k 1 + 3k
=
2 1 + 3k 1 + 3k
1 1
11
11
1 1
and S 1 =
With these we see that Ak is given by
Ak = S k S 1
11
=
1 1
.
Problem 23
Since B is upper triangular the eigenvalues of B are given by the elements on the diagonal
and are therefore 3 and 2. The eigenvector for = 2 is given by the nullspace of
11
00
or
1
1
.
The eigenvector for = 3 is given by the nullspace of
01
0 1
1
0
or
.
Thus our matrix S and are given by
S=
11
1 0
so S 1 =
and
=
20
03
0 1
11
.
Thus B k is given by S k S 1 which in this case is
Bk =
=
11
1 0
2k 0
0 3k
2k 3k
2k 0
0 1
11
0 1
11
=
3k 3k 2k
0
2k
Problem 24
If A = S S 1 , then |A| = |S S 1| = |S ||||S 1| = ||. But since is a diagonal matrix its
determinant is the product of its diagonal elements. Thus we see that |A| = n=1 i . This
i
quick proof works only when A is diagonalizable.
Problem 25
We have the product of A and B given by
AB =
ab
cd
qr
st
=
aq + bs ar + bt
cq + sd cr + st
,
so the trace of AB is given by Tr(AB ) = aq + bs + cr + dt. The product in the other direction
is given by
q a + rc qb + rd
ab
qr
,
=
AB =
sa + tc sb + td
cd
st
Thus we have Tr(BA) = aq + rc + sb + td, which is the same as we had before.
Now choose A as S and B as S 1 . Then the product S (S 1) has the same trace as the
product in the reverse order i.e. (S 1)S = . The later matrix , has its trace given by
m
i=1 i . This argument again assumes that A is diagonalizable. For a general m m matrix
the product AB has elements given by m aik bkj and the product BA has terms given by
k =1
m
k =1 bik akj , so the trace of AB is given by summing the diagonal terms of AB or
m
m
aik bki
Tr(AB ) =
i=1
.
k =1
while the trace of BA is given by summing the diagonal terms of BA or
m
m
bik aki
Tr(BA) =
i=1
.
k =1
We can see that these expressions are equal to each other, showing that the two traces are
equal.
Problem 26
Now to have AB BA = I is impossible since the trace of the left hand side id given by
Tr(AB ) Tr(BA) = 0 ,
while the trace of the right hand side equals the trace of the m m identity matrix or m.
Let
10
1 1
A=E=
and B =
,
1 1
01
so that the products AB and BA are given by
AB =
1 1
1 2
and BA =
2 1
1 1
.
With these two matrices we see that the dierence AB BA is given by
1 0
, which
01
has a trace of zero as required.
Problem 27
If A = S S 1 and B in block form is given by B =
A0
0 2A
then we can decompose
(factor) B as
B=
S S 1
0
0
S (2)S 1
=
S0
0S
0
0 2
S 1 0
0 S 1
.
We can easily check that this is indeed a factorization of B by explicitly multiplying the
matrices on the right hand side together. We nd multiplying the two right most matrices
together that the above is equal to
S0
0S
S 1
0
0
(2)S 1
.
Finally multiplying these two matrices together we have
S S 1
0
0
S (2)S 1
=
A0
0 2A
,
proving that we have found the decomposition for B . Thus the eigenvalue matrix for the
0
A0
and the eigenvector matrices are given by
is given by
block matrix
0 2
0 2A
S=
S0
0S
and S 1 =
S 1 0
0 S 1
.
Problem 28
Let our set S be dened as all four by four matrices such that
S = {A| = S AS} ,
for a xed given S . Then if A1 and A2 are in S we have that
A1 + A2 = S 1 S 1 + S S 1 = S (1 + 2 )S 1 ,
so we see that A1 + A2 is in S . If A1 S then cA1 = S (c1 )S 1 so cA1 S . Thus S is a
subspace. If S = I then the only possible As in S are the diagonal ones. This space has
dimension four.
Problem 29
Suppose A2 = A, then the column space of A must contain eigenvectors with = 1. In fact
all columns of A are eigenvectors with eigenvalue equal to one. Thus all vectors in the column
space are eigenvectors with eigenvalue = 1. The vectors with = 0 lie in the nullspace
and from the rst fundamental theorem of linear algebra the dimension of the column space
plus the dimension of the nullspace equals n. Thus A will be diagonalizable since we are
guaranteed to have enough (here n) eigenvectors.
Problem 30
When A has a nonempty nullspace we do indeed get n r linearly independent eigenvectors.
If x is not in the nullspace of A there is no guarantee that Ax = x for any constant . Thus
the r vectors in the column space of A may have no basis (of the column space) such that
Ax = x. In addition, the nullspace and the column space can overlap if for instance one of
the nullspace vectors is in fact a column of the original A.
Problem 31
The eigenvectors of A for = 1 are given by the nullspace of
44
44
or the span of
1
1
The eigenvectors of A for = 9 are given by the nullspace of
4 4
4 4
or the span of
1
1
Thus S =
11
1 1
so that S 1 =
R = S S 1 =
1
2
11
1 1
1 1
11
10
03
.
and therefore
1/2 1/2
1 / 2 1/ 2
=
21
12
.
Note that the product RR is given by
55
21
21
,
=
55
12
12
which should be A, since if R = S S 1 then
RR = S S 1 S S 1 = S S 1 .
RR =
The square root of would require the square roots of the numbers 9 and 1. The latter is
imaginary and the product R = S R1 could not be real, since S and S 1 are both real
but the matrix is not. Therefore the product S S 1 could not be real.
Problem 32
We have for xT x the following
xT x = xT Ix = xT (AB BA)x = xT ABx xT BAx
= (Ax)T (Bx) + (Bx)T (Ax) = 2(Ax)T (Bx) 2||Ax||||Bx|| ,
where we have used the fact that AT = A and B T = B to simplify the inner products
xT ABx = (Ax)T (Bx) and xT BAx = (Bx)T (Ax) .
Thus ||x||2 2||Ax||||Bx|| so that
1
||Ax|| ||Bx||
.
2
||x|| ||x||
Problem 33
If A and B have the same independent eigenvectors and the same eigenvalues then A =
S S 1 and B = S S 1 so we see that A = B .
Problem 34
If S is such that A = S 1 S 1 and B = S 2 S 1 then
AB = S 1 S 1 S 2 S 1 = S (1 2 )S 1 = S (2 1 )S 1 ,
since diagonal matrices commute and therefore
AB = S 2 S 1 S 1 S 1 = BA .
Problem 35
If A is diagonalizable then A = S S 1 and the product matrix
P (A 1 I )(A 2 I ) (A n I ) ,
can be simplied as
P = (S S 1 1 SS 1 )(S S 1 2 SS 1 ) (S S 1 n SS 1 )
= S ( 1 I )S 1 S ( 2 I )S 1 S S ( n I )S 1
= S ( 1 I )( 2 I ) ( n I )S 1 .
If we consider the product ( 1 I )( 2 I ) ( n I ), we recognize it as the product
of diagonal matrices and we see that it is given by
0
2 1
3 1
..
.
n 1
1 2
0
3 2
..
.
n 2
1 n
2 n
3 n
.
..
.
0
This matrix product simplies to a diagonal matrix Z whos diagonal elements are given by
d11 = 0(1 2 ) (1 n ) = 0
d22 = (2 1 )0(2 3 ) (2 n ) = 0
d33 = (3 1 )(3 2 )0 (3 n ) = 0
.
.
.
dnn = (n 1 )(n 2 ) (n n1 )0 = 0 .
Since each diagonal element of a diagonal matrix is zero, the total product must also be zero
i.e.
(A 1 I )(A 2 I ) (A n I ) = 0 .
Problem 36
If A =
3 4
2 3
then the characteristic polynomial of A is given by
|A I | =
3
4
2
3
= (3 )(3 ) + 8 = 2 1 .
Now the matrix expression A2 I which we compute equals
3 4
2 3
3 4
2 3
10
01
=
9 8 12 + 12
6 6 8 + 9
10
01
= 0.
Thus A2 = I and it looks like A1 = A. To check this directly we can explicitly compute
A1 we nd that
1
3 4
3 4
A1 =
=
= A,
2 3
2 3
9 + 8
as claimed.
Problem 37
Part (a): Always. A vector in the nullspace of A is automatically an eigenvector with
eigenvalue zero.
Part (b): The eigenvectors with = 0 will span the column space if there are r independent
vectors.
Section 6.3 (Applications to Dierential Equations)
Problem 1
Let
A=
43
01
,
to nd the eigenvalues and eigenvectors. From the eigenvalue trace and determinant identity
we have
1 + 2 = 5 and 1 2 = 4
From which we can see that two eigenvalues are given by = 1 and = 4. For = 1 the
eigenvector is given by the nullspace of the following matrix
33
00
1
1
which has
,
,
as an eigenvector. For = 4, the eigenvector is given by the nullspace of the following matrix
03
0 3
which has
1
0
,
,
as an eigenvector. Thus the two solutions to the given dierential equation is given by
x1 (t) =
1
1
et
and x2 (t) =
1
0
e4t
The general solution is then a linear combination of the above solutions. To have the general
solution equal the given initial condition we have that
5
2
= c1
1
1
+ c2
1
0
which gives c1 = 2 and c2 = 3. Thus the entire solution is given by
x(t) = 2
1
1
et + 3
1
0
e4t .
Problem 2
Solving
have
dz
dt
= z with z (0) = 2 gives z (t) = 2et . Then using this in the equation for y we
dy
= 4y + 3z = 4y 6et .
dt
To solve this equation we solve the homogeneous part dy = 4y and then nd a particular
dt
solution to the inhomogeneous part. The homogeneous solution is given by y (t) = C2 e4t and a
particular solution can be found by substituting a solution that looks like the inhomogeneous
term. We try a solution of the form y (t) = Aet . When this is put into our inhomogeneous
term we obtain
Aet 4Aet = 6et ,
which gives A = 2. Thus we have a total solution for y (t) given by
y (t) = C2 e4t + 2et .
To satisfy the initial condition of y (0) = 5 we have that C2 must be given by the equation
C2 + 2 = 5 or C2 = 3. Thus the solution to our full system is then
z (t) = 2et
y (t) = 3e4t + 2et .
Problem 3
If we dene v = y we see that y = 5v +4y so our dierential equation becomes the following
system
d
y
01
y
y
.
=
=
y
45
5 y + 4y
y
dt
In this case, our coecient matrix A is given by
01
. The two eigenvalues of this A
45
must satisfy the trace determinant identities
1 + 2 = 5 and 1 2 = 4 .
From the rst condition we see that 1 = 5 2 which when we put this into the second
condition gives a quadratic for 2 . Solving this gives
5 41
2 =
.
2
We can verify these results by substituting et directly into the dierential equation y =
5y + 4y and solving for . When we do this we nd that must satisfy
2 5 4 = 0 ,
the same characteristic equation we found earlier.
Problem 4
From the problems statement the functions r (t) and w (t) must satisfy
dr
= 6r 2 w
dt
dw
= 2r + w .
dt
In matrix form our system is given by
d
dt
r
w
=
6 2
21
r
w
.
The coecient matrix above has eigenvalues 1 and 2 that must satisfy
1 2 = 10 and 1 + 2 = 7 ,
Thus by inspection 1 = 2 and 2 = 5 are the two eigenvalues. For = 2 the eigenvector is
given by the nullspace of the following matrix
4 2
2 1
which has
x=
2 1
00
1
2
,
,
as an eigenvector. For = 5, the eigenvector is given by the nullspace of the following matrix
1 2
2 4
which has
x=
2
1
1 2
00
,
as an eigenvector. Thus the total solutions to the given dierential equation is given by a
linear combination of the two solutions x1 and x2 given by
1
2
x1 (t) =
e2t
2
1
and x2 (t) =
e5t .
That is u(t) has the following form
u(t) = c1
1
2
2
1
e2t + c2
e5t .
The initial condition of u(0) forces c1 and c2 to satisfy the following
30
30
1
2
= c1
2
1
+ c2
12
21
=
c1
c2
.
Solving this linear system for c1 and c2 gives
c1
c2
=
10
10
.
Thus the entire solution is given by
u(t) = 10
1
2
e2t + 10
2
1
e5t ,
so the population of rabbits and wolves is given by
r (t) = 10e2t + 20e5t
w (t) = 20e2t + 10e5t .
After a long time the ratio of rabbits to wolves is given by
r (t)
10e2t + 20e5t
=
2,
w (t)
20e2t + 10e5t
as t .
Problem 5
Our dierential equations become
dw
= vw
dt
dv
= wv.
dt
Now consider the variable y dened as y = v + w . Taking the derivative of y we see that
dv dw
dy
=
+
= w v +v w = 0.
dt
dt
dt
So the function y (t) = v (t) + w (t) is a constant for all time. This means that y (t) is always
equal to its initial condition y (t) y (0). The constant value of y is easilty computed
y (0) = v (0) + w (0) = 30 + 10 = 40 .
v (t)
w (t)
Dening the vector of unknowns u as u =
du
=
dt
wv
vw
then we have that u satises
1 1
1 1
=
v
w
.
In the above system of dierential equations the coecient matrix is given by A =
which has eigenvalues given by the solution of
1
1
1
1
1 1
,
1 1
=0
Expanding this determinant we have 2 + 2 = 0 or = 0 and = 2. The eigenvectors of
1
11
. The eigenvectors
, or the span of
A for = 2 are given by the nullspace of
1
11
1
1 1
, or the span of
of A for = 0 are given by the nullspace of
. The total
1 1
1
solutions to the given dierential equation is given by
u(t) = c1
1
1
e2t + c2
1
1
.
Given the initial conditions of v (0) = 30 and w (0) = 10 to nd c1 and c2 we regonize that
they have to satisfy the initial condition requirement of u at 0. That is
30
10
= c1
1
1
+ c2
1
1
,
which has a solution given by c1 = 10 and c2 = 20. In this case u(t) is given by
u(t) = 10
1
1
e2t + 20
1
1
.
We can check that v (t) + w (t) = 40 for all time by adding the two functions found above.
When we do this we nd
10e2t + 20 10e2t + 20 = 40 ,
as required. When t = 1 we have that
u(1) =
v (1)
u(1)
=
10e2 + 20
10e2 + 20
.
Problem 6
Now our coecient matrix is 1 times A means that the eigenvectors of Ax = x becomes
Ax = x. From which we see that the eigenvectors of A are the same as the eigenvectors
of A, and the eigenvalues of A are the negative of the eigenvalues of A. Thus the two
1
eigenvalues of A are given by = 0 and = 2, with eigenvectors given by
and
1
1
, so again the solution is given by
1
v (t)
w (t)
1
1
= 10
1
1
e2t + 20
.
Thus v (t) = 10e2t + 20 as t .
Problem 7
y
y
Let the vector u be dened as u(t) =
as its solution
y (t)
y (t)
then
= eAt
du
dt
=
y (0)
y (0)
y
y
=
01
00
.
We can evaluate eAt using the denition in terms of a Taylor series, that is
1
1
eAt = I + At + A2 t2 + A3 t3 +
2
6
Now
A2 =
01
00
so that
eAt = I + At =
10
01
01
00
=
00
00
+
01
00
t=
,
1t
01
.
y (0) + y (0)t
y (0)
,
From this we see that
y (t)
y (t)
=
1t
01
y (0)
y (0)
=
The rst component gives y (t) = y (0) + y (0)t.
Problem 8
Substituting y = et into our dierential equation gives
2 = 6 9 .
y
, which has
y
When we solve this for we nd that = 3 is a double root. The matrix representation for
y = 6y 9y is given by
d
dt
y (t)
y (t)
=
01
9 6
y (t)
y (t)
.
This coecient matrix has eigenvalues given by the solution of ( 3)2 = 0 as earlier. To
look for the eigenvectors consider
3 1
,
9 3
1
as the only eigenvector. To show that y = te3t is a second solution, evaluate
3
the dierential equation for this value of y . We compute
which has
y = te3t
y = e3t + 3te3t
y = 3e3t + 3e3t + 9te3t = 6e3t + 9te3t .
Then
6y 9y = 6e3t + 18te3t 9te3t = 6e3t + 9te3t ,
which is y showing how y (t) satises the dierential equation.
Problem 9
Part (a): We have
d2
(u1 + u2 + u2 ) = 2u1u1 + 2u2 u2 + 2u3 u3
2
3
dt
= 2u1(cu2 bu3 ) + 2u2(au3 cu1 ) + 2u3 (bu1 au2 )
= 0.
Since u2 + u2 + u2 = ||u||2, we see that ||u|| must be a constant.
1
2
3
Part (b): ||eAt u(0)|| = ||u(0)|| so eAt is an orthogonal matrix. When A is skew symmetric
Q = eAt is an orthogonal matrix.
Problem 10
1
01
with eigenvalue
we have two eigenvectors. The rst
i
1 0
1
with eigenvalue = i. To superimpose these two vectors
= i, and the second
i
1
we have
into
0
1
1
1
,
+ c2
= c1
i
i
0
Part (a): When A =
so our constants c1 =
1
2
1
and c2 = 2 .
Part (b): Thus the solution to
du
=
dt
01
1 0
u(t) = c1 eit
1
i
+ c2 eit
1
i
,
1
i
1
+ eit
2
1
i
.
u1 (t)
u2 (t)
,
is given by
with c1 = c2 = 1/2 this becomes
1
u(t) = eit
2
Using Eulers formula of
eit = cos(t) + i sin(t)
eit = cos(t) i sin(t) .
we have that u(t) becomes
1
1
(cos(t) + i sin(t))
+
i
2
1
2 cos(t)
=
=
2 sin(t) sin(t)
u(t) =
1
(cos(t) i sin(t))
2
cos(t)
.
sin(t)
1
i
Problem 11
2
y
Part (a): The equation d 2 = y is solved by y (t) = A cos(t) + B sin(t). To have y (0) = 1
dt
and y (0) = 0 we must have y (t) = cos(t).
Part (b): We write the matrix form for the dierential equation y = y , by dening the
y (t)
so that
vector u to be u =
y (t)
du
=
dt
y (t)
y (t)
=
01
1 0
y (t)
y (t)
.
From Part (a) we have that y (t) = cos(t), so y (t) = sin(t), then
u=
cos(t)
sin(t)
and
du
=
dt
sin(t)
cos(t)
,
1
sin(t)
cos(t)
01
, showing that this
and u(0) =
=
0
cos(t)
sin(t)
1 0
vector solution u solves the dierential equation and has the correct initial conditions.
which equals
Problem 12
If A is invertible then a particular solution to
du
= Au b ,
dt
will be u a constant if and only if
du
dt
= 0 or 0 = Au b or u = A1 b.
Part (a): For du = 2u 8. The particular solution is given by 2u = 8 (or u = 4), and the
dt
homogeneous solution is given by du = 2u u = Ce2t . Thus the complete solution is given
dt
by u(t) = 4 + Ce2t .
20
03
assuming u is a constant)
Part (b): For
du
dt
=
8
. Then a particular solution is given by (again
6
u
20
03
8
6
u=
u1
u2
4
2
=
a particular solution is given by the solution to
du
=
dt
The coecient matrix A is then given by
eigenvectors
1
0
and
20
03
u.
20
, which has eigenvalues 2 and 3, with
03
0
, then the total solution is then
1
1
0
c1
0
1
e2t + c2
e3t ,
so that the total solution (particular plus the homogeneous) is given by
u = c1
1
0
e2t + c2
0
1
e3t +
4
2
.
Problem 13
Assume that c is not an eigenvalue of A. Let u = ect v , where v is a constant vector. Then
du
= cect v and
dt
Au = Aect v = ect Av ,
so that the equation
du
dt
= Au ect b becomes
cect ect Av ect b
cv = Av b
(A cI )v = b
v = (A cI )1 b .
Since c is not an eigenvector of A A cI is invertible, showing that u = ect v = ect (A cI )1 b
is a particular solution to the dierential equation
du
= Au ect b .
dt
If c is an eigenvector of A, then A cI is not invertible and there exists a nonzero v such that
Av = cv , so that when ect v is substituted into our dierential equation we have cv = Av b
or 0 = b a contradiction.
Problem 14
For a dierential equation to be stable we require that u 0 as t . For the dierential
equation du = Au, when A is a matrix, this will happen when all the eigenvalues of A have
dt
negative real parts. For a two by two systems, this eigenvalue condition breaks down into
conditions on the trace (T ) and determinant (D ) of A. The conditions are that T a + d < 0
ab
are given
and D ad bc > 0. Since the eigenvalues of a two by two system A =
cd
by the characteristic equation or
a
b
c
d
= 0.
This becomes
(a )(d ) bc = 0
(a + d) + ad bc = 0
2 T + D = 0 ,
2
when expressed in terms of T and D . From which using the quadratic equation we nd the
roots given by
T T 2 4D
.
=
2
So the value of the expression T 2 4D separates real from complex eigenvalues. Plotting
T 2 4D = 0 on the determinant D v.s. trace axis T gives the following plot
Dening 1 and 2 as
1 =
T
T 2 4D
2
and 2 =
T+
T 2 4D
.
2
Part (a): For 1 < 0 and 2 > 0 let A be given by
A=
1 0
01
or A =
1 2
01
.
Part (b): For 1 > 0 and 2 > 0 let A be given by
00
01
A=
.
Part (c): For complex with real part we need a > 0. To nd a matrix A that works we
know that the components of A must satisfy
a + d = 1 + 2
ad bc = 1 2 .
From which we might try 1 = 1 + i and 2 = 1 i. Then 1 + 2 = 2 and 1 2 = 2. Now
to obtain the required A we recall that A = S S 1 in this case would be given by
1 1
11
A=
=
1
2
1+i
0
0
1i
1 + i 1 + i
1+i 1i
11
1 1
1
2
11
1 1
=
1i
i1
,
which is not real and this experiment did not work. As another attempt consider A dened
22
then |A| = 2 and Tr(A) = 2. Lets verify that indeed the eigenvalues are
as A =
1 0
given by 1 i. The characteristic equation for this A is given by
2 2
1
= 0 2 2 + 2 = 0 ,
which has solutions given by
=
2
4 4(2)
= 1 i,
2
and thus this A works.
Problem 15
Consider the denition of the matrix exponential
1
1
1
1
eAt = I + At + A2 t2 + A3 t3 + A4 t4 + A5 t5 +
2
6
24
5!
taking the time derivative of both sides of this expression we compute
1
1
1
d At
e
= A + A2 t + A3 t2 + A4 t3 + A5 t4 +
dt
2
6
4!
1 22 1 33 1 44
= A(I + At + A t + A t + A t + )
2
6
4!
= AeAt .
Problem 16
For the matrix B =
0 1
, we see that the square of B is given by
00
B2 =
0 1
00
0 1
00
=
00
00
,
and thus all higher powers of B are also the zero matrix. Because of this property of the
powers of B the matrix exponential is also simple to calculate
1
1
eBt = I + Bt + B 2 t2 + B 3 t3 +
2
6
1 t
= I + Bt =
.
01
Then
d
d Bt
e=
dt
dt
1 t
01
=
0 1
00
.
Problem 17
The solution at time t + T can also be written as eA(t+T ) u(0) and since we can view this as
the solution at time T propagated for t more time we have
eAt eAT u(0) = eA(t+T ) u(0) ,
so that we see
eAt eAT = eA(t+T ) .
Problem 18
11
we have that
00
1 + 2 = 1 and 1 2 = 0. From which by trial and error we see that 1 = 0 and 2 = 1.
1
, and the second eigenvector (for 2 = 1) is
The rst eigenvector (for 1 = 0) is
1
0 1
11
1
and the matrix of eigenvalues is
so that S 1 =
. Thus S =
11
1 0
0
00
. Thus A is given by
=
01
From the trace determinant identity for the eigenvalues for A =
A = S S 1 =
11
00
=
11
1 0
00
01
0 1
11
.
Then we have
A2 t2 A3 t3
+
+
2
3!
t2
t3
= I + S S 1 + S 2 S 1 + S 3 S 1 +
2
6
t3
t2
= S + 2 + 3 + S 1
2
6
1
0
S 1
=S
3
t2
0 1+t + 2 + t +
3!
eAt = I + A +
10
0 et
=S
11
1 0
=
S 1
0 1
11
10
0 et
et 1 + et
0
1
=
Note also that eAt = Set S 1 which may have been a quicker way of deriving the above.
Problem 19
For the general case if A2 = A, then
A2 t2 A3 t3
+
+
2
6
At2 At3
+
+
= I + At +
2
6
t2 t3
= I + A(t + + + )
2
6
= I + A(et 1) .
eAt = I + At +
For the specic case were A =
11
00
A2 =
11
00
we see that indeed A2 = A as
11
00
=
11
00
= A,
so the above formula gives for eAt
eAt =
the same as we had before.
10
01
+
11
00
(et 1) =
et et 1
0
1
,
Problem 20
0 1
e e1
11
using Problem 18. For B =
, we have that eA =
00
0
1
00
1 1
, since B 2 = 0 and all higher order terms in the Taylor
we have eB = I + B =
01
10
expansion denition of eB are zero. For the matrix A + B =
we have
00
For A =
eA+B = I + (e 1)(A + B ) .
since (A + B )2 = A + B . Thus eA+B is
10
01
+ (e 1)
10
00
e0
01
=
.
Now consider the product of two matrices eA eB which is given by
eA eB =
e e1
0
1
1 1
01
=
e 1
01
= eA+B =
e0
01
.
And the product in the opposite order
eB eA =
1 1
01
e e1
0
1
=
e e2
0
1
= eA eB .
Problem 21
11
, we have eigenvalues given by = 1 and = 3. The eigenvector
03
01
1
for = 1 is given by the nullspace of
, or the span of
. The eigenvectors for
02
0
2 1
1
11
= 3 are given by the nullspace of
, or the span of
. Then S =
so
00
2
02
10
2 1
1
. Thus we have
with a matrix of eigenvalues given by =
that S 1 = 2
03
01
that eAt is given by
For the matrix A =
eAt = Set S 1
11
et 0
=
02
0 e3t
1 2et et + e3t
=
0
2e3t
2
1
2
=
2 1
01
1
et 2 et + 1 e3t
2
0
e3t
.
When t = 0 we have eA0 = e0 = I and the right hand side of the above gives the same (the
identity matrix).
Problem 22
If A =
13
00
then A2 =
13
00
13
00
=
13
00
= A, so from Problem 19 we have
that
eAt = I + (et 1)A
10
=
+ (et 1)
01
13
00
=
et 3(et 1)
0
1
.
Problem 23
Part (a): Since (eAt )1 = eAt , them matrix eAt is never singular.
Section 6.4 (Symmetric Matrices)
Problem 1
124
A = 4 3 0 = M + N , with M T = M and N T = N . For a square matrix
865
124
148
136
1
1
1
M = (A + AT ) = 4 3 0 + 2 3 6 = 3 3 3 .
2
2
2
865
405
635
Then N must be given by
1
1
N = A M = A (A + AT ) = (A AT ) .
2
2
In this case we nd that N is given by
0 1 2
N = 1 0 3 .
23
0
Thus A = M + N is decomposed as
124
136
0 1 2
4 3 0 = 3 3 3 + 1 0 3 .
865
635
23
0
Problem 2
If C is symmetric then AT CA is also symmetric since
(AT CA)T = AT C T A = AT CA .
When A is 6 3, AT is 3 6 and C must be 6 6, so that nally AT CA is 3 3.
Problem 3
The dot product of Ax with y equals
(Ax)T y = xT AT y = xT Ay ,
which is the dot product of x with Ay . If A is not symmetric then
(Ax)T y = xT AT y .
Problem 4
Note that since A is symmetric so that it has real eigenvalues and orthogonal eigenvectors.
The eigenvalues of A are given by
2
6
6
7
= 0 2 5 50 = 0 ,
This has solutions given by = 5 and = 10. The eigenvectors for = 5 are given by
2
36
. The eigenvector for = 10 is given by
, or the span of
the nullspace of
1
6 12
1
12 6
, which is orthogonal to the previously
, or the span of
the nullspace of
2
6 3
computed eigenvector as it must be. To obtain an orthogonal matrix we need to normalize
each vector giving
1
Q=
5
Thus
21
1 2
1
A = QQT =
5
1
so Q1 = QT =
5
21
1 2
5 0
0 10
1
5
2 1
12
2 1
12
Problem 5
10
2
For A = 0 1 2 since A = AT the eigenvalues must be real and the eigenvectors
2 2 0
will be othogonol. To nd the eigenvalues we nd the roots of the characteristic polinomial
1
0
2
0
1 2
2
2
= 0.
Expanding the determinant we nd that it equals (2 9) = 0 or = 0 and = 3. For
1 = 3 the eigenvector is given by the nullspace of
40
2
1 0 1/ 2
1 0 1/ 2
1 0 1/ 2
0 2 2 0 1 1 0 1 1 0 1 1 .
2 2 3
1 1 3/2
0 1 1
00 0
1
Which has a nullspace given by the span of 2 . For 2 = 0 the eigenvector is given by
2
the nullspace of
10
2
1 0 1/ 2
102
0 1 2 0 1
2 0 1 2 .
2 2 0
0 2 4
000
2
Which has a nullspace given by the span of 2 . For 3 = 3 the eigenvector is given by
1
the nullspace of
2 0
2
10
1
10
1
1 0 1
0 4 2 0 1
1/ 2 0 1
1/ 2 0 1 1/ 2
2 2 3
1 1 3/2
0 1 1/2
00 0
2
Which has a nullspace given by the span of 1 . Thus the matrix with columns of our
2
eigenvectors is given by
1 2 2
Q= 2
2 2.
2 1 2
To make Q an orthogonal matrix we need
have that
1
1
2
Q=
4+4+1
2
to normalize each vector
1 2
22
= 1 2
2
22
3
2 1
1 2
by its length. Thus we
2
2
2
So that
Q1
1 2
2
1
= QT = 2
2 1
3
2 1 2
3 0 0
and = 0 0 0 , so that A = QQT with the denitions of Q and given above.
0 03
Problem 8
If A3 = 0, then = 0 must be an eigenvalue of A. This is because we can recognize A3 as
A operating on the columns of A2 , which we are told results in the zero matrix. Thus each
column of A2 is an eigenvector of A with eigenvalue zeros. It is easy to nd a 2 2 matrix
01
. I dont in general see why all of the
that has A2 = 0. One such matrix is A =
00
eigenvalues of A must be zero. If |A3 | = 0, since |A3 | = |A|3 , we see that |A3 | = 0 is the
same as ( i i )3 = 0 so it seems that all is to be required is that we have one eigenvalue
of A zero and the product will be zero. In the case when A is symmetric we know that it
has an eigenvector decomposition with real eigenvalues and orthogonal eigenvectors. Thus
A = QQT . In this case, from the third power of A we see that
A3 = Q3 QT = 0 3 = 0 = 0 ,
so that A must have all zero eigenvalues and in fact must be the zero matrix.
Problem 9
The characteristic equation of a 3 3 matrix A is a third order polynomial. As such, it
can have at most two complex roots (which must be complex conjugates) and still be a real
polynomial. Thus A must have at least one real eigenvalue. Another way to see this is to
consider the trace of A. This must be real since it is a sum of the diagonal elements of A.
By the trace, eigenvalue identity we have that Trace(A) = 1 + 2 + 3 , if all three of these
s were complex then Tr(A) would be complex. Thus at least one eigenvalue of A is real.
Problem 10
0 1
then
10
the characteristic equation is 2 + 1 = 0 or = i. For 1 = i, we have eigenvalues given
by the nullspace of
i1
i 1
,
00
1i
It is not stated the x must be real. For example consider the matrix A =
i
. For the eigenvalue 2 = +i the second eigenvector x2 will be the
1
T
i
. Then the expression xxTAx will be complex (since the
complex conjugate of x1 , or
x
1
eigenvectors x are).
or the span of
Problem 11
31
the spectral theorem requires calculating QQT . We begin by computing
13
the eigenvalues of A. We have
For A =
3
1
1
3
= 0 (3 )2 1 = 0 .
The roots of this quadratic are given by = 2 and = 4. For = 2 the eigenvectors are
given as the nullspace of
1
11
.
or x1 =
1
11
For = 4 we have the eigenvectors given by the vectors in the nullspace of
1 1
1 1
Then Q =
1
2
11
1 1
or x2 =
and Q1 = QT =
1
2
11
1 1
A = QQT =
=
1
2
1
2
11
1 1
1
1
.
1 1
. Thus we have that
11
2 2
44
20
04
1 1
11
.
From which we see that our spectral decomposition of A is given by
A=
1
2
=2
11
1
2 2 +
44
1
21
1
1
1
2 2
+4
2 1
2
1
2
1
1
1
2
44
.
For the matrix B we perform the same manipulations as for A. First computing the eigenvalues we have
9
12
= 0 (9 )(16 ) 144 = 0 .
12 16
The roots of this quadratic are given by = 0 and = 25. From the spectral theorem for
A we have the following decomposition
A = 1 x1 xT + 2 x2 xT + + n xn xT .
1
2
n
This means that all eigenvalues with = 0 dont contribute to the decomposition above.
Thus we only need to the calculate the eigenvector for = 25. This is given by the nullspace
of
4 3
4 3
16 12
9 25
12
.
=
00
4 3
12 9
12
16 25
From which we see that the second eigenvector is given by x2 =
3
. Thus the spectral
4
1
5
decomposition of B is given by
B = 25
1
5
3
4
1
5
34
.
Problem 12
06
, because AT = A, A must have imaginary eigenvalues.
6 0
These are given by the characteristic equation or
For the matrix A =
6
6
= 0 2 + 62 = 0 = 6i .
Consider the following 3 3 skew-symmetric matrix
0
12
B = 1 0 3 ,
2 3 0
which has eigenvalues given by its characteristic equation
1
2
1 3
2 3
= 0.
Expanding the above determinant by cofactors we see that above is equivalent to
12
1
2
3
2
+1
3
3
3
= 0.
or
(2 + 9) + 1( + 6) 2(3 + 2) = 0 .
or simplifying
(2 + 14) = 0 .
So nally we see that = 0 or = i 14.
Problem 15
For Bx = x is given by
0A
AT 0
y
z
y
z
=
.
Which in components gives
Az = y
AT y = z .
Part (a): Multiplying the rst equation n by AT gives
AT Az = AT y = 2 z ,
is an eigenvalue of AT A.
Part (b): If A = I then 2 is an eigenvalue I which are only ones. Thus = 1, are the
eigenvalues of B . Since B is of of size four by four we need four eigenvalues and they are
1, 1, 1, 1. The eigenvectors of B can be obtained from the system of above. Thus z must
1
and 0 1 . In this same way AT y = z gives
be on eigenvalues of I and therefore is
0
four systems for y (providing the four eigenvectors of B ) the (since AT = I we can drop this
obtaining).
y=
1
0
and = 1 and z =
1
0
y=
0
1
and = 1 and z =
0
1
y=1
1
0
and = 1 and z =
1
0
y=1
1
0
and = 1 and z =
0
1
Thus the eigenvector/eigenvalue system is given by
1 0 1
0 1 0
Q=
1
01
0
10
with Diag = diag(1, 1, 1, 1).
0
1
,
0
1
.
Problem 16
If A =
1
, then from AT Az = 2 z we have that
1
11
1
1
z = 2 z , or
2 z = 2 z
2 = 2
= 2,
If z = 0 any vector. Now 1 is 1 1 from the denition of B . Also z = 0 with any will
work. To evaluate y consider AT y = 2 or
1 1 y = 2.
so that
y=
and consider
11
2
2
1
1
.
y = + 2, for = + 2 so
y=
2
2
1
1
.
Finally consider if z = 0 and unknown to obtain
1 1 y = 0.
so that y =
1
. Then the eigensystem for B is given by
1
1
2
1
Q = 2
1
1
2
1
2
1
1
1
0
with Q1 = QT required and = diag( 2, + 2, 0), where I have taken 3 = 0 since
as
1
1
B 1 = 0 1
0
0
Problem 17
Every 2 by 2 symmetric system can be written as
A = 1 x1 xT + 2 x2 xT = 1 P1 + 2 P2 .
1
2
here P1 and P2 are projection matrices (when ||x1 || = 1 and ||x2 || = 2).
Part (a): Now we have
P1 + P2 = x1 xT + x2 xT = [x1 x2 ]
1
2
xT
1
xT
2
= QQT = I
since Q is an orthogonal matrix.
Part (b): Also we have
P1 P2 = x1 xT (x2 xT ) = x1 (xT x2 )xT = 0
1
2
1
2
since xT x2 = 0 as x1 and x2 can be made orthogonal (since A is symmetric).
1
Problem 18
Suppose Ax = x and Ay = 0y with = 0, here y is in the nullspace and x is in the column
space.
xT A = xT
xT Ay = xT y ,
since Ay = 0 then xT y = 0 since = 0, then xT y = 0. Also y in the nullspace and x in
the column space but since A = AT , x in the column space means x in the row space but
the row space and the nullspace are orthogonal so xT y = 0. If the second eigenvector is not
zero say B , then we have Ay = By and Ax = x so we consider the matrix B = A I , so
Bx = (A I )x = Ax x = x x = ( )x
so
By = (A I )y = Ay y = y y = 0 .
So we see that x is an eigenvector of B with eigenvalues , and y is an eigenvector or B
with eigenvalue 0), so x and y are orthogonal by the previous arguments.
Problem 19
1 0 1
For B = 0 1 0 which is not symmetric. It has eigenvalues given by
0 0d
1
0
1
0
1
0
0
0
d
= 0.
On expanding we have
(1 )
0 1
1
0
+1
0
0
0
d
= (1 + )(1 )(d ) = 0 .
So = 1, d, +1, which has eigenvectors given by (for = 1) the nullspace of the following
matrix
00
1
0 2
0 .
0 0 d+1
Problem 27
See the Matlab le prob 6 4 27.m. There since A does not have linearly independent
columns, the direct calculation of AT A will not be invertible. Since the projection matrix will project onto the columns of A we can take any set of linearly independent columns
from A and construct the projection matrix using A(AT A)1 AT with A now understood
to contain only linearly independent columns. When this is done Matlab gives computed
eigenvectors with a dot product of exactly 1.0. Maybe there is an error somewhere?
Section 6.5 (Positive Denite Matrices)
Problem 15
Consider xT (A + B )x which by the distributive law equals xT Ax + xT Bx. Since both both
A and B are positive denite we know that xT Ax > 0 and xT Bx > 0 for all x = 0. Since
each term individually is positive, the sum xT Ax + xT Bx must be positive for all x = 0. As
this is the denition of positive denite, A + B is positive denite.
Problem 19
If x is an eigenvector of A then
xT Ax = xT (x) = xT x .
If A is positive denite then xT Ax > 0. From the above we have that
xT x > 0 or > 0
so the eigenvalues of a positive denite matrix must be positive.
Problem 20
Part (a): All the eigenvalues are positive so = 0 is not possible, therefore A is invertible
Part (b): To be positive denite a matrix must have positive (non-zero) diagonal elements.
To achieve this for a permutation of the identity we must put all the ones on the diagonal
giving the identity matrix.
Part (c): To be a positive denite projection matrix one must have
xT P x > 0 ,
for every x = 0. If P = I , there exist non-zero xs that are in the orthogonal complement
of the column space of P . These xs give P x = 0. Thus P will only be positive denite if it
has a trivial column space orthogonal complement or P = I .
Part (d): A diagonal matrix as described gives
xT Dx > 0
for all x = 0 so D would be positive denite.
Part (e): Let A be give by
1 1
1 2
Then |A| = 2 1 = 1 > 0, but a = 1 < 0 so A is not positive denite.
Section 6.6 (Similar Matrices)
Problem 1
If B = M 1 AM and C = N 1 BN we then have that
C = N 1 (M 1 AM )N = (MN )1 A(MN )
So dening T = MN we have C = T 1 AT . This states that if B is similar to A and C is
similar to B then C is similar to A.
Problem 2
If C = F 1 AF and also C = G1 BG then F 1 AF = G1 BG which gives
B = GF 1 AF G1 = (F G1 )1 A(F G1 )
Dening M = F G1 we see that B = M 1 AM , so if C is similar to A and C is similar to B
then A is similar to B .
Problem 3
We are looking for a matrix M such that A = M 1 BM or MA = BM . To nd such a
matrix let
ab
.
M=
cd
then MA = BM is given by
ab
cd
10
10
=
01
01
=
cd
cd
ab
cd
or upon multiplying both sides we have
a0
c0
,
which to be satised imposes that d = 0 and a = c. If we let a = 1 and b = 2 the selected
12
. For the next pair of A and B we have that MA = BM or
matrix becomes M =
10
ab
cd
11
11
1 1
1 1
=
ab
cd
or upon multiplying together the matrices on each side we have
a+b a+b
c+d c+d
ac
bd
a + c b + d
=
,
which after we set each component of the above equal gives the following system of equations
a
b
c
d
=
=
=
=
d
c
b
a
Thus we have the restriction that b = c and a = d. Picking a = 1 and b = 2 gives
M=
1
2
2 1
For the next pair of A and B we have that MA = BM or
ab
cd
12
34
=
43
21
ab
cd
or upon multiplying together the matrices on each side we have
a + 3b 2 a + 4b
c + 3d 2 c + 4d
=
4 a + 3c 4 b + 3d
2a + c 2b + d
,
which after we set each component of the above equal gives the following system of equations
3a + 3b 3c
2a 3d
2a 3d
2b 2c 3d
=
=
=
=
This gives the following system for the coecients a,
3 3 3 0
2 0 0 3
0 2 2 3
0
0
0
0
b, c, and d
a
b
=0
c
d
Performing Gaussian elimination on our coecient matrix produces
3 3 3 0
2 0 0 3
20 0
3
2 0 0 3 3 3 3 0 0 3 3 9/2
0 2 2 3
0 2 2 3
0 2 2 3
20 0 0
20 0
3
0 1 1 3/2 0 1 1 0 .
00 0 1
00 0
6
Which implies that d = 0, a = 0, and c = b. If we take b = 1, our matrix M becomes
M=
01
10
.
Problem 4
If A has eigenvalues 0 and 1 it has two linearly independent eigenvectors and therefore can be
factorized into A = S S 1 , which says that A and are similar. Now from Problem 2, since
10
every matrix with eigenvalues 0 and 1 are similar to =
, then they themselves are
00
similar.
Problem 5
A1 =
10
01
has = 1 only.
A2 =
01
10
has = 1 and = +1.
A3 =
11
00
has = 1 and = 0.
A4 =
00
11
has = 1 and = 0.
A5 =
10
10
has = 1 and = 0.
A6 =
01
01
has = 1 and = 0. Thus A3 , A4 , A5 , and A6 are similar.
Problem 7
Part (a): If x is in the nullspace of A, then Ax = 0 so M 1 x when multiplied on the left
by M 1 AM gives
M 1 AM (M 1 x) = M 1 Ax = M 1 0 = 0 .
so M 1 x is in the nullspace of M 1 AM .
Part (b): Since for every vector x in the nullspace of A there exists a vector M 1 x in
the nullspace of M 1 AM and for every vector x in the nullspace of M 1 AM there exists a
vector Mx in the nullspace of A (since M 1 AMx must then equal zero). Thus the nullspace
of A and M 1 AM have the same number of elements and therefore the dimension of the
nullspace is the same.
Problem 8
No, the order or association of eigenvectors to eigenvalues could be dierent among the two
matrices. If the association is the same I would think that A = B . With n independent
eigenvectors again the answer is no to the question of A = B . The logic from the previous
discussion still holds. If A has a double eigenvalue of 0 with a single eigenvector proportional
to (1, 0), then
01
= M 1 AM
00
or
01
A=M
M 1
00
with M a matrix the rst column of which is the vector [1, 0]T and the second column of
which must be linearly independent from the rst column. This gives many possible As.
Consider two dierent M s
M1 =
10
01
1a
0b
and M2 =
then the inverses are given by
M1 1 =
10
01
and M2 1 =
1
b
b a
01
Thus A1 =
01
00
and A2 is given by
A2 = M2
=
1
b
1
=
b
=
01
00
1a
0b
1a
0b
M2 1
01
00
1
b
b a
01
01
00
01
00
which does not equal A1 unless b = 1. Thus in this case also there is the possibility of two
dierent matrices with this property.
Chapter 8 (Applications)
Section 8.2 (Markov Matrices and Economic Models)
Problem 13
Since the rows/columns of B are linearly dependent we know that = 0 is an eigenvalue.
The other eigenvalue can be obtained by the eigenvalue trace theorem or
.2 .3 = 0 + 2 2 = 0.5 .
Since 1 = 0 when e1 t multiplies x1 we have only a multiplication by 1 to the eigenvector
x1 . The factor e2 t will decay to zero since 2 < 0 and therefore the steady state for this
ODE is given by the eigenvector x1 corresponding to 1 = 0, which in this case is give by
x=
0.3
0.2
.
Therefore the solution, when decomposed in terms of its initial condition, will approach c1 x1 .
Problem 14
The matrix B = A I has each column summing to 0. The steady state is the same as that
of A, but with 1 = 0 and therefore e1 t = 1.
Problem 15
If each row of a matrix adds to a constant value (say C ) this means that the vector
[1, 1, . . . , 1]T is an eigenvector of A, with the corresponding sum, C , the eigenvalue.
Problem 16
The required product is given by
(I A)(I + A + A2 + A3 + . . .) = I + A + A2 + A3 + . . . A A2 A3 A4 . . .
=I
Problem 20
If A is a Markov matrix then = 1 is an eigenvalue of A and therefore (I A)1 does not
exist, so the given sum cannot sum to (I A)1 .
Chapter 9 (Numerical Linear Algebra)
Section 9.1 (Gaussian Elimination in Practice)
Problem 5
We wish to count the number of operations required to solve
Ux = c with semiband width w/2 or
u1,1 u1,2 u1,3 . . . u1,w/2
u2,2 u2,3 . . . u2,w/2 u2,w/2+1
un1,n1 un1,n
un,n
the following banded system
x = c
so at row i we have non-zero elements in columns j = i, i + 1, i + 2, . . . i + w/2, assuming that
i + w/2 is less than n. Then a pseudo-code implementation of row-oriented back substitution
would look something like the following
Counting the number of ops this requires, we have approximately two ops for every execution of the line c(j ) = c(j ) U (i, j ) c(j ), giving the following expression for the number
of ops
1
i=n
Now since
i+1
j =min(n,i+w/2)
2 + 1 .
i+1
2 = O (2(w/2)) = O (w )
j =min(n,i+w/2)
the above sum simplies (using order notation) to
O (n + wn) = O (wn) ,
as requested.
Problem 6
If one knows L and U to solve LUx = b requires one forward and one back solve. The back
solve requires O (1/2n2) ops and the forward solve requires the same op count O (1/2n2).
Thus to solve Ax = b when one has both L and U requires O (n2 ) operations. To solve for x
when one has A = QR one could rst multiply by Q1 = QT to get Rx = QT b. The product
of QT with b requires O (n2 ) ops, in addition to the back solve requires to invert R. Thus
to solve Ax = b when A = QR requires O ((1 + 1/2)n2 ) = O (3/2n2) ops. Thus it is better
to use the LU decomposition.
Problem 7
To invert an upper triangular matrix R we could repeatedly solve Rx = ei where ei is
the vector of all zeros with a 1 in the i-th component. When i = 1, Rx = e1 requires
only 1 op, since x2 , x3 , through xn are all zero. When i = 2, Rx = e2 requires solving
a 2x2 upper triangular matrix and as such requires O (22 /2) = O (2) operations. This is
because in this case x3 , x4 , through xn are all zero. Eectively the leading zeros in the back
substitutions allow many of the unknown xi s to be explicitly determined. In the same way
solving Rx = e3 requires O (32 /2) ops. So in general to solve Rx = ei requires O (i2 /2) ops.
Thus to compute the entire inverse of a triangular system R requires
n
i=1
i2
1
=
2
2
n
i=1
1 n3
n3
i2 = O ( ) = O ( ) .
2
3
6
Problem 8
To solve Ax = b for x with partial pivoting when,
10
22
A=
we would rst exchange the rst two rows with a permutation matrix P to obtain
01
10
A=
22
10
22
0 1
where we have multiplied P A by E21 dened as
1
0
1/2 1
E21 =
so that we now have
E21 P A =
22
0 1
.
Thus we have for our requested factorization of P A = LU the following
PA =
01
10
10
22
=
10
1/2 1
22
0 1
= LU .
For the second example where A is given by
we begin by exchanging the rst
01
P1 A = 1 0
00
101
A= 2 2 0
020
two rows with a
0
22
0 A = 1 0
1
02
permutation P1 to obtain
0
220
1 0 1 1 .
0
020
where the last transformation is obtained by multiplying the above matrix by the elementary
elimination matrix E21 given by
1
00
E21 = 1/2 1 0
0
01
giving the following result for the matrix product thus far
220
E21 P1 A = 0 1 1 .
020
To continue our elimination with partial pivoting we next exchange rows 2 and 3 with a
permutation matrix P2 dened as
100
P2 = 0 0 1
010
then our chain of matrix products becomes
220
220
P2 E21 P1 A = 0 2 0 0 2 0 .
0 1 1
001
Which can be obtained from P2 E21 P1 A by multiplying on the left by the elementary elimination matrix E32 dened by
100
E32 = 0 1 0 .
0 1/ 2 1
In total we then have E32 P2 E21 P1 A = U , which
100
100
1
0
0 1 0 0 0 1 1/2 1
0 1/ 2 1
010
0
0
in matrix form is the following
0
010
220
0 1 0 0 A = 0 2 0 .
1
001
001
The next step is to pass the permutation matrices through the elementary elimination
matrices so that we can get all elimination matrices on the left and all permutation matrices
on the right. Something like E32 E21 P2 P1 A = U . This can be performed by recognizing that
the product of P2 and E21 can be factored as
1
00
1
00
100
P2 E21 = 0
0 1 = 0
1 0 0 0 1 = E21 P2 .
1/2 1 0
1/2 0 1
010
Thus the initial factorization of E32 P2 E21 P1 A = U ,
and we then have that P2 P1 A = E211 E321 U , which in
100
010
10
0 0 1 1 0 0 A = 0 1
010
001
1/2 0
can be written as E32 E21 P2 P1 A = U ,
matrix form is given by
0
1
0
0
0 0
1
0 U .
1
0 1/2 1
which after we multiply all matrices in the above we
position as
010
1
0
0 0 1 A = 0
1
100
1/2 1/2
can obtain our nal P A = LU decom
0
220
0 0 2 0 .
1
001
This can easily be checked for correctness by multiplying the matrices on both sides and
showing that they are the same.
Problem 9
For the A given
1
1
A=
0
0
1
1
1
0
0
1
1
1
0
0
1
1
we can compute specic elements of A1 from the cofactor expansion formula, which is
A1 =
1
CT
Det(A)
with Cij = (1)i+j Det(Mij )
with Mij the minor (matrix) of the (i, j )-th element. Then based on the A above we can
investigate if the (1, 3), (1, 4), (2, 4), (3, 1), (4, 1), and (4, 2) elements of A1 are zero. These
are the elements of A which are zero and one might hope that a zero element in A would
imply a zero element in A1 . We can compute each element in tern. First (A1 )1,3 ,
(A1 )1,3 =
1
1
C31 =
(1)3+1 Det(M31 )
Det(A)
Det(A)
Since every term in the inverse will depend on the value of Det(A) we will compute it now.
We nd
100
110
Det(A) = [phone]
011
011
11
11
1
01
11
=11
=
11
01
Then we have that
Det(M31 ) =
so that (A1 )1,3 =
1
(1)
1
= 1 = 0.
1 1
11
11
= 1
100
110
011
=1
10
11
=1
Problem 10
We rst nd the LU factorization of the given A
A=
1
11
obtained without partial pivoting. Note that in a realistic situation on would want to use
partial pivoting since we assume that 1. Now our A can be reduced to
1
11
A=
1
0 1
,
1
by multiplying A by the elementary elimination matrix E21 dened as
10
1 1
E21 =
.
Thus we have the direct LU factorization (without partial pivoting) given by
A=
1
11
1
0 1
10
1
1
=
1
.
Thus our system Ax = b is given by
10
1
1
1
0 1
1
x1
x2
=
1+
2
.
Note that for this simple system we could solve Ly = b and then solve Ux = y exactly.
Doing so would not emphasis the rounding errors that are present in this particular example. Thus we have chosen to solve this system by Gaussian elimination without pivoting
using the teaching code slu.m. Please see the Matlab le prob 9 1 10.m for the requested
computational experiments. There we see that without pivoting when is near 1015 (near
the unit round for double precision numbers) the error in the solution can be on the order
of 10%. When one introduces pivoting (by switching the rst two rows in this system) this
error goes away and the solution is computed at an accuracy of O (1016 ).
Problem 14
To directly compute Qij A would require two steps. First multiplying row i of A by cos()
by row j of A by sin() and adding these two rows. This step requires 2n multiplications
and n additions. Second, multiply row i by sin() and adding to cos() multiplied by row j .
Again requiring the same number of multiplications and additions as the rst step. Thus in
total we require 4n multiplications and 2n additions to compute Qij A.
Section 9.2 (Norms and Condition Numbers)
Problem 4
Since the condition number is dened as (A) = ||A||||A1|| from ||AB || ||A||||B || with
B = A1 we have
||I || ||A||||A1|| = (A) ,
but ||I || = 1 so (A) 1 for every A.
Problem 5
To be symmetric implies the matrix is diagonalizable and A = S S 1 becomes A = QQT .
Since every eigenvalue must be 1 we have = I and A = QQT = I , so A is actually the
identity matrix.
Problem 6
If A = QR then we have ||A|| ||Q||||R|| = ||R||. We also have R = QT A so ||R||
||QT ||||A|| = ||A||. Thus ||A|| = ||R||. To nd an example of A = LU such that ||A|| <
||L||||U ||. Let
21
10
.
and U =
L=
02
2 1
then we have
LT L =
1 2
01
and
UT U =
20
12
10
2 1
=
5 2
2 5
21
02
=
42
25
Problem 7
Part (a): The triangle inequality gives ||(A + B )x|| ||Ax|| + ||Bx||
Part (b): It is easier to prove this with denition three from this section, that is
||A|| = Maxx=0
||Ax||
.
||x||
Thus we have
||(A + B )x||
||x||
||Ax|| + ||Bx||
Maxx=0
||x||
||Ax||
||Bx||
Maxx=0
+ Maxx=0
||x||
||x||
||A|| + ||B ||
||A + B || = Maxx=0
Problem 8
From Ax = x we have that ||Ax|| = ||x|| = ||||x||, but since ||Ax|| ||A||||x|| we then
have that ||||x|| ||A||||x|| or || ||A|| as requested.
Problem 9
Dening (A) = |max | to nd counter examples to the requested norm properties we will
note that from previous discussions A and B cannot have the same eigenvectors or else
A + B = A+B . The requirement of not having the same eigenvalues can be simplied to
the requirement that AB = BA. Thus diagonal matrices wont work for nding a counter
example. Thus we look to the triangular matrices for counter examples. Consider A and B
dened as
1 10
10
A=
and B =
01
10 1
Then since each matrix is triangular the eigenvalues are easy to calculate (they are the
elements on the diagonal) and we have (A) = (B ) = 1. Also note that
AB =
101 10
10 101
=
1 10
10 101
= BA
so A and B dont share the same eigenvectors and (A + B ) = (A) + (B ). Now the sum
of A and B is given by
1 10
A+B =
10 1
which has eigenvalues given by the solution to 2 Tr(A + B ) + Det(A + B ) = 0, which
for this problem has 1 = 9 and 2 = 11 so (A + B ) = 11. Thus we see that
(A + B ) = 11 > (A) + (B ) = 1 + 1 = 2
and we have a counterexample for the rst condition (the triangle inequality for matrix
norms). For the second condition we have the product AB given by
AB =
101 10
10 101
which has eigenvalues given by 1 = 91 and 2 = 111, thus we have
(AB ) = 111 > (A)(B ) = 1 ,
providing a contradiction to the second triangle like inequality (this time for matrix multiplication). These eigenvalue calculations can be found in the Matlab le prob 9 2 9.m.
Problem 10
Part (a): The condition number of A is dened by (A) = ||A||||A1||, while the condition
number of A1 is dened by (A1 ) = ||A1 ||||(A1)1 || = ||A1 ||||A|| = (A)
Part (b): The norm of A is given by max (AT A)1/2 , and the norm of AT is given by
max ((AT )T AT )1/2 = max (AAT )1/2 . From the SVD of A we have that AT A = V 2 V T and
AAT = U 2 U T , so both AT A and AAT have the same eigenvalues, i.e. the singular values
of A and therefore max (AT A) = max (AAT ), showing that A and AT have the same matrix
norm.
Problem 11
From the denition of the condition number of a matrix (A) = ||A||||A1||, since A is
symmetric ||A|| = Max(|(A)|) and A1 will be symmetric so
||A1 || = Max((A1 )) = Max
1
1
=
(A)
Min(|(A)|)
From the A given we will have eigenvalues given by the solution of
2 Tr(A) + Det(A) = 0
which for this problem has solutions given by (these are computed in the Matlab le
prob 9 2 11.m) 1 = 0.00004999, and 2 = 2.00005. Thus an estimate of the condition
number is given by
2.00005
|max |
=
= 40000 .
(A) =
|min |
0.00004999
Section 9.3 (Iterative Methods for Linear Algebra)
Problem 15 (eigenvalues and vectors for the 1,-2,1 matrix)
In general, for banded matrices, where the values on each band are constant, explicit formulas
for the eigenvalues and eigenvectors can be obtained from the theory of nite dierences. We
will demonstrate this theory for the 1,-2,1 tridiagonal matrix considered here. Here we will
change notation from the book and let the unknown vector, usually denoted by be denoted
x
by w . In addition, because we will use the symbol i for the imaginary unit ( 1), rather
than the usual i subscript convention we will let our independent variable (ranging over
components of the vector x or w ) be denoted t. Thus notationally xi w (t). Converting our
eigenvector equation Aw = w into a system of equations we have that w (t), must satisfy
w (t 1) 2w (t) + w (t + 1) = w (t) for t = 1, 2, . . . , N ,
with boundary conditions on w (t) taken such that w (0) = 0 and w (N + 1) = 0. Then the
above equation can be written as
w (t 1) (2 + )w (t) + w (i + 1) = 0 .
Substituting w (t) = mt into the above we get
m2 (2 + )m + 1 = 0 .
Solving this quadratic equation for m gives
m=
(2 + )2 4
2
(2 + )
From this expression if |2 + | 2 the expression under the square root is positive and the
two roots are both real. With two real roots, the only solution that satises the boundary
conditions is the trivial one (w (t) = 0). If |2 + | < 2 then m is a complex number and
the boundary conditions can be satised non-trivially. To further express this, dene such
that
2 + = 2 cos()
then the expression for m (in terms of ) becomes
m=
4 cos()2 4
= cos()
2
2 cos()
cos()2 1
or
m = cos() i sin() = ei
from the theory of nite dierences the solution w (t) is a linear combination of the two
fundamental solutions or
w (t) = Aeit + Beit .
(4)
Imposing the two homogeneous boundary condition we have the following system that must
be solved for A and B
i (N +1)
Ae
A+B = 0
+ Be
=0
i (N +1)
Putting the rst equation into the second gives
B (ei(N +1) ei(N +1) ) = 0
Since B cannot be zero (else the eigenfunction w (t) is identically zero) we must have satisfy
sin((N + 1)) = 0
Thus (N + 1) = n or
=
n
N +1
for
n = 1, 2 , . . . , N
Tracing back to the denition of we have that
= 2 + 2 cos() = 2 + 2 cos(
n
)
N +1
Using the trigonometric identity
1 cos( ) = 2 sin( )2
2
we get
n = 4 sin(
n
)2
2(N + 1)
for n = 1, 2, 3, . . . , N
For the eigenvalues of the 1, 2, 1 discrete one dimensional discrete Laplacian. To evaluate
the eigenvectors we go back to Eq. 4 using our new denition of . We get that
w (t) eit eit
sin(t)
n
t)
sin(
N +1
for n = 1, 2, 3, . . . , N
Here the range of t is given by t = 1, 2, . . . , N . These are the results given in the book when
n = 1 i.e. we are considering only the rst eigenvalue and eigenvector.
Problem 18 (an example of the QR method)
If A is given by
A=
cos() sin()
sin()
0
= QR
with a QR decomposition given by
QR =
cos() sin()
sin() cos()
1x
0y
Then expanding the matrix product above we must have for x and y the following equations
to hold
x cos() y sin() = sin()
x sin() + y cos() = 0 .
cos(
Then solving the second equation for x we have x = ysin() , which when put into the rst
)
equation gives
y cos()
cos() y sin() = sin()
sin()
which gives for y the solution of y = sin()2 . Thus we have for x that x = sin() cos().
With these two values our QR decomposition is given by
QR =
cos() sin()
sin() cos()
1 sin() cos()
0
sin()2
This gives for RQ product the following
RQ =
=
1 sin() cos()
0
sin()2
cos() sin()
sin() cos()
cos() + sin()2 cos() sin() + sin() cos()2
sin()3
cos() sin()2
Showing that the (2, 1) entry is now sin()3 as expected.
Problem 19
If A is an orthogonal matrix itself then the QR decomposition for A has Q = A and R = I
so RQ = IA = A. Thus the QR method for computing the eigenvalues of A will fail.
Problem 20
If A cI = QR, then let A1 = RQ + cI , and by multiplying this equation by Q on the left
we obtain
QA1 = QRQ + cQ .
Next since QR = A cI , the above QA1 becomes
QA1 = (A cI )Q + cQ = AQ
Now multiplying by QT = Q1 on the left of the above we obtain A1 = Q1 AQ, so A1 is a
similarity transformation of A and therefore has the same eigenvalues as A.
Problem 21
From the given decomposition Aqj = bj 1 qj 1 + aj qj + bj qj +1 , since the qj are orthogonal
T
T
then qj qi = ij so multiplying on the left by qj gives
T
T
qj Aqj = 0 + aj qj qj + 0
q T Aqj
j
so we have that aj = qT qj . Our equation says that AQ = QT where T is a tridiagonal
j
matrix with main diagonal given by the aj and b on the sub and super diagonal.
Problem 22
See the Matlab code prob 9 3 21.m and lanczos.m.
Problem 23
If A is symmetric, from the shifted QR method and Problem 20 we know that A1 is related
to A by A1 = Q1 AQ. Since Q1 = QT we have that A1 = QT AQ, so the transpose of this
expression gives
AT = QT AT Q = QT AQ = A1
1
so A1 is symmetric. Next let A1 = RAR1 an show that A1 is tridiagonal. Since R is
upper triangular R1 is upper triangular. Then A1 is the product of an upper triangular
matrix times a tridiagonal matrix times an upper triangular matrix. Now a tridiagonal
matrix A, times an upper triangular matrix R1 gives a matrix that is upper triangular with
an additional nonzero subdiagonal. Such a matrix is called an upper Hessenberg matrix.
Now an upper triangular matrix R times an upper Hessenberg matrix (AR1 ) will be upper
Hessenberg, so the entire product RAR1 is upper Hessenberg. From the rst part of this
problem A1 is symmetric and therefore since (A1 )ij = 0 for i > j + 1 we must have (A1 )ij = 0
for j > i + 1 and A1 is therefore triangular.
Problem 24
Following the hint in the book if |xi | |xj | for all j , then we have
|
j
aij xj | = |xi ||
aij
j
xj
| |xi |
xi
j
|aij |
xj
|xi |
xi
j
|aij | < |xi | .
Since the sum j |aij | < 1. Thus if x is an eigenvector with eigenvalue we have that the
i-th component of Ax = x is given by
xi =
aij xj
j
so taking the absolute value of both sides and using the above we obtain |xi | < |xi | which
by dividing by |xi | on both sides give || < 1.
Problem 25
For the rst A we have that (from the Gershgorin circle theorem) that
| 0.3| 0.5
| 0.2| 0.7
| 0.1| 0.6
Since the sum of the absolute values of the elements along every row is less than 1, from
problem 24 in this book we know that || < 1, and therefore that ||max < 1. The three
Gershgorin circles for the rst A are given by the above. Thus incorporating the above we
can derive that
0.2 0.8
0.5 0.9
0.5 0.7
Thus all eigenvalues must satisfy 0.5 0.9.
For the second matrix the rows dont add to something less than 1, so we cant conclude
that || < 1. But the Gershgorin circle theorem still holds and we can conclude that
| 2| 1
| 2| 2
| 2| 1
Thus the most restrictive condition holds and we have only that the eigenvalues of A can be
bounded by 1 3.