to Introduction MATRIX ALGEBRA
KAW
Copyrighted to Autar K. Kaw 2002
1
Introduction to MATRIX ALGEBRA
Autar K. Kaw University of South Florida
Autar K. Kaw Professor & Jerome Krivanek Distinguished Teacher Mechanical Engineering Department University of South Florida, ENB 118 4202 E. Fowler Avenue Tampa, FL 33620-5350. Office: [phone] Fax: [phone]
E-mail: [e-mail] URL: http://www.eng.usf.edu/~kaw
2
Table of Contents
Chapter 1: Introduction ..................................................... 6 What is a matrix? So what is a matrix? What are the special types of matrices? Do non-square matrices have diagonal entries? When are two matrices considered to be equal? KEYTERMS CH1 Chapter 2: Vectors ......................................................... 20 What is a vector? When are two vectors equal? How do you add two vectors? What is a null vector? What is a unit vector? How do you multiply a vector by a scalar? What do you mean by a linear combination of vectors? What do you mean by vectors being linearly independent? What do you mean by the rank of a set of vectors? How can vectors be used to write simultaneous linear equations? What is the definition of the dot product of two vectors? KEYTERMS CH2 Chapter 3: Binary Matrix Operations ................................. How do you add two matrices? How do you subtract two matrices? How do I multiply two matrices? What is a scalar product of a constant and a matrix? What is a linear combination of matrices? What are some of the rules of binary matrix operations? KEYTERMS CH3 41
Chapter 4: Unary Matrix Operations ................................. 55 What is a skew-symmetric matrix? How does one calculate the determinant of any square matrix? Is there a relationship between det (AB), and det (A) and det (B)? Are there some other theorems that are important in finding the determinant? KEYTERMS CH4 Chapter 5: System of Equations ........................................ 72 Matrix algebra is used for solving system of equations. Can you illustrate this concept? A system of equations can be consistent or inconsistent. What does that mean?
3
How can one distinguish between a consistent and inconsistent system of equations? But, what do you mean by rank of a matrix? If a solution exists, how do we know whether it is unique? If we have more equations than unknowns in [A] [X] = [C], does it mean the system is inconsistent? Consistent system of equations can only have a unique solution or infinite solutions. Can a system of equations have a finite (more than one but not infinite) number of solutions? Can you divide two matrices? How do I find the inverse of a matrix? Is there another way to find the inverse of a matrix? If the inverse of a square matrix [A] exists, is it unique? KEYTERMS CH5 Chapter 6: Gaussian Elimination ....................................... 107 How are a set of equations solved numerically? Are there any pitfalls of Nave Gauss Elimination Method? What are the techniques for improving Nave Gauss Elimination Method? How does Gaussian elimination with partial pivoting differ from Nave Gauss elimination? Can we use Nave Gauss Elimination methods to find the determinant of a square matrix? KEYTERMS CH6 Chapter 7: LU Decomposition ........................................... 129 I hear about LU Decomposition used as a method to solve a set of simultaneous linear equations? What is it and why do we need to learn different methods of solving a set of simultaneous linear equations? How do I decompose a non-singular matrix [A], that is, how do I find [A] = [L ][U ] ? How do I find the inverse of a square matrix using LU Decomposition? KEYTERMS CH7 Chapter 8: Gauss- Seidal Method....................................... 144 Why do we need another method to solve a set of simultaneous linear equations? Chapter 9: Adequacy of Solutions....................................... 158 What does it mean by ill conditioned and well-conditioned system of equations? So what if the system of equations is ill conditioning or well conditioning? To calculate condition number of an invertible square matrix, I need to know what norm of a matrix means. How is the norm of a matrix defined? How is norm related to the conditioning of the matrix? What are some of the properties of norms? X / X and C / C Is there a general relationship that exists between or between X / X A / A and ? If so, it could help us identify well-conditioned and ill conditioned system of equations.
4
If there is such a relationship, will it help us quantify the conditioning of the matrix, that is, tell us how many significant digits we could trust in the solution of a system of simultaneous linear equations? How do I use the above theorems to find how many significant digits are correct in my solution vector? KEYTERMS CH9 Chapter 10: Eigenvalues and Eigenvectors .......................... 173 What does eigenvalue mean? Can you give me a physical example application of eigenvalues and eigenvectors? What is the general definition of eigenvalues and eigenvectors of a square matrix? How do I find eigenvalues of a square matrix? What are some of the theorems of eigenvalues and eigenvectors? How does one find eigenvalues and eigenvectors numerically? KEYTERMS CH10
5
6
Chapter 1 Introduction
_________________________________
After reading this chapter, you should be able to Know what a matrix is Identify special types of matrices When two matrices are equal
_________________________________
What is a matrix? Matrices are everywhere. If you have used a spreadsheet such as Excel or Lotus or written a table, you have used a matrix. Matrices make presentation of numbers clearer and make calculations easier to program. Look at the matrix below about the sale of tires in a Blowoutr'us store given by quarter and make of tires. Quarter 1 Quarter 2 Quarter 3 Quarter 4
25 5 6
20 10 16 3 15 7 2 25 27
Tirestone Michigan Copper
If one wants to know how many Copper tires were sold in Quarter 4, we go along the row `Copper' and column `Quarter 4' and find that it is 27.
So what is a matrix?
A matrix is a rectangular array of elements. The elements can be symbolic expressions or numbers. Matrix [A] is denoted by
a11 a [A] = 21 M a m1 a12 a 22 am2 ....... a1n ....... a 2 n M ....... a mn
Row i of [A] has n elements and is [ai1 ai 2 ....a in ] and
7
a1 j a 2j Column j of [A] has m elements and is M a mj
Each matrix has rows and columns and this defines the size of the matrix. If a matrix [A] has m rows and n columns, the size of the matrix is denoted by m n. The matrix [A] may also be denoted by [A]m n to show that [A] is a matrix with m rows and n columns. Each entry in the matrix is called the entry or element of the matrix and is denoted by aij where i is the row number and j is the column number of the element. The matrix for the tire sales example could be denoted by the matrix [A] as
25 20 3 2 [A] = 5 10 15 25 6 16 7 27
There are 3 rows and 4 columns, so the size of the matrix is 3 4. In the above [A] matrix, a 34 = 27 .
What are the special types of matrices? Vector: A vector is a matrix that has only one row or one column. There are two types of vectors row vectors and column vectors. Row vector: If a matrix has one row, it is called a row vector
[B] = [b1 b 2 KK b m ]
and `m' is the dimension of the row vector.
_________________________________
Example
Give an example of a row vector.
Solution
[B] = [25 20 3 2 0] is an example of a row vector of dimension 5.
_________________________________
Column vector: If a matrix has one column, it is called a column vector
8
c1 M [C] = M c n
and n is the dimension of the vector.
_________________________________
Example
Give an example of a column vector.
Solution
25 [C] = 5 is an example of a column vector 6
of dimension 3.
_________________________________
Submatrix: If some row(s) or/and column(s) of a matrix [A] are deleted (no rows or columns may be deleted), the remaining matrix is called a submatrix of [A]. Example
Find some of the submatrices of the matrix
[A] =
4 6 2 3 - 1 2
Solution
4 6 2 4 6 2 , , [4 6 2], [4], are some of the submatrices of [A]. Can you find 3 - 1 2 3 - 1 2 other submatrices of [A]?
_________________________________
Square matrix: If the number of rows (m) of a matrix is equal to the number of columns (n) of the matrix, (m = n), it is called a square matrix. The entries a11, a22, . . . ann are
9
called the diagonal elements of a square matrix. Sometimes the diagonal of the matrix is also called the principal or main of the matrix.
_________________________________
Example
Give an example of a square matrix.
Solution
25 20 3 [A] = 5 10 15 6 15 7
is a square matrix as it has same number of rows and columns, that is, three. The diagonal elements of [A] are a11 = 25, a22 = 10, a33 = 7.
_________________________________
Upper triangular matrix: A m n matrix for which aij = 0, i>j is called an upper triangular matrix. That is, all the elements below the diagonal entries are zero.
_________________________________
Example
Give an example of an upper triangular matrix.
Solution
-7 0 10 0 - 0.001 [A] = 6 0 0 15005
is an upper triangular matrix.
_________________________________
Lower triangular matrix: A m n matrix for which aij = 0, j > i is called a lower triangular matrix. That is, all the elements above the diagonal entries are zero.
_________________________________
10
Example
Give an example of a lower triangular matrix.
Solution
0 0 1 0 .3 1 0 [A] = 0 .6 2 .5 1
is a lower triangular matrix.
_________________________________
Diagonal matrix: A square matrix with all non-diagonal elements equal to zero is called a diagonal matrix, that is, only the diagonal entries of the square matrix can be non-zero, (aij = 0, i j).
_________________________________
Example
Give examples of a diagonal matrix.
Solution
3 0 0 [A] = 0 2.1 0 0 0 5
is a diagonal matrix. Any or all the diagonal entries of a diagonal matrix can be zero. For example
3 0 0 [A] = 0 2.1 0 0 0 0
is also a diagonal matrix.
_________________________________
Identity matrix: A diagonal matrix with all diagonal elements equal to one is called an identity matrix, (aij = 0, i j; and aii = 1 for all i).
11
.
_________________________________
Example
Give an example of an identity matrix.
Solution
1 0 [A] = 0 0 0 1 0 0 0 0 1 0 0 0 0 1
is an identity matrix.
_________________________________
Zero matrix: A matrix whose all entries are zero is called a zero matrix, (aij = 0 for all i and j).
_______________________________
Example
Give examples of a zero matrix.
Solution
0 0 0 [A] = 0 0 0 0 0 0
[B] =
0 0 0 0 0 0
0 0 0 0 [C] = 0 0 0 0 0 0 0 0
[D] = [0
0 0]
are all examples of a zero matrix.
12
_________________________________
Tridiagonal matrices: A tridiagonal matrix is a square matrix in which all elements not on the following are zero - the major diagonal, the diagonal above the major diagonal, and the diagonal below the major diagonal.
_________________________________
Example
Give an example of a tridiagonal matrix.
Solution
2 2 [A] = 0 0 4 0 0 3 9 0 0 5 2 0 3 6
is a tridiagonal matrix.
_________________________________
Do non-square matrices have diagonal entries?
Yes, for a m n matrix [A], the diagonal entries are a11 , a 22 ..., a k -1,k -1 , a kk where k=min
{m,n}.
_________________________________
Example
What are the diagonal entries of
3.2 5 6 7 [A]= 2.9 3.2 5.6 7.8
Solution
The diagonal elements of [A] are a11 = 3.2 and a 22 = 7.
_________________________________
13
Diagonally Dominant Matrix: A n n square matrix [A] is a diagonally dominant matrix if
a ii | a ij | for all i =1, 2, ..., n
j =1 i j
n
and a ii > | a ij | for at least one i,
j =1 i j
n
that is, for each row, the absolute value of the diagonal element is greater than or equal to the sum of the absolute values of the rest of the elements of that row, and that the inequality is strictly greater than for at least one row. Diagonally dominant matrices are important in ensuring convergence in iterative schemes of solving simultaneous linear equations.
_________________________________
Example
Give examples of diagonally dominant matrices and not diagonally dominant matrices.
Solution
7 15 6 2 - 4 - 2 [A] = 3 2 6
is a diagonally dominant matrix as
a11 = 15 = 15 a12 + a13 = 6 + 7 = 13
a 22 = - 4 = 4 a 21 + a 23 = 2 + 2 = 4 a 33 = 6 = 6 a31 + a 32 = 3 + 2 = 5
and for at least one row, that is Rows 1 and 3 in this case, the inequality is a strictly greater than inequality.
9 - 15 6 2 [A] = -4 2 3 - 2 5.001
14
is a diagonally dominant matrix as
a11 = - 15 = 15 a12 + a13 = 6 + 9 = 15 a 22 = - 4 = 4 a 21 + a 23 = 2 + 2 = 4 a 33 = 5.001 = 5.001 a31 + a32 = 3 + - 2 = 5
the inequalities are satisfied for all rows and it is satisfied strictly greater than for at least one row (in this case it is Row 3)
25 5 1 [A] = 64 8 1 144 12 1
is not diagonally dominant as
a 22 = 8 = 8 a 21 + a 23 = 64 + 1 = 65
_________________________________
When are two matrices considered to be equal?
Two matrices [A] and [B] are equal if the size of [A] and [B] is the same (number of rows and columns are same for [A] and [B]) and aij = bij for all i and j.
_________________________________
Example
What would make
[A] =
[B] =
2 3 to be equal to 6 7
3 , b 22
b11 6
Solution
The two matrices [A] and [B ] would be equal if b11 = 2, b22 = 7.
15
Key Terms
Matrix
Vector
Sub-matrix
Square matrix Diagonal matrix Tridiagonal matrix
Upper triangular matrix Identity matrix
Lower triangular matrix Zero matrix
Diagonally dominant matrix Equal matrices.
Homework Assignment
1.
Write an example of a row vector of dimension 4. Answer: [5 6 2 3]
2. Write an example of a column vector of dimension 4.
5 - 7 Answer: 3 2.5
3. Write an example of a square matrix of order 4 4. 9 0 - 2 3 - 2 3 5 1 Answer: 1.5 6 7 8 1.1 2 3 4
4. Write an example of a tridiagonal matrix of order 4 4. 3 0 0 6 2.1 2 2.2 0 0 6.2 - 3 3.5 0 2.1 4.1 0
Answer:
16
5. Write an example of a identity matrix of order 5 5.
1 0 Answer: 0 0 0
0 1 0 0 0
0 0 1 0 0
0 0 0 1 0
0 0 0 0 1
6. Write an example of a upper triangular matrix of order 4 4.
6 0 Answer: 0 0
2 1 0 0
3 2 4 0
9 3 5 6
7. Write an example of a lower triangular matrix of order 4 4.
2 3 Answer: 4 5
0 1 2 3
0 0 4 5
0 0 0 6
8. Are these matrices strictly diagonally dominant?
15 6 7 a) [A] = 2 - 4 2 3 2 6 7 5 6 2 - 4 2 b) [A] = 3 2 - 5
17
5 3 2 c) [A] = 6 - 8 2 7 - 5 12
Answer: a) Yes b) No c)No
9. .Find all the submatrices of 0 -7 10 [ A] = 0 - 0.001 6 Answer: [10] , [- 7], [0] , [0] , [- 0.001] , [6] 10 - 7 0 0 , - .001 , 6 , [10 - 7 0], 10 -7 [0 - 0.001 6], 0 - 0.001 , 0 10 0 - 7 0 6 , - 0.001 6 , [10,-7], [10,0], [-7,0], [0,6], [0,-0.001],[-0.001,6]
4 - 1 If [A] = 0 2
10.
What are b11 and b12 in
b [B] = 11 0
b12 4
if [B] = 2[A]. Answers: 8, -2
18
11.
0 10 0 -7 10 and [ B] = - 7 - 0.001 equal? Are [ A] = 0 - 0.001 6 0 6 Answer: No
12. A square matrix [A] is lower triangular if
A. B. C. D.
aij = 0 for i>j aij = 0 for j>i aij = 0 for i=j aij = 0 for i+j=odd integer Answer: B
13. A square matrix [A] is upper triangular if A. aij = 0 for i>j B. aij = 0 for j>i C. aij = 0 for i=j D. aij = 0 for i+j=odd integer Answer: A
19
Chapter 2 Vectors
_________________________________
After reading this chapter, you should be able to
Know what a vector is How to add and subtract vectors How to find linear combination of vectors and their relationship to a set of equations Know what it means to have linearly independent set of vectors How to find the rank of a set of vectors
_______________________________
What is a vector?
A vector is a collection of numbers in a definite order. If it is a collection of `n' numbers, r it is called a n-dimensional vector. So the vector A given by
a1 r a 2 A= M a n
is a n-dimensional column vector with n components, a1, a2 ,. . . , an. The above is a column vector. A row vector [B] is of the form r B = [b1, b2,. . . , bn] r where B is a n-dimensional row vector with n components b1, b2, . . ., bn.
_________________________________
Example
Give an example of a 3-dimensional column vector.
Solution
20
Assume a point in space is given by its (x,y,z) coordinates. Then if the value of x=3, y=2, x 3 z=5, the column vector corresponding to the location of the points is y = 2 . z 5
_________________________________
When are two vectors equal? r r Two vectors A and B are equal if they are of the same dimension and if their corresponding components are equal.
Given
a1 r a 2 A= M a n
and
b1 r b B = 2 M bn r r then A = B if ai = bi, i = 1, 2, . . . ., n.
_________________________________
Example
r What are the values of the unknown components in B if
b1 2 3 r r 3 A = and B = 4 4 1 b4 r r and A = B .
Solution
b1 = 2, b4 = 1 .
21
How do you add two vectors?
Two vectors can be added only if they are of the same dimension and the addition is given by
a1 b1 a b [A] + [B] = 2 + 2 M M a n bn a1 + b1 a + b 2 = 2 M a n + bn
_________________________________
Example
Add the two vectors
2 5 3 r r and B = - 2 A= 4 3 1 7
Solution
2 5 r r 3 - 2 A+ B = + 4 3 1 7 2 + 5 3 - 2 = 4 + 3 1 + 7
22
7 1 = 7 8
_________________________________
Example
A store sells three brands of tires, Tirestone, Michigan and Cooper. In quarter 1, the sales are given by the vector
25 r A1 = 5 6
where the rows represent the three brands of tires sold Tirestone, Michigan and Cooper. In quarter 2, the sales are given by
20 r A2 = 10 6
What is the total sale of each brand of tire in the first half year?
Solution
The total sales would be given by r r r C = A1 + A2
25 20 = 5 + 10 6 6 25 + 20 = 5 + 10 6+6 45 = 15 12
23
So number of Tirestone tires sold is 45, Michigan is 15 and Cooper is 12 in the first half year.
What is a null vector?
A null vector is where all the components are zero.
_________________________________
Example
Give an example of a null vector or zero vector.
Solution
0 0 The vector is an example of a zero or null vector. 0 0
_________________________________
What is a unit vector? r A unit vector U is defined as
u1 r u 2 U = M u n
where
2 2 2 u12 + u 2 + u 3 + K + u n = 1
_________________________________
Example
Give examples of 3-dimensional unit column vectors.
Solution
Examples include
24
1 3 1 1 , 0 , 3 0 1 3
1 2 0 1 , 1 , etc. 2 0 0
_________________________________
How do you multiply a vector by a scalar? r If k is a scalar and A is a n-dimensional vector, then
a1 ka1 a ka r kA = k 2 = 2 M M a n ka n
_________________________________
Example
r What is 2 A if
25 r A = 20 5
Solution
25 r 2 A = 2 20 5 (2)(25) = (2)(20) (2)(5) 50 = 40 10
25
_________________________________
Example
A store sells three brands of tires, Tirestone, Michigan and Cooper. In quarter 1, the sales are given by the vector
25 r A = 25 6
If the goal is to increase the sales of all tires by at least 25% in the next quarter, how many of each brand should be the goal of the store?
Solution
r Since the goal is to increase the sales by 25%, one would multiply the A vector by 1.25,
25 r B = 1.2525 6 31.25 = 31.25 7.5
Since the number of tires is an integer we can say that the goal of sales would be
32 r B = 32 8
What do you mean by a linear combination of vectors? r r r Given A1 , A2 , .......... , Am as m vectors of same dimension n, then if k1, k2, ....... , km
are scalars, then r r r k1 A1 + k2 A2 + . . ............. + km Am is a linear combination of the m vectors.
_________________________________
Example
26
Find the linear combinations a) [A] [B], and b) [A] + [B] 3[C], where
2 1 10 r r r A = 3 , B = 1 , C = 1 6 2 2
Solution
2 1 r r a) A - B = 3 - 1 6 2 2 - 1 = 3 - 1 6 - 2 1 = 2 4 2 1 10 r r r b) A + B - 3C = 3 + 1 - 3 1 6 2 2 2 + 1 - 30 = 3 +1- 3 6+ 2-6 - 27 = 1 2
_________________________________
What do you mean by vectors being linearly independent? r r r A set of vectors A1 , A2 , K , Am are considered to be linearly independent if
27
r r r r k1 A1 + k2 A2 + . ............ . + km Am = 0
has only one solution of k1 = k2 = .... = km= 0.
_________________________________
Example
Are the three vectors
25 5 1 r r r 8 , A = 1 A1 = 64 , A2 = 3 144 12 1
linearly independent?
Solution
Writing the linear combination of the three vectors
25 5 1 0 64 + k 8 + k 1 = 0 k1 2 3 144 12 1 0
gives
25k1 + 5k 2 + k 3 0 64k + 8k + k = 0 1 2 3 144k1 + 12k 2 + k 3 0
The above equations has only one solution, k1 = k2 = k3 = 0. But how do we show that this is the only solution? This is shown below. The above equations are 25k1 + 5k 2 + k 3 = 0 64k1 + 8k 2 + k 3 = 0 144k1 + 12k 2 + k 3 = 0 Subtracting eqn (1) from eqn (2) gives
39k1 + 3k 2 = 0
k 2 = -13k1
(1) (2) (3)
(4)
Multiplying eqn (1) by 8 and subtracting it from eqn (2) that is first multiplied by 5 gives
28
120k1 - 3k 3 = 0
k 3 = 40k1
Remember we found eqn (4) and eqn (5) just from eqns (1) and (2). Substitution of eqn (4) and (5) in eqn (3) for k1 and k 2 gives
144k1 + 12( -13k1 ) + 40k1 = 0 28k1 = 0
k1 = 0
(5)
This means that k1 has to be zero, and coupled with equations (4) and (5), k2 and k3 are also zero. So the only solution is k1 = k2 = k3 = 0. The three vectors hence are linearly independent.
_________________________________
Example
Are the three vectors
1 2 6 r r 5, A = 14 A1 = 2, A2 = 3 5 7 24
linearly independent?
Solution
By inspection, r r r A3 = 2 A1 + 2 A2 or r r r - 2 A1 - 2 A2 + A3 = 0 So the linear combination r r r r k1 A1 + k 2 A2 + k 3 A3 = 0 has a non-zero solution
k1 = -2, k 2 = -2, k 3 = 1
Hence the set of vectors is linearly dependent.
29
What if I cannot prove by inspection, what do I do? Put the linear combination of three vectors equal to the zero vector,
1 2 6 0 2 + k 5 + k 14 = 0 k1 2 3 5 7 24 0
give
k1 + 2 k 2 + 6 k 3 = 0
2k1 + 5k 2 + 14k 3 = 0 5k1 + 7k 2 + 24k 3 = 0 Multiplying eqn (1) by 2 and subtract from eqn (2) gives
(1) (2) (3)
k 2 + 2k 3 = 0 k 2 = -2k 3
Multiplying eqn (1) by 2.5 and subtract from eqn (2) gives
- 0.5k1 - k 3 = 0
(4)
k1 = -2k 3
Remember we found eqn (4) and eqn (5) just from eqns (1) and (2). Substitute eqn (4) and (5) in eqn (3) for k1 and k 2 gives 5(- 2k 3 ) + 7(- 2k 3 ) + 24k 3 = 0
- 10k 3 - 14k 3 + 24k 3 = 0
0=0
(5)
This means any values satisfying eqns (4) and (5) will satisfy eqns (1), (2) and (3) simultaneously. For example chose
k 3 = 6 , then
k 2 = -12 from eqn (4)
and
k1 = -12 from eqn (5).
30
Hence we have nontrivial solution of [k1
k2
k 3 ] = [- 12 - 12 6]. This implies the
three given vectors are linearly dependent. Can you find another nontrivial solution?
What about the following three vectors? 1 2 6 2, 5, 14 5 7 25 Are they linearly dependent or linearly independent?
Note that the only difference between this set of vectors and the previous one is the third entry in the third vector. Hence, equations (4) and (5) are still valid. What conclusion do you draw when you plug in equations (4) and (5) in the third equation: 5k1 + 7k 2 + 25k 3 = 0 ? What has changed?
Example
Are the three vectors
25 5 1 r r r 8 , A = 1 A1 = 64, A2 = 3 89 13 2
linearly independent.
Solution
Writing the linear combination of the three vectors
25 5 1 0 64 + k 8 + k 1 = 0 k1 2 3 89 13 2 0
gives
25k1 + 5k 2 + k 3 0 64k + 8k + k = 0 1 2 3 89k1 + 13k 2 + 2k 3 0
In addition to k1 = k2 = k3 = 0, one can find other solutions for which k1, k2, k3 are not equal to zero. For example k1 = 1, k2 = -13, k3 = 40 is also a solution. This implies
31
25 5 1 0 64 - 13 8 + 40 1 = 0 1 89 13 2 0
So the linear combination that gives us a zero vector consists of non-zero constants. r r r Hence A1 , A2 , A3 are linearly dependent.
_________________________________
What do you mean by the rank of a set of vectors?
From a set of n-dimensional vectors, the maximum number of linearly independent vectors in the set is called the rank of the set of vectors. Note that the rank of the vectors can never be greater than its dimension.
_________________________________
Example
What is the rank of
25 5 1 r r r 8 , A = 1 A1 = 64 , A2 = 3 144 12 1
r r r Since we found in a previous example that A1 , A2 , and A3 are linearly independent, the r r r rank of the set of vectors A1 , A2 , A3 is 3.
Solution:
_________________________________
Example
What is the rank of
25 5 1 r r r 8 , A = 1 A1 = 64, A2 = 3 89 13 2
32
Solution
r r r In an earlier example, we found that A1 , A2 and A3 are linearly dependent, the rank of r r r A1 , A2 , A3 is hence not 3, and is less than 3. Is it 2? Let us choose
25 5 r r A1 = 64, A2 = 8 89 13 r r Linear combination of A1 and A2 equal to zero has only one solution. So the rank is 2.
_________________________________
Example
What is the rank of
1 2 3 r r r 2, A = 3 A1 = 1 , A2 = 3 2 4 5
Solution
r From inspection, A 2 = 2A1 , that implies r r 2 A1 - A 2 + 0 A 3 = 0.
Hence
r r k1 A1 + k 2 A 2 + k 3 A 3 = 0.
has a nontrivial solution. r So A1 , A 2 , A 3 are linearly dependent, and hence the rank of the three vectors is not 3. Since
A 2 = 2 A1 , A1 and A 2 are linearly dependent,
but
r r k1 A1 + k 3 A3 = 0.
has trivial solution as the only solution. So A1 and A 3 are linearly independent. The rank of the above three vectors is 2.
___________________________
33
Prove that if a set of vectors contains the null vector, the set of vectors is linearly dependent. r r r Let A1 , A2 ,K, Am be a set of n-dimensional vectors, then r r r k1 A1 + k 2 A2 + K + k m Am = 0 r is a linear combination of the `m' vectors. Then assuming if A1 is the zero or null vector, any value of k1 coupled with k2 = k3 = K = k m = 0 will satisfy the above equation.
Hence the set of vectors is linearly dependent as more than one solution exists. Prove that if a set of vectors are linearly independent, then a subset of the m vectors also has to be linearly independent. Let this subset be r r r Aa1 , Aa 2 ,K, Aap where p < m. Then if this subset is linearly dependent, the linear combination r r r k1 Aa1 + k 2 Aa 2 + K + k p Aap = 0 has a non-trivial solution. So r r r r r k1 Aa1 + k 2 Aa 2 + K + k p Aap + 0 Aa ( p +1) + ....... + 0 Aam = 0 has a non-trivial solution too, where A a ( p +1) ,K , A am are the rest of the (m-p) vectors. However, this is a contradiction. Therefore, a subset of linearly independent vectors cannot be linearly dependent. Prove that if a set of vectors is linearly dependent, then at least one vector can be written as a linear combination of others. r r r Let A1 , A2 ,K, Am be linearly dependent, then there exists a set of numbers k1 ,K, k m not all of which are zero for the linear combination r r r k1 A1 + k 2 A2 + K + k m Am = 0 that one of non-zero values of k i , i = 1,K , m , is for let i = p, then k p -1 k p +1 k k r A p -1 - A p +1 - K - m A m . A p = - 2 A2 - K - kp kp kp kp
and that proves the theorem. Prove that if the dimension of a set of vectors is less than the number of vectors in the set, then the set of vectors is linearly dependent. Can you prove it??
34
How can vectors be used to write simultaneous linear equations?
A set of m linear equations with n unknowns is written as a11 x1 + K + a1n x n = c1 a 21 x1 + K + a 2 n x n = c 2
M M
M M
a m1 x1 + K + a mn x n = c n where x1 , x 2 ,K, x n are the unknowns, then in the vector notation they can be written as r r r r x1 A1 + x 2 A2 + K + x n An = C where a r 11 A1 = M am1 a r 12 A2 = M am 2 a1n r An = M a mn The problem now becomes whether you can find the scalars x1 , ........., x n such that the linear combination r r r x1 A1 + .......... + x n An = C .
Example
Write 25 x1 + 5 x 2 + x3 = 106.8 64 x1 + 8 x 2 + x3 = 177.2 144 x1 + 12 x 2 + x3 = 279.2
35
as a linear combination of vectors.
Solution
25 x1 64 x 1 144 x1 + 5x2 + 8 x2 + 12 x 2 + x3 106.8 + x3 = 177.2 + x3 279.2
25 5 1 106.8 64 + x 8 + x 1 = 177.2 x1 2 3 144 12 1 279.2
What is the definition of the dot product of two vectors? r r Let A = [a1 , a 2 , K , a n ] and B = [b1 , b2 , K , bn ] be two n-dimensional vectors. Then the dot r r product of the two vectors A and B is defined as
n r r A B = a1b1 + a 2 b2 + K + a n bn = ai bi i =1
A dot product is also called an inner product or scalar.
_________________________________
Example
r r Find the dot product of the two vectors A = (4, 1, 2, 3) and B = (3, 1, 7, 2).
Solution r r A B = (4,1,2,3) (3,1,7,2)
= (4)(3)+(1)(1)+(2)(7)+(3)(2) = 33
_________________________________
Example
A product line needs three types of rubber as given in the table below.
Rubber Type Weight Cost per pound
lbs
$
36
A B C
200 250 310
20.23 30.56 29.12
How much is the total price of the rubber needed?
Solution
The weight vector is given by r W = (200,250,310) and the cost vector is given by r C = (20.23,30.56,29.12) .
r r The total cost of the rubber would be the dot product of W and C . r r W C = (200,250,310 ) (20.23,30.56,29.12)
= (200)(20.23) + (250)(30.56) + (310)(29.12) = 4046 + 7640 + 9027.2 = $20713.2
_________________________________
Key Terms
Vector Unit vectors
Addition of vectors Null vector
Subtraction of vectors Scalar multiplication of vectors Linearly independent vectors
Linear combination of vectors Rank Dot products.
_________________________________
Homework
1.
For
37
2 3 1 r r r A = 9 , B = 2, C = 1 - 7 5 1
find a) b)
r r A+ B r r r 2 A - 3B + C
5 -4 11 ; b) 13 - 28 - 2
Answer: a)
2. Are
1 1 1 r r r A = 1, B = 2, C = 4 1 5 25
linearly independent?. What is the rank of the above set of vectors?
Answer:3
3.
Are
1 1 3 r r r A = 1, B = 2, C = 5 1 5 7
linearly independent?. What is the rank of the above set of vectors?
Answer:3
4.
38
Are
1 2 1 .1 r r r A = 2 , B = 4 , C = 2 .2 5 10 5 .5
linearly independent? What is the rank of the above set of vectors?
Answer:1
5.
If a set of vectors contains the null vector, the set of vectors is linearly A. independent. B. dependent.
Answer:B
6.
If a set of vectors is linearly independent, a subset of the vectors is linearly A. independent. B. dependent.
Answer:A
7. If a set of vectors is linearly dependent, then A. at least one vector can be written as a linear combination of others. B. at least one vector is a null vector.
Answer:A
8. If the dimension of a set of vectors is less than the number of vectors in the set, then the set of vectors is linearly
39
A. dependent. B. independent.
Answer:A
9.
Find the dot product of r r A = (2,1,2.5,3) and B = (-3,2,1,2.5)
Answer:6
10.
r r r If u , v , w are three nonzero vector of 2-dimensions, then r r r A. u , v , w are linearly independent r r r B. u , v , w are linearly dependent r r r C. u , v , w are unit vectors r r r r D. k1u + k 2 v + k 3 v = 0 has a unique solution. Answer:B
11.
u and v are two non-zero vectors of dimension `n'. Prove that if u and v are linearly
dependent, there is a scalar `q' such that v = q u .
12.
u and v are two non-zero vectors of dimension `n'. Prove that if there is a scalar `q'
such that v = q u , then u and v are linearly dependent.
40
Chapter 3 Binary Matrix Operations
_________________________________
After reading this chapter, you will be able to
Add, subtract and multiply matrices Learn rules of binary operations on matrices
________________________________
How do you add two matrices?
Two matrices [A] and [B] can be added only if they are the same size, then the addition is shown as [C] = [A] + [B] where cij = aij + bij
_________________________________
Example
Add two matrices
[A] = [B] =
Solution
5 2 3 1 2 7
6 7 - 2 3 5 19
[C ] = [A] + [B]
5 2 3 6 7 - 2 = + 1 2 7 3 5 19
41
5 + 6 2 + 7 3 - 2 = 1 + 3 2 + 5 7 + 19 11 9 1 = 4 7 26
_________________________________
Example
Blowout r'us store has two locations `A' and `B', and their sales of tires are given by make (in rows) and quarters (in columns) as shown below.
25 20 3 2 [A] = 5 10 15 25 6 16 7 27 20 5 4 0 [B] = 3 6 15 21 4 1 7 20
where the rows represent sale of Tirestone, Michigan and Copper tires and the columns represent the quarter number - 1, 2, 3, 4. What are the total sales of the two locations by make and quarter?
Solution
[C ] = [A] + [B]
25 20 3 2 20 5 4 0 = 5 10 15 25 + 3 6 15 21 6 16 7 27 4 1 7 20 (25 + 20 ) = (5 + 3) (6 + 4 )
(20 + 5) (3 + 4) (2 + 0) (10 + 6) (15 + 15) (25 + 21) (16 + 1) (7 + 7 ) (27 + 20)
45 25 7 2 = 8 16 30 46 10 17 14 47
So if one wants to know the total number of Copper tires sold in quarter 4 in the two locations, we would look at Row 3 Column 4 to give
42
c 34 = 47.
_________________________________
How do you subtract two matrices?
Two matrices [A] and [B] can be subtracted only if they are the same size and the subtraction is given by [D] = [A] [B] where dij = aij - bij
_________________________________
Example
Subtract matrix [B] from matrix [A].
[A] = [B] =
Solution
5 2 3 1 2 7
6 7 - 2 3 5 19
[C ] = [A] - [B]
5 2 3 6 7 - 2 = - 1 2 7 3 5 19 5 - 6 2 - 7 3 - (-2) = 1 - 3 2 - 5 7 - 19 -1 - 5 5 = - 2 - 3 - 12
_________________________________
Example
Blowout r'us store has two locations A and B and their sales of tires are given by make (in rows) and quarters (in columns) as shown below.
43
25 20 3 2 [A] = 5 10 15 25 6 16 7 27 20 5 4 0 [B] = 3 6 15 21 4 1 7 20
where the rows represent sale of Tirestone, Michigan and Copper tires and the columns represent the quarter number- 1, 2, 3, 4. How many more tires did store A sell than store B of each brand in each quarter?
Solution
[D] = [A] - [B ]
25 20 3 2 20 5 4 0 = 5 10 15 25 - 3 6 15 21 6 16 7 27 4 1 7 20 2-0 25 - 20 20 - 5 3 - 4 5 - 3 10 - 6 15 - 15 25 - 21 = 6 - 4 16 - 1 7 - 7 27 - 20 5 15 - 1 2 = 2 4 0 4 2 15 0 7
So if you want to know how many more Copper Tires were sold in quarter 4 in Store A than Store B, d34 = 7. Note that d13 = -1 implies that store A sold 1 less Michigan tire than Store B in quarter 3.
_________________________________
How do I multiply two matrices?
Two matrices [A] and [B] can be multiplied only if the number of columns of [A] is equal to the number of rows of [B] to give
[C ]mxn = [A]mxp [B] pxn
44
If [A] is a m p matrix and [B] is a p n matrix, the resulting matrix [C] is a m n matrix. So how does one calculate the elements of [C] matrix?
cij = aik bkj
k =1 p
= ai1b1 j + ai 2 b2 j + KK + aip b pj
for each i = 1, 2, KK , m , and j = 1, 2, KK , n . To put it in simpler terms, the ith row and jth column of the [C] matrix in [C] = [A][B] is calculated by multiplying the ith row of [A] by the jth column of [B], that is,
b1 j b 2j cij = ai1 ai 2 KK aip M M b pj = a i1 b1j + a i2 b 2j + ........ + a ip b pj .
[
]
= aik bkj
k =1
p
_________________________________
Example
Given
[A] =
5 2 3 1 2 7
3 - 2 [B] = 5 - 8 9 - 10
find
[C ] = [A][B ]
Solution
c12 can be found by multiplying the first row of [A] by the second column of [B],
45
-2 c12 = [5 2 3] - 8 - 10
= (5)(-2) + (2)(-8) + (3)(-10) = -56 Similarly, one can find the other elements of [C] to give
[C ] =
52 - 56 76 - 88
_________________________________
Example
Blowout r'us store location A and the sales of tires are given by make (in rows) and quarters (in columns) as shown below
25 20 3 2 [A] = 5 10 15 25 6 16 7 27
where the rows represent sale of Tirestone, Michigan and Copper tires and the columns represent the quarter number - 1, 2, 3, 4. Find the per quarter sales of store A if following are the prices of each tire. Tirestone = $33.25 Michigan = $40.19 Copper = $25.03
Solution
The answer is given by multiplying the price matrix by the quantity sales of store A. The price matrix is [33.25 40.19 25.03] , then the per quarter sales of store A would be given by
25 20 3 2 [C ] = [33.25 40.19 25.03] 5 10 15 25 6 16 7 27
c ij = a ik b kj
k =1
3
46
c11 = a 1k b k1
k =1
3
= a 11 b11 + a 12 b 21 + a 13 b 31 = (33.25)(25) + (40.19 )(5) + (25.03)(6 ) = $1182.38 Similarly
c12 = $1467.38, c13 = $877.81, c14 = $1747.06.
So each quarter sales of store A in dollars are given by the four columns of the row vector
[C ] = [1182.38
1467.38 877.81 1747.06]
Remember since we are multiplying a 1 3 matrix by a 3 4 matrix, the resulting matrix is a 1 4 matrix.
_________________________________
What is a scalar product of a constant and a matrix?
If [A] is a n n matrix and k is a real number, then the scalar product of k and [A] is another matrix [B], where bij = k aij .
_________________________________
Example
2.1 3 2 Let [ A] = . Find 2 [A] 5 1 6
Solution
2.1 3 2 [ A] = 5 1 6 Then
2[ A]
2.1 3 2 =2 5 1 6
47
(2)(2.1) (2)(3) (2)(2) = (2)(5) (2)(1) (2)(6) 4.2 6 4 = 10 2 12
_________________________________
What is a linear combination of matrices?
If [A1], [A2], ......, [Ap] are matrices of the same size and k1, k2, ......., kp are scalars, then
k1 [A1 ] + k 2 [A2 ] + ........+ k p [Ap ]
is called a linear combination of [A1 ], [ A2 ]LL , [Ap ].
_________________________________
Example
If
[A1 ] =
5 6 2 2.1 3 2 0 2.2 2 , [ A2 ] = 5 1 6, [ A3 ] = 3 3.5 6 3 2 1
then find
[A1 ] + 2[A2 ] - 0.5[A3 ]
Solution
5 6 2 2.1 3 2 0 2.2 2 = + 2 5 1 6 - 0.5 3 3.5 6 3 2 1 5 6 2 4.2 6 4 0 1.1 1 = + - 3 2 1 10 2 12 1.5 1.75 3 9.2 10.9 5 = 11.5 2.25 10
What are some of the rules of binary matrix operations?
48
Commutative law of addition
If [A] and [B] are m n matrices, then
[A] + [B ] = [B] + [A]
Associative law of addition
If [A], [B] and [C] all are m n matrices, then
[A] + ([B] + [C ]) = ([A] + [B]) + [C ]
Associative law of multiplication
If [A], [B] and [C] are m n, n p and p r size matrices, respectively, then
[A]([B ][C ]) = ([A][B])[C ]
and the resulting matrix size on both sides is m r.
Distributive law:
If [A] and [B] are m n size matrices, and [C] and [D] are n p size matrices
[A]([C ] + [D]) = [A][C ] + [A][D] ([A] + [B ])[C ] = [A][C ] + [B ][C ]
and the resulting matrix size on both sides is m p.
_________________________________
Example
Illustrate the associative law of multiplication of matrices using
1 2 2 5 2 1 [ A] = 3 5, [ B] = , [C ] = 3 5 9 6 0 2
Solution
2 5 2 1 19 27 [ B][C ] = = 9 6 3 5 36 39
1 2 91 105 3 5 19 27 = 237 276 [ A][ B][C ] = 36 39 72 78 0 2
49
1 2 20 17 3 5 2 5 = 51 45 [ A][ B] = 9 6 18 12 0 2
91 105 20 17 51 45 2 1 = 237 276 [ A][ B ][C ] = 3 5 72 78 18 12
The above illustrates the associative law of multiplication of matrices.
_________________________________
Is [A][B]=[B][A]?
If [A] [B ] exists, number of columns of [A] has to be same as the number of rows of [B ] and if [B ] [A] exists, number of columns of [B ] has to be same as the number of rows of [A] . Now for [A][B]=[B][A], the resulting matrix from [A][B] and [B][A] has to be of the same size. This is only possible if [A] and [B ] are square and are of the same size. Even then in general [A] [B ] [B ] [A] .
_________________________________
Example
Illustrate if [A] [B ] = [B ] [A] for the following matrices
[A] =
6 3 - 3 2 , [ B ] = 1 5 2 5
Solution
[ A][ B]
6 3 - 3 2 = 2 5 1 5 - 15 27 - 1 29
[ B][ A] =
50
- 3 2 6 3 1 5 2 5 - 14 1 = 16 28 [A][B] [B][A]
_________________________________
Key Terms
Addition of matrices Scalar product of matrices
Subtraction of matrices
Multiplication of matrices
Linear combination of matrices
Rules of binary matrix operation.
_________________________________
Homework
1.
For the following matrices
0 3 [A] = - 1 2 , 1 1
4 - 1 [B] = , 0 2
5 2 [C] = 3 5 6 7
find where possible a) 4[A] + 5[C] b) [A][B] c) [A] 2[C]
37 10 Answers a) = 11 33 b) 34 39 12 - 3 -7 -4 - 4 5 c) - 7 - 8 4 - 11 - 13 1
2.
Food orders are taken from two engineering departments for a takeout. The order is tabulated below.
51
Food order:
Mechanical 25 Civil 21
Chicken Fries Drink Sandwich
35 20
25 21
However they have a choice of buying this food from three different restaurants. Their prices for the three food items are tabulated below
McFat Burcholestrol
Chicken Sandwich 2.42 0.93 Fries Price Matrix: 0.95 Drink
Kentucky Sodium
2.38 0.90 1.03
2.46 0.89 1.13
Show how much each department will pay for their order at each restaurant. Which restaurant would be more economical to order from for each department? Answer: The cost in dollars is 116.80, 116.75, 120.90 for the Mechanical Department at three fast food joints. So BurCholestrol is the cheapest for the Mechanical Department. The cost in dollars is 89.37, 89.61, 93.19 for the Civil Department at three fast food joints. McFat is the cheapest for the Civil Department.
3. Given
2 3 5 3 5 5 2 [A] = 6 7 9, [B] = 2 9.[C ] = 3 9 2 1 3 1 6 7 6
Illustrate the distributive law of binary matrix operations
[A]([B] + [C ]) = [A][B] + [A][C ] .
4. Let [I] be a n n identity matrix. Show that [A][I] = [I][A] = [A] for every n n matrix [A]. Let [C ]nn = [ A]nn [I ]nn
52
Hint: cij = aip i pj
p =1
n
= ai1i1 j + KK + ai , j -1i j -1, j + aij i jj + ai ( j +1)i( j +1) j + KK + ain inj Since iij = 0 for i j = 1 for i = j cij = aij
So
[A] = [A][I ]
Similarly do the other case [I ][A] = [A]. JUST DO IT!
5. Consider there are only two computer companies in a country. The companies are named "Dude" and "Imac". Each year, company Dude keeps 1/5th of its customers, while the rest switch to Imac. Each year, Imac keeps 1/3rd of its customers, while the rest switch to Dude. If in 2002, Dude has 1/6th of the market and Imac has 5/6th of the market, what is the distribution of the customers between the two a) companies in 2003. Write the answer first as multiplication of two matrices. what would be distribution when the market becomes stable? b) Answer: a) [0.5889 0.4111] b) Stable distribution is [10/22 12/22]
(Try to do this part of the problem by finding the distribution five years from now).
6.
Given
4 12.3 - 12.3 10.3 2 11.3 - 10.3 - 11.3 [B ] = - 5 [ A] = 6 , 10.3 - 11.3 - 12.3 11 - 20
[A] [B] matrix size is _______________ Answer: 3 2
53
7.
Given
4 12.3 - 12.3 10.3 2 [ A] = 11.3 - 10.3 - 11.3 , [B ] = - 5 6 10.3 - 11.3 - 12.3 11 - 20
if [C]=[A][B], then c31= _____________________
Answer: 10.3x2+(-5)(-11.3)+11(-12.3) = -58.2
8.
Consider there are only two computer companies in a country. The companies are named "Dude" and "Imac". Each year, company Dude keeps 1/5th of its customers, while the rest switch to Imac. Each year, Imac keeps 1/3rd of its customers, while the rest switch to Dude. If in 2002, Dude has 1/6th of the market and Imac has 5/6th of the market, what is the distribution of the customers between the two companies in 2003. Write the answer as multiplication of two matrices. Answer: At the end of 2002, Dude has
1 1 2 5 + = 0.589 . Imac has 5 6 3 6
4 1 1 5 + = 0.411 5 6 3 6
1 In matrix form 5 4 5 2 1 3 6 = 0.589 1 5 0.411 3 6
54
Chapter 4 Unary Matrix Operations
_________________________________
After reading this chapter, you should be able to
Know what unary operations mean Find the transpose of a square matrix and it relationship to symmetric matrices How to find the trace of a matrix How to find the determinant of a matrix by the cofactor method
_________________________________
Transpose of a matrix: Let [A] be a m x n matrix. Then [B] is the transpose of the [A] if bji = aij for all i and j. That is, the ith row and the jth column element of [A] is the jth row and ith column element of [B]. Note, [B] would be a n m matrix. The transpose of [A] is denoted by [A]T.
_________________________________
Example
Find the transpose of
25 20 3 2 [A]= 5 10 15 25 6 16 7 27
Solution
The transpose of [A] is
[A]T
25 5 6 20 10 16 = 3 15 7 2 25 27
_________________________________
55
Note, the transpose of a row vector is a column vector and the transpose of a column vector is a row vector. Also, note that the transpose of a transpose of a matrix is the matrix itself, that is,
([A] )
T T
= [A] . Also, ( A + B ) = AT + B T ; (cA) = cAT .
T T
Symmetric matrix: A square matrix [A] with real elements where aij = aji for i=1,....n and j=1,...,n is called a symmetric matrix. This is same as, if [A] = [A]T, then [A] is a symmetric matrix.
_________________________________
Example
Give an example of a symmetric matrix.
Solution
6 21.2 3.2 3.2 21.5 8 [A] = 6 8 9.3
is a symmetric matrix as a12 = a 21 = 3.2; a 13 = a31 = 6 and a 23 = a32 = 8 .
_________________________________
What is a skew-symmetric matrix?
A nxn matrix is skew symmetric for which aij =-aji for all i and j. This is same as
[A] = -[A]T .
_________________________________
Example
Give an example of a skew-symmetric matrix.
Solution
0 1 2 - 1 0 - 5 - 2 5 0
is skew-symmetric as
56
a12 = -a21 = 1; a13 = -a31 = 2; a23 = - a32 = -5. Since aii = -aii only if aii = 0, all the diagonal elements of a skew symmetric matrix have to be zero.
Trace of a matrix: The trace of a n n matrix [A] is the sum of the diagonal entries of [A], that is,
tr [A] = aii
i =1
n
_________________________________
Example
Find the trace of
15 6 7 [A] = 2 - 4 2 3 2 6
Solution
tr [A] = a ii
i =1 3
= (15) + (-4) + (6)
= 17
_________________________________
Example
The sale of tires are given by make (rows) and quarters (columns) for Blowout r'us store location A.
25 20 3 2 [A] = 5 10 15 25 6 16 7 27
where the rows represent sale of Tirestone, Michigan and Cooper tires, and the columns represent the quarter number 1,2,3,4. Find the total yearly revenue of Store A if the prices of tires vary by quarters as follows.
57
33.25 30.01 35.02 30.05 [B] = 40.19 38.02 41.03 38.23 25.03 22.02 27.03 22.95
where the rows represent the cost of each tire made by Tirestone, Michigan and Cooper, the columns represent the quarter numbers.
Solution
To find the total sales of store A for the whole year, we need to find the sales of each brand of tire for the whole year and then add the total sales. To do so, we need to rewrite the price matrix so that the quarters are in rows and the brand names are in the columns, that is find transpose of [B ] .
[C ] = [B]T
33.25 30.01 35.02 30.05 = 40.19 38.02 41.03 38.23 25.03 22.02 27.03 22.95
T
33.25 30.01 [C ] = 35.02 30.05
40.19 38.02 41.03 38.23
25.03 22.02 27.03 22.95 40.19 38.02 41.03 38.23 25.03 22.02 27.03 22.95
Recognize now that if we find [A] [C], we get 33.25 25 20 3 2 30.01 [D] = [A][C ] = 5 10 15 25 35.02 6 16 7 27 30.05
1597 1965 1193 = 1743 2152 1325 1736 2169 1311
The diagonal elements give the sales of each brand of tire for the whole year, that is d11 = $1597 d22 = $2152 d33 = $1311 (Tirestone sales) (Michigan sales) (Cooper sales)
58
The total yearly sales of all three brands of tires are = d ii
i =1
3
= 1597 + 2152 + 1311 = $5060 and this is the trace of the matrix [D].
______________________________
Define the determinant of a matrix.
A determinant of a square matrix is a single unique real number corresponding to a matrix. For a matrix [A], determinant is denoted by |A| or det(A). So do not use [A] and |A| interchangeably. For a 2 2 matrix,
[A] =
a11
a 21
a12 a 22
det (A) = a11 a22 - a12 a21
How does one calculate the determinant of any square matrix?
Let [A] be n n matrix. The minor of entry aij is denoted by Mij and is defined as the determinant of the (n - 1) (n 1) submatrix of [A], where the submatrix is obtained by deleting the ith row and jth column of the matrix [A]. The determinant is then given by
det ( A) = (- 1)
j =1
n
i+ j
aij M ij for any i = 1, 2, L , n
or det (A ) = (- 1) a ij M ij for any j = 1, 2, L, n
i+ j i =1 n
coupled with that
det ( A) = a11 for a 1 1 matrix [ A]. as we can always reduce the determinant of a matrix to
determinants of 1 1 matrices. The number (-1)i+j Mij is called the cofactor of aij and is denoted by Cij. The above equation for the determinant can then be written as
det ( A) = aij C ij for any i = 1, 2, L , n
j =1 n
or
59
det ( A) = aij C ij for any j = 1, 2, L, n
i =1
n
The only reason why determinants are not generally calculated using this method is that it becomes computationally intensive. For a n n matrix, it requires arithmetic operations proportional to n!.
_________________________________
Example
Find the determinant of
25 5 1 [A] = 64 8 1 144 12 1
Solution
Method 1 det ( A) = (- 1)
j =1 3 i+ j
aij M ij for any i = 1, 2, 3
Let i = 1 in the formula det ( A) = (- 1)
3 j =1 1+1 1+ j
a1 j M 1 j
1+ 2
= (- 1) a11 M 11 + (- 1)
a12 M 12 + (- 1) a 13 M 13
1+ 3
= a11 M 11 - a12 M 12 + a13 M 13
25 5 1
M 11 = 64 8 1 144 12 1
=
8 1 12 1
= -4
25 5 1
M 12 = 64 8 1 144 12 1
60
=
64 1 144 1
= -80
25
5
1
M 13 = 64 8 1 144 12 1
=
64 8 144 12
= -384
det(A) = a11 M 11 - a12 M 12 + a13 M 13 = 25(- 4 ) - 5(- 80 ) + 1(- 384 )
= -100 + 400 - 384 = -84
Also for i=1, det ( A) = a1 j C1 j
j =1 3
C11 = (- 1) M 11
1+1
= M 11
= -4
C12 = (- 1)
= - M 12
1+ 2
M 12
= 80 C13 = (- 1)
1+ 3
M 13
= M 13
= -384
det ( A) = a11C11 + a 21C 21 + a31C 31
= (25)(- 4 ) + (5)(80 ) + (1)(- 384 ) = -100 + 400 - 384
= -84
61
Method 2 det ( A) = (- 1)
3 i =1 i+ j
aij M ij for any j = 1,2,3.
Let j=2 in the formula det ( A) = (- 1)
3 i =1 i+2
ai 2 M i 2
2+ 2
= (- 1)
1+ 2
a12 M 12 + (- 1)
a 22 M 22 + (- 1)
3+ 2
a 32 M 32
= -a12 M 12 + a 22 M 22 - a 32 M 32
25
5 8
1 1
M 12 = 64
144 12 1
=
64 1 144 1
= -80
25 5 8 1 1
M 22 = 64
144 12 1
=
25 1 144 1
= -119
25 5 8 1 1
M 32 = 64
144 12 1
=
25 1 64 1
= -39
det(A) = -a12 M 12 + a 22 M 22 - a 32 M 32 = -5(-80) + 8(-119) 12(-39) = 400 952 + 468 = -84.
62
In terms of cofactors for j=1, det ( A) = ai 2 C i 2
j =1 3
C12 = (- 1)
1+ 2
M 12
= - M 12 = 80
C 22 = (- 1)
= M 22
2+ 2
M 22
= -119
C 32 = (- 1)
3+ 2
M 32
= - M 32
= 39
det ( A) = a12 C12 + a 22 C 22 + a32 C 32
= (5)(80 ) + (8)(- 119 ) + (12)(39 )
= 400 952 + 468 = -84
_________________________________
Is there a relationship between det (AB), and det (A) and det (B)?
Yes, if [A] and [B] are square matrices of same size, then det (AB) = det (A) det (B).
Are there some other theorems that are important in finding the determinant?
Theorem 1: If a row or a column in a n n matrix [A] is zero, then det (A) =0 Theorem 2: Let [A] be a n n matrix. If a row is proportional to another row, then det(A) = 0. Theorem 3: Let [A] be a n n matrix. If a column is proportional to another column, then det (A) = 0 Theorem 4: Let [A] be a n n matrix. If a column or row is multiplied by k to result in matrix [B]. Then det(B)=k det(A).
63
_________________________________
Example
What is the determinant of 0 0 [A] = 0 0
Solution
2 3 4 5
6 7 9 2
3 4 5 1
Since one of the columns (first column in the above example) of [A] is a zero, det(A) = 0.
_________________________________
Example
What is the determinant of 2 3 [A] = 5 9
Solution
1 2 4 5
6 4 7 6 2 10 3 18
det(A) is zero because the fourth column 4 6 10 18 is 2 times the first column 2 3 5 9
_________________________________
64
Example
If the determinant of
25 5 1 [A] = 64 8 1 144 12 1
is 84, then what is the determinant of
25 10.5 1 [B] = 64 16.8 1 144 25.2 1
Solution
Since the second column of [B] is 2.1 times the second column of [A], det(B) = 2.1 det(A) = (2.1)(-84) = -176.4
_________________________________
Example
Given the determinant of
25 5 1 [A] = 64 8 1 144 12 1
is -84, what is the determinant of
5 1 25 0 - 4.8 - 1.56 [B] = 144 12 1
Solution
Since [B] is simply obtained by subtracting the second row of [A] by 2.56 times the first row of [A], det(B) = det(A) = -84.
65
_________________________________
Example
What is the determinant of
5 1 25 0 - 4.8 - 1.56 [A] = 0 0 0 .7
Solution
Since [A] is an upper triangular matrix det ( A) = aii
i =1 3
= (a11 )(a 22 )(a 33 ) = (25)(- 4.8)(0.7 )
= -84.
_________________________________
Key Terms
Transpose
Symmetric
Skew symmetric
Trace
Determinant
_________________________________
Homework
1.
25 3 6 T Let [A] = . Find [A] 7 9 2
25 7 Answer: 3 9 6 2
66
2. If [A] and [B] are two nxn symmetric matrices, show that [A]+[B] is also symmetric. Hint: Let
[C ] = [A] + [B]
cij = aij + bij for all i, j.
and
c ji = a ji + b ji for all i, j. c ji = aij + bij as [A] and [B ] are symmetric
Hence c ji = cij . 3. Give an example of a 4 4 symmetric matrix. 4. Give an example of a 4 4 skew-symmetric matrix. 5. What is the trace of 2 3 4 7 - 5 - 5 - 5 - 5 [A] = 6 6 7 9 3 10 - 5 2 Answer: 19
For
- 7 0 10 - 3 2.099 6 [A] = 5 - 1 5
67
Find the determinant of [A] using the cofactor method. Answer: -150.05 6. det (3[A]) of a n n matrix is a. 3 det (A) b. det (A) c. 3n det (A) d. 9 det (A) Answer (C.) 7. For a 5 5 matrix [A], the first row is interchanged with the fifth row, the determinant of the resulting matrix [B] is A. det (A) B. det (A) C. 5 det (A) D. 2 det (A) Answer (A) 8. 0 0 det 0 1 1 0 0 0 A. B. C. D. 0 1 0 0 0 0 is 1 0 0 1 1 Answer (C.)
68
9.
Without using the cofactor method of finding determinants, find the determinant of
0 0 0 [A] = 2 3 5 6 9 2
Answer: 0 can you answer why
10. Without using the cofactor method of finding determinants, find the determinant of 0 2 3 0 0 2 3 5 [A] = 6 7 2 3 6.6 7.7 2.2 3.3 Answer: 0 can you answer why
11. Without using the cofactor method of finding determinants, find the determinant of 5 0 [A] = 2 1 0 3 5 2 0 0 6 3 0 0 0 9
Answer: 5 x3 x6 x9 = 810 - can you answer why?
12. Given the matrix 125 25 5 1 512 64 8 1 [A] = 1157 89 13 1 4 2 1 8
69
and det(A) =-32400 find the determinant of 125 25 5 1 512 64 8 1 a) [A] = 1141 81 9 - 1 4 2 1 8 125 25 1 5 512 64 1 8 b) [A] = 1157 89 1 13 4 1 2 8 125 25 5 1 1157 89 13 1 c) [B ] = 512 64 8 1 4 2 1 8 125 25 5 1 1157 89 13 1 d) [C ] = 8 4 2 1 512 64 8 1 125 25 5 1 512 64 8 1 e) [D ] = 1157 89 13 1 8 4 2 16 Answer: a) 32400 b) 32400 c) 32400 d) 32400 e) 64800
13.
What is the transpose of
25 20 3 2 [A]= 5 10 15 25 6 16 7 27
70
Answer:
[A]T
25 5 6 20 10 16 = 3 15 7 2 25 27
14.
What values of the missing numbers will make this a skew-symmetric matrix?
2 [A]= ? 21 2 Answer: - 3 21
3 ? 6 ? ? 5 3 - 21 6 4? -4 5
15.
What values of the missing number will make this a symmetric matrix?
2 [A]= ? 21 2 3 Answer: 21
3 ? 6 ? ? 5 3 21 6 7 7 5
16.
Find the determinant of
25 5 1 [A] = 64 8 1 144 12 5
Answer: The determinant of [A] is
8 1 64 1 64 8 25 - 5 + 1 = 25(28)-5(176)+1(-384) = -564 12 5 144 5 144 12 17.
71
What is the determinant of an upper triangular matrix [A] that is of order nxn?
Answer: The determinant of an upper triangular matrix is the product of its diagonal elements.
18.
Given the determinant of
25 5 1 25 5 1 64 8 1 is -564, what is the determinant of [B ] = 39 3 0 [A] = 144 12 a 144 12 a
Justify your answer?
Answer: By inspection, Row 2 of B is (Row 2-Row 1) of A. Hence, det(B) = -564. It is not necessary to find a.
19.
Why is the determinant of the following matrix zero?
0 0 0 [A] = 2 3 5 6 9 2
Answer: The first row of the matrix is zero, hence, the determinant of the matrix is zero.
20.
Why is the determinant of the following matrix zero?
0 2 3 0 0 2 3 5 [A] = 6 7 2 3 6.6 7.7 2.2 3.3
Answer: Row 4 of the matrix is 1.1 times Row 3. Hence, its determinant is zero.
21.
Show that if [A][B]=[I], where [A], [B] and [I] are matrices of n n size and [I] is an identity matrix, then det(A) 0 and det(B) 0.
72
Answer: We know that det(AB) = det(A)det(B).
[A][B] = [I] det(AB) = det(I) det(I) =
a
i =1
n
ii
= 1 = 1
i =1
n
det(A)det(B) = 1 Therefore, det(A) 0 and det(B) 0.
73
Chapter 5 System of Equations
_________________________________
After reading this chapter, you will be able to
Setup simultaneous linear equations in matrix form and vice-versa Understand the concept of inverse of a matrix Know the difference between consistent and inconsistent system of linear equations Learn that system of linear equations can have a unique solution, no solution or infinite solutions
_________________________________
Matrix algebra is used for solving system of equations. Can you illustrate this concept?
Matrix algebra is used to solve a system of simultaneous linear equations. In fact, for many mathematical procedures such as solution of set of nonlinear equations, interpolation, integration, and differential equations, the solutions reduce to a set of simultaneous linear equations. Let us illustrate with an example for interpolation.
_________________________________
Example
The upward velocity of a rocket is given at three different times on the following table
Time, t Velocity, v
s 5 8 12
m/s 106.8 177.2 279.2
74
The velocity data is approximated by a polynomial as v(t ) = at 2 + bt + c , 5 t 12. Set up the equations in matrix form to find the coefficients a, b, c of the velocity profile.
Solution
The polynomial is going through three data points (t1 , v1 ), (t 2 , v 2 ), and (t 3 , v3 ) where from the above table
t1 = 5, v1 = 106.8 t 2 = 8, v 2 = 177.2
t 3 = 12, v3 = 279.2 Requiring that v(t ) = at 2 + bt + c passes through the three data points gives v(t1 ) = v1 = at12 + bt1 + c
2 v(t 2 ) = v 2 = at 2 + bt 2 + c
2 v(t 3 ) = v3 = at 3 + bt 3 + c
Substituting the data (t1 , v1 ), (t 2 , v 2 ), (t 3 , v3 ) gives a (5 2 ) + b(5) + c = 106.8 a (8 2 ) + b(8) + c = 177.2 a (12 2 ) + b(12) + c = 279.2 or
25a + 5b + c = 106.8 64a + 8b + c = 177.2
144a + 12b + c = 279.2
This set of equations can be rewritten in the matrix form as
25a + 5b + c 106.8 64a + 8b + c = 177.2 144a + 12b + c 279.2
The above equation can be written as a linear combination as follows
75
25 5 1 106.8 64 + b 8 + c 1 = 177.2 a 144 12 1 279.2
and further using matrix multiplications gives
25 5 1 a 106.8 64 8 1 b = 177.2 144 12 1 c 279.2
_________________________________
The above is an illustration of why matrix algebra is needed. The complete solution to the set of equations is given later in this chapter. For a general set of "m" linear equations and "n" unknowns, a11 x1 + a 22 x2 + LL + a1n xn = c1 a 21 x1 + a 22 x2 + LL + a 2 n x n = c 2 .......................................... ........................................... a m1 x1 + a m 2 x 2 + ........ + a mn x n = c m can be rewritten in the matrix form as
a11 a 21 M M a m1 a12 a 22
. . . .
am2
.
.
a1n a2n M M a mn
x1 c1 x c 2 2 = x n c m
Denoting the matrices by [A] , [X ] , and [C ] , the system of equation is
[A] [X ] = [C ] , where [A] is called the coefficient matrix, [C ] is called the right hand side vector and [X ] is called the solution vector. Sometimes [A] [X ] = [C ] systems of equations is written in the augmented form. That
is
76
a 11 a 21 [A M C] = M M a m1
a 12 a 22
a m2
...... a 1n M c1 ...... a 2n M c 2 M M ...... a mn M c n
A system of equations can be consistent or inconsistent. What does that mean?
A system of equations [A] [X]=[C] is consistent if there is a solution, and it is inconsistent if there is no solution. However, consistent system of equations does not mean a unique solution, that is, a consistent system of equation may have a unique solution or infinite solutions.
[A] [X] = [B]
Consistent System
Inconsistent System
Unique Solution
Infinite Solutions
_________________________________
Example
Give examples of consistent and inconsistent system of equations.
Solution
a) The system of equations 2 4 x 6 1 3 y = 4 is a consistent system of equations as it has a unique solution, that is, x 1 y = 1 . b) The system of equations
77
2 4 x 6 1 2 y = 3 is also a consistent system of equations but it has infinite solutions as given as follows. Expanding the above set of equations, 2x + 4 y = 6 x + 2y = 3 you can see that they are the same equation. Hence any combination of ( x, y ) that satisfies 2x + 4 y = 6 is a solution. For example ( x, y ) = (1,1) is a solution and other solutions include (x, y ) = (0.5,1.25) , (x, y ) = (0, 1.5) and so on. c) The system of equations 2 4 x 6 1 2 y = 4 is inconsistent as no solution exists.
_________________________________
How can one distinguish between a consistent and inconsistent system of equations?
A system of equations [A] [X] = [C] is consistent if the rank of A is equal to the rank of the augmented matrix [AMC ] , inconsistent if the rank of A is less then the rank of the augmented matrix [AMC ] .
But, what do you mean by rank of a matrix?
The rank of a matrix is defined as the order of the largest square submatrix whose determinant is not zero.
_________________________________
Example
What is the rank of
78
3 1 2 [A] = 2 0 5 1 2 3
Solution
The largest square submatrix possible is of order 3 and that is [A] itself. Since det(A) = 25 0, the rank of [A] = 3.
_________________________________
Example
What is the rank of
3 1 2 [A] = 2 0 5 5 1 7
Solution
The largest square submatrix of [A] is of order 3, and is [A] itself. Since det(A) = 0, the rank of [A] is less than 3. The next largest square submatrix would be a 2 2 matrix. One of the square submatrices of [A] is
[B] =
3 1 2 0
and det [B] = -2 0. Hence the rank of [A] is 2.
_________________________________
Example
How do I now use the concept of rank to find if
25 5 1 x1 106.8 64 8 1 x = 177.2 2 144 12 1 x3 279.2
is a consistent or inconsistent system of equations?
Solution
The coefficient matrix is
79
25 5 1 [A] = 64 8 1 144 12 1
and the right hand side vector
106.8 [C ] = 177.2 . 279.2
The augmented matrix is
25 5 1 M 106.8 [B] = 64 8 1 M 177.2 144 12 1 M 279.2
Since there are no square submatrices of order 4 as [B] is a 3x4 matrix, the rank of [B] is at most 3. So let us look at the square submatrices of [B] of order 3 and if any of these square submatrices have determinant not equal to zero, then the rank is 3. For example, a submatrix of the augmented matrix [B] is
25 5 1 [D] = 64 8 1 144 12 1
has det (D) = -84 0. Hence the rank of the augmented matrix [B] is 3. Since [A]=[D] , the rank of [A] =3. Since the rank of augmented matrix [B] = rank of coefficient matrix [A], the system of equations is consistent.
_________________________________
Example
Use the concept of rank of matrix to find if
25 5 1 x1 106.8 64 8 1 x = 177.2 2 89 13 2 x3 284.0
is consistent or inconsistent?
Solution
80
The coefficient matrix is given by
25 5 1 [A] = 64 8 1 89 13 2
and the right hand side
106.8 [C ] = 177.2 284.0
The augmented matrix is
25 5 1 106.8 [B] = 64 8 1 177.2 89 13 2 284.0
Since there are no square submatrices of order 4 as [B] is a 4 3 matrix, the rank of the augmented [B] is at most 3. So let us look at square submatrices of the augmented matrix [B] of order 3 and see if any of these square submatrices have determinant not equal to zero, then the rank is 3. For example a square submatrix of the augmented matrix [B] is
25 5 1 [D] = 64 8 1 89 13 2
has det(D) = 0. This means, we need to explore other square submatrices of order 3 of the augmented matrix [B]. That is,
5 1 106.8 [E ] = 8 1 177.2 13 2 284.0
det(E) = 0,
25 5 106.8 [F ] = 64 8 177.2 89 13 284.0
det(F) = 0,
81
and
25 1 106.8 [G ] = 64 1 177.2 89 2 284.0
det(G) = 0. All the square submatrices of order 3 of the augmented matrix [B] have a zero determinant. So the rank of the augmented matrix [B] is less than 3. Is the rank of [B] = 2? A 2 2 submatrix of the augmented matrix [B] is
[H ] =
and
25 5 64 8
det(H) = -1200 So the rank of the augmented matrix [B] is 2. Now we need to find the rank of the coefficient matrix [A].
25 5 1 [A] = 64 8 1 89 13 2
and det(A) = 0. So the rank of the coefficient matrix [A] is less than 3. A square submatrix of the coefficient matrix [A] is
[J ] =
5 1 8 1
det (J) = -3 0 So the rank of the coefficient matrix [A] is 2. Hence, rank of the coefficient matrix [A] = rank of the augmented matrix [B]. So the system of equations [A] [X] = [B] is consistent.
_________________________________
Example
82
Use the concept of rank to find if
25 5 1 x 1 106.8 64 8 1 x = 177.2 2 89 13 2 x 3 280.0
is consistent or inconsistent.
Solution
The augmented matrix is
25 5 1 106.8 [B] = 64 8 4 177.2 89 13 2 280.0
Since there are no square submatrices of order 4 as the augmented matrix [B] is a 4 3 matrix, the rank of the augmented matrix [B] is at most 3. So let us look at square submatrices of the augmented matrix (B) of order 3 and see if any of the 3 3 submatrices have a determinant not equal to zero. For example a square submatrix of order 3 3 of [B]
25 5 1 [D] = 64 8 1 89 13 2
det(D) = 0 So it means, we need to explore other square submatrices of the augmented matrix [B],
5 1 106.8 [E ] = 8 1 177.2 13 2 280.0
det(E) 12.0 0. So rank of the augmented matrix [B] = 3. The rank of the coefficient matrix [A] = 2 from the previous example. Since rank of the coefficient matrix [A] < rank of the augmented matrix [B], the system of equations is inconsistent. Hence no solution exists for [A] [X] = [C].
_________________________________
83
If a solution exists, how do we know whether it is unique?
In a system of equations [A] [X] = [C] that is consistent, the rank of the coefficient matrix [A] is same as the augmented matrix [A|C]. If in addition, the rank of the coefficient matrix [A] is same as the number of unknowns, then the solution is unique; if the rank of the coefficient matrix [A] is less than the number of unknowns, then infinite solutions exist.
[A] [X] = [B]
Consistent System if rank (A) = rank (A.B)
Inconsistent System if rank (A) < rank (A.B)
Unique solution if rank (A) = number of unknowns
Infinite solutions if rank (A) < number of unknowns
_________________________________
Example:
We found that the following system of equations
25 5 1 x1 106.8 64 8 1 x = 177.2 2 144 12 1 x3 279.2
is a consistent system of equations. Does the system of equations have a unique solution or does it have infinite solutions.
Solution
The coefficient matrix is
25 5 1 [A] = 64 8 1 144 12 1
and the right hand side
106.8 [C ] = 177.2 279.2
84
While finding the whether the above equations were consistent in an earlier example, we found that rank of the coefficient matrix (A) = rank of augmented matrix [ AMC ] = 3 The solution is unique as the number of unknowns = 3 = rank of (A).
________________________________
Example:
We found that the following system of equations
25 5 1 x1 106.8 64 8 1 x = 177.2 2 89 13 2 x3 284.0
is a consistent system of equations. Is the solution unique or does it have infinite solutions.
Solution
While finding the whether the above equations were consistent, we found that rank of coefficient matrix [A] = rank of augmented matrix ( AMC ) = 2 Since rank of [A] = 2 < number of unknowns =3, infinite solutions exist.
________________________________
If we have more equations than unknowns in [A] [X] = [C], does it mean the system is inconsistent?
No, it depends on the rank of the augmented matrix [ AMC ] and the rank of [A]. a) For example 106.8 25 5 1 64 8 1 x1 177.2 x2 = 144 12 1 279.2 x3 284.0 89 13 2 is consistent, since rank of augmented matrix = 3 rank of coefficient matrix = 3.
85
Now since rank of (A) = 3 = number of unknowns, the solution is not only consistent but also unique. b) For example 106.8 25 5 1 64 8 1 x1 177.2 x2 = 144 12 1 279.2 x3 280.0 89 13 2 is inconsistent, since rank of augmented matrix = 4 rank of coefficient matrix = 3. c) For example 106.8 25 5 1 64 8 1 x1 177.2 x2 = 50 10 2 213.6 x3 280.0 89 13 2 is consistent, since rank of augmented matrix = 2 rank of coefficient matrix = 2. But since the rank of [A] = 2 < the number of unknowns = 3, infinite solutions exist.
Consistent system of equations can only have a unique solution or infinite solutions. Can a system of equations have a finite (more than one but not infinite) number of solutions?
No, you can only have a unique solution or infinite solutions. Let us suppose [A] [X]=[C] has two solutions [Y] and [Z] so that [A] [Y]=[C] [A] [Z]=[C] If r is a constant, then from the two equations
r [A][Y ] = r [C ]
(1 - r )[A][Z] = (1 - r )[C ]
86
Adding the above two equations gives
r [ A][Y ] + (1 - r )[ A][Z] = r [C ] + (1 - r )[C ]
[A](r[Y ] + (1 - r )[Z ]) = [C ]
Hence
r [Y ] + (1 - r )[Z ]
is a solution to
[A][X] = [C ].
Since r is any scalar, there are infinite solutions for [A] [X] =[C] of the form r[Y]+(1r)[Z].
Can you divide two matrices?
If [A] [B]=[C] is defined, it might seem intuitive that [A]=
[C ] , but matrix division is not [B]
defined. However an inverse of a matrix can be defined for certain types of square matrices. The inverse of a square matrix [A], if existing, is denoted by [A]-1 such that [A][A] -1 = [I] = [A] -1 [A]. In other words, let [A] be a square matrix. If [B] is another square matrix of same size such that [B][A] = [I], then [B] is the inverse of [A]. [A] is then called to be invertible or nonsingular. If [A]-1 does not exist, [A] is called to be noninvertible or singular. If [A] and [B] are two nxn matrices such that [B] [A] = [I], then these statements are also true a) [B] is the inverse of [A] b) [A] is the inverse of [B] c) [A] and [B] are both invertible d) [A] [B]=[I]. e) [A] and [B] are both nonsingular f) all columns of [A] or [B]are linearly independent g) all rows of [A] or [B] are linearly independent.
_________________________________
87
Example
Show if 3 2 - 3 2 [ B] = is the inverse of [A] = 5 - 3 5 3
Solution
[ B][ A] 3 2 - 3 2 = 5 3 5 - 3 1 0 = 0 1 = [I] Since [B] [A] =[I], [B] is the inverse of [A] and [A] is the inverse of [B]. But we can also show that
[ A][ B]
- 3 2 3 2 = 5 - 3 5 3 1 0 = = [I] 0 1 to show that [A] is the inverse of [B].
_________________________________
Can I use the concept of the inverse of a matrix to find the solution of a set of equations [A] [X] = [C]?
Yes, if the number of equations is same as the number of unknowns, the coefficient matrix [A] is a square matrix. Given [A][X] = [C] Then, if [A]-1 exists, multiplying both sides by [A]-1. [A]-1 [A][X] = [A]-1 [C] [I][X] = [A]-1[C]
88
[X] = [A]-1 [C] This implies that if we are able to find [A]-1, the solution vector of [A][X] = [C] is simply a multiplication of [A]-1 and the right hand side vector, [C].
How do I find the inverse of a matrix?
If [A] is a n n matrix, then [A]-1 is a n n matrix and according to the definition of inverse of a matrix [A][A]-1 = [I]. Denoting
a11 a 21 [ A] = a n1
' a11 ' a 21 -1 [ A] = a ' n1
a12 a 22 an2
' a12 ' a 22
a1n a2n a nn a1' n ' a2n ' a nn
' an2
1 0 0 1 0 [I ] = 0
1
0 0 1
Using the definition of matrix multiplication, the first column of the [A]-1 matrix can then be found by solving
a 11 a 21 a n1 a 12 a 22 a n2 a 1n a 2n a nn
' a 11 1 ' a 21 0 = a ' 0 n1
89
Similarly, one can find the other columns of the [A]-1 matrix by changing the right hand side accordingly.
_________________________________
Example
The upward velocity of the rocket is given by
Time, t s Velocity m/s
5 8 12
106.8 177.2 279.2
In an earlier example, we wanted to approximate the velocity profile by v(t ) = at 2 + bt + c, 5 8 12 We found that the coefficients a, b, c are given by
25 5 1 a 106.8 64 8 1 b = 177.2 144 12 1 c 279.2
First find the inverse of
25 5 1 [A] = 64 8 1 144 12 1
and then use the definition of inverse to find the coefficients a, b, c.
Solution
If [A]
-1
' a11 ' = a 21 ' a31
' a12 ' a 22 ' a32
' a13 ' a 23 ' a33
is the inverse of [A],
90
Then
' 25 5 1 a11 64 8 1 a ' 21 ' 144 12 1 a31 ' a12 ' a 22 ' a32 ' a13 1 0 0 ' a 23 = 0 1 0 ' a33 0 0 1
gives three sets of equations
' 25 5 1 a11 1 64 8 1 a ' = 0 21 ' 144 12 1 a31 0 ' 25 5 1 a12 0 64 8 1 a ' = 1 22 ' 144 12 1 a32 0 ' 25 5 1 a13 0 64 8 1 a ' = 0 23 ' 144 12 1 a33 1
Solving the above three sets of equations separately gives
' a11 0.04762 ' a 21 = - 0.9524 ' a31 4.571 ' a12 - 0.08333 ' a 22 = 1.417 ' a32 - 5.000 ' a13 0.03571 ' a 23 = - 0.4643 ' a33 1.429
Hence
0.04762 - 0.08333 0.03571 - 0.4643 [ A] -1 = - 0.9524 1.417 4.571 - 5.000 1.429
Now
91
[A][X] = [C ]
where
a [ X ] = b c 106.8 [C ] = 177.2 279.2
Using the definition of [A] ,
-1
[A]-1 [A ][X] = [A]-1 [C]
[X] = [A]-1 [C]
- 0.04762 - 0.08333 0.03571 106.8 1.417 - 0.4643 177.2 = - 0.9524 4.571 - 5.000 1.429 279.2
a 0.2900 b = 19.70 c 1.050
So
v(t ) = 0.2900t 2 + 19.70t + 1.050, 5 t 12
Is there another way to find the inverse of a matrix?
For finding inverse of small matrices, inverse of an invertible matrix can be found by
[A]-1 =
where
1 adj ( A) det ( A)
C11 C adj ( A) = 21 M C n1
C12 C 22 Cn2
L C1n C2n - C nn
T
92
C11 C12 L C1n C 22 C 22 L C 2 n itself is called the where Cij are the cofactors of aij. The matrix M M C n1 L L C nn matrix of cofactors from [A]. Cofactors are defined in Chapter 4.
Example
Find the inverse of
25 5 1 [A] = 64 8 1 144 12 1
Solution
From the example in Chapter 4, we found
det ( A) = -84 .
Next we need to find the adjoint of [A]. The cofactors of A are found as follows. The minor of entry a11 is
25 5 1 M 11 = 64 8 1 144 12 1
=
8 1 12 1
= -4 The cofactors of entry a11 is
C11 = (- 1) M 11 = M 11 = -4
1+1
The minor of entry a12 is
25 5 1 M 12 = 64 8 1 144 12 1
=
64 1 144 1
93
= -80
The cofactor of entry a12 is
C12 = (- 1)
= - M 12 = 80
1+ 2
M 12
Similarly
C13 = 384
C 21 = 7 C22 = -119
C 23 = 420 C 31 = -3 C 32 = 39 C 33 = -120
Hence the matrix of cofactors of [A] is
80 - 384 - 4 [C ] = 7 - 119 420 39 - 120 - 3
The adjoint of matrix [A] is [C]T,
adj ( A) = [C ]
T
7 -3 -4 = 80 - 119 39 - 384 420 - 120
Hence
[A]-1 =
1 adj ( A) det ( A)
7 -3 -4 1 = 80 39 - 119 - 84 - 384 420 - 120
94
0.04762 - 0.08333 0.03571 = - 0.9524 1.417 - 0.4643 - 5.000 1.429 4.571
_________________________________
If the inverse of a square matrix [A] exists, is it unique?
Yes, the inverse of a square matrix is unique, if it exists. The proof is as follows. Assume that the inverse of [A] is [B] and if this inverse is not unique, then let another inverse of [A] exist called [C]. [B] is inverse of [A], then [B][A] = [I] Multiply both sides by [C], [B][A][C] = [I][C] [B][A][C] = [C] Since [C] is inverse of [A], [A][C] = [I] [B][I] = [C] [B] = [C] This shows that [B] and [C] are the same. So inverse of [A] is unique.
_________________________________
Key Terms
Consistent system Infinite solutions Rank
Inconsistent system Unique solution Inverse
_________________________________
Homework
1. For a set of equations [A] [X] = [B], a unique solution exists if
95
A. rank (A) = rank ( AM B ) B. rank (A) = rank ( AM B ) and rank (A) = number of unknowns rank (A) = rank ( AM B ) and rank (A) = number of rows of (A). 2. Rank of 4 4 A= 4 4 is A. 1 B. 2 C. 3 A 3 4 matrix can have a rank of at most A. 3 B. 4 C. 12 D. 5 4 4 4 4 4 4 4 4 4 4 4 4
3.
4.
If [A][X] = [C] has a unique solution, where the order of [A] is 3 3, [X] is 3 1, then the rank of [A] is A. 2 B. 3 C. 4 D. 5
5. Show if the following system of equations is consistent or inconsistent. If they are
96
consistent, determine if the solution would be unique or infinite ones exist.
1 2 5 x1 8 7 3 9 x = 19 2 8 5 14 x3 27
Answer: consistent; infinite solutions
6.
Show if the following system of equations is consistent or inconsistent. If they are consistent, determine if the solution would be unique or infinite ones exist.
1 2 5 x1 8 7 3 9 x = 19 2 8 5 14 x3 28
Answer: inconsistent
7.
Show if the following system of equations is consistent or inconsistent. If they are consistent, determine if the solution would be unique or infinite ones exist.
1 2 5 x1 8 7 3 9 x = 19 2 8 5 13 x3 28
Answer: consistent; unique
8.
For what values of `a' will the following equation have a. unique solution b. no solution c. infinite solutions
9.
For what (which) value(s) of `a' does the following system have zero, one, infinitely many solutions.
x1 + x 2 + x3 = 4 x3 = 2
97
(a
2
- 4 x1 + x3 = a - 2
if a = +2 or - 2, then there will be no solution. Possibility of infinite solutions does not exist.
)
Answer: if a +2 or - 2, then there will be a unique solution
10.
Find if 5 - 2.5 0.3 0.25 [ A] = and [ B] = 0.2 0.5 3 - 2 are inverse of each other.
Answer: Yes
11. Find if 0.3 - 0.25 5 2.5 [ A] = and [ B] = 0.2 0.5 2 3 are inverse of each other.
Answer: No
12.
Find the a. cofactor matrix b. adjoint matrix of
1 3 4 2 - 7 - 1 [A] = 8 1 5
13.
Find [A]-1 using any method
98
1 3 4 [A] = 2 - 7 - 1 8 1 5
Answer: [A ]
-1
2.931x10-1 1.638x10-1 = 1.552x10-1 - 6.034x10- 2 - 5.000x10-1 - 2.500x10-1
- 2.586x10-2 - 4.310x10- 2 2.500x10-1
14. Prove that if [A] and [B] are both invertible and are square matrices of same order, then ([A][B]) -1 =[B] -1 [A] -1
Hint:
([A][B ])-1 = [B]-1[A]-1 Let [C ] = [A][B ]
[C ][B ]-1 = [A][B ][B ]-1 = [A][I ] = [ A]
Again [C ] = [A][B ]
[A]-1[C ] = [A]-1[A][B] = [I ][B ] = [B ]
So
[C ][B]-1 = [A] - - - - - (1) [A]-1[C ] = [B] - - - - - (2)
99
From (1) and (2)
[C ][B]-1[A]-1[C ] = [A][B]
[A][B][B]-1[A]-1[A][B] = [A][B]
[A]-1[A][B][B ]-1[A]-1[A][B] = [A-1 ][A][B]
[B][B]-1 [A-1 ][A][B] = [B]
[B ][B][B] [A ][A][B] = [B] [B] [B] [A ][A][B] = [I ].
-1 -1 -1 -1
-1
-1
15. What is the inverse of a square diagonal matrix? Does it always exist? Hint: Inverse of a square n n diagonal matrix [A] is
[A]
-1
1 a 11 0 = 0 M
0 1 a22
L
L 0 0 M 1 L ann
L
So inverse exists only if aii 0 for all i.
16. [A] and [B] are square matrices. If [A][B] = [0] and [A] is invertible, show[B] = 0. Hint:
[A][B] = [0]
[A ][A][B] = [A] [0]
-1 -1
17. If [A] [B] [C]=[I], where [A], [B] and [C] are of the same size,
100
show that [B] is invertible.
18. Prove if [B] is invertible, [A][B]-1 = [B]-1 [A] if and only if [A][B] =[B][A] Hint: Multiply by [B ] on both sides,
-1
[A][B][B]-1 = [B]-1[A][B]-1
19. For
- 7 0 10 - 3 2.099 6 [A] = 5 - 1 5
[A]
Show det ( A) =
-1
0.2799 - 0.1099 - 0.2333 - 0.2999 - 0.3332 = 0.3999 0.04995 0.1666 6.664 10 -5
1 . det A -1
( )
20.
For what values of `a' does the linear system have x + y= 2 6x + 6 y = a a) infinite _________________________ unique solution _ solutions
b)
______________
Answer: a) 12 b) not possible
21.
Three kids - Jim, Corey and David receive an inheritance of $2,253,453. The money is put in three trusts but is not divided equally to begin with. Corey
101
gets three times more than David because Corey made an "A" in Dr. Kaw's class. Each trust is put in an interest generating investment. The three trusts of Jim, Corey and David pays an interest of 6%, 8%, 11%, respectively. The total interest of all the three trusts combined at the end of the first year is $190,740.57. How much money was invested in each trust? Set the following as equations in a matrix form. Identify the unknowns. Do not solve for the unknowns. Answer: J + C + D = $2,253,453; C = 3D ; 0.06J+0.08C+0.11D = $190,740.57
1 1 J 2,253,453 1 0 In matrix form 1 - 3 C = 0 0.06 0.08 0.11 D 190,740.57
22.
1 2 3 What is the rank of 4 6 7 ? Justify your answer. 6 10 13
Answer: In the above matrix, 2(Row 1) + Row 2 = Row 3. Hence, rank is less than 3. Row 1 and Row 2 are linearly independent. Hence, the rank of the matrix is 2.
23.
1 2 3 6 What is the rank of 4 6 7 17 ? Justify your answer. 6 10 13 29 Answer: The determinant of all the 3 3 sub-matrices is zero. Hence, the rank
is less than 3. Determinant of
2 3 = -4 0 . Hence, the rank is 2. 6 7
102
24.
1 2 3 6 What is the rank of 4 6 7 18 ? Justify your answer. 6 10 13 30
Answer: In the above matrix, 2(Row 1) + Row 2 = Row 3. Hence, rank is less than 3 as the 3 rows are linearly dependant. Determinant
of
2 3 = -4 0 . Hence, the rank is 2. 6 7
25.
How many solutions does the following system of equations have
1 2 3 a 6 4 6 7 b = 17 ? Justify your answer. 6 10 13 c 29
Answer: Rank of A = 2 Rank of A|C = 2 # of unknowns = 3. There are infinite solutions since rank of A is less than the # of unknowns.
26.
How many solutions does the following system of equations have
1 2 3 a 6 4 6 7 b = 18 ? Justify your answer. 6 10 13 c 30
Answer: Rank of A = 2 Rank of A|C = 2 # of unknowns = 3. There are infinite solutions since rank of A is less than the # of unknowns.
103
27.
By any scientific method, find the second column of the inverse of
1 2 0 4 5 0 0 0 13 ' 1 2 0 X a12 X 1 0 0 ' Answer: 4 5 0 X a22 X = 0 1 0 ' 0 0 13 X a32 X 0 0 1
' ' a12 + 2a22 = 0 ' ' 4a12 + 5a22 = 1 ' 13a32 = 0
' a12 0.667 Simplifying, ' a22 = - 0.333 ' a32 0
28.
Just write out the inverse of (no need to show any work)
1 0 0 0
1 0 0 0
0 2 0 0
0 1 2 0 0
0 0 4 0
0 0 1 4 0
Answer:
0 0 0 5 0 0 0 1 5
29.
Solve [A][X] = [B] for [X] if
10 - 7 0 7 2 and [B] = 2.5 [ A] = 2 5 2 6.012 0 6
-1
104
Answer: [X] = [A]-1[B]
10 - 7 0 7 = 2 2 5 2.5 2 0 6 6.012 52.5 = 49.06 50.072
30.
7 Let [A] be a 3 3 matrix. Suppose [ X ] = 2.5 is a solution to the 6.012
homogeneous set of equations [A] [X] =[0] (the right hand side is a zero vector of order 3 1). Does [A] have an inverse? Justify your answer. Given [A][X] = 0 Answer: -1 If [A] exists, then [A]-1[A][X] = [A]-10 [I][X] = 0 [X] = 0 This contradicts the given value of [X]. Hence, [A]-1 does not exist.
31.
Is the set of vectors
1 1 1 r r r A = 1, B = 2, C = 4 linearly independent? Justify your answer. 1 5 25
Answer: The set of vectors are linearly independent.
32.
What is the rank of the set of vectors
1 1 1 r r r A = 1, B = 2, C = 3 . Justify your answer. 1 5 6
105
Answer: Since, the 3 vectors are linearly independent as proved above, the rank of the 3 vectors is 3.
33.
What is the rank of
1 2 3 r r r A = 1, B = 2, C = 3 . Justify your answer. 1 4 5 v r r Answer: By inspection, C = A + B . Hence, the 3 vectors are
linearly dependent, and the rank is less that 3. Linearly r r r r combination of A & B , i.e., K1 A + K 2 B = 0 has only one solution K1= K2 = 0. So the rank is 2.
106
Chapter 6 Gaussian Elimination
_________________________________
After reading this chapter, you will be able to
Solve a set of simultaneous linear equations using Nave Gauss Elimination Learn the pitfalls of Nave Gauss Elimination method Understand the effect of round off error on a solving set of linear equation by Nave Gauss Elimination Method Learn how to modify Nave Gauss Elimination method to Gaussian Elimination with Partial Pivoting Method to avoid pitfalls of the former method. Find the determinant of a square matrix using Guassian Elimination Understand the relationship between determinant of coefficient matrix and the solution of simultaneous linear equations.
_________________________________
How are a set of equations solved numerically?
One of the most popular techniques for solving simultaneous linear equations is the Gaussian elimination method. The approach is designed to solve a general set of n equations and n unknowns
a11 x1 + a12 x2 + a13 x3 + ... + a1n xn = b1 a21 x1 + a22 x2 + a23 x3 + ... + a2 n xn = b2
. . . . . .
an1 x1 + an 2 x2 + an 3 x3 + ... + ann xn = bn
Gaussian elimination consists of two steps
107
1.
Forward Elimination of Unknowns: In this step, the unknown is eliminated in each equation starting with the first equation. This way, the equations are "reduced" to one equation and one unknown in each equation. Back Substitution: unknowns is found. In this step, starting from the last equation, each of the
2.
Forward Elimination of Unknowns: In the first step of forward elimination, the first unknown, x1 is eliminated from all rows below the first row. The first equation is selected as the pivot equation to eliminate x1. So, to eliminate x1 in the second equation, one divides the first equation by a11 (hence called the pivot element) and then multiply it by a21. That is, same as multiplying the first equation by a21/ a11 to give
a21 x1 +
a a21 a a12 x2 + ... + 21 a1n xn = 21 b1 a11 a11 a11
Now, this equation can be subtracted from the second equation to give a a a a22 - 21 a12 x2 + ... + a2 n - 21 a1n xn = b2 - 21 b1 a11 a11 a11 or
' ' ' a22 x2 + ... + a2 n xn = b2
where
' a 22 = a 22 -
a 21 a12 a11 a 21 a1n a11
M
' a2n = a2n -
This procedure of eliminating x1 , is now repeated for the third equation to the nth equation to reduce the set of equations as
a11 x1 + a12 x2 + a13 x3 + ... + a1n xn = b1
' ' ' ' a22 x2 + a23 x3 + ... + a2 n xn = b2
' ' ' a32 x2 + a33 x3 + ... + a3n xn = b3'
. . .
. . .
. . .
108
' ' ' ' an 2 x2 + an 3 x3 + ... + ann xn = bn
This is the end of the first step of forward elimination. Now for the second step of forward elimination, we start with the second equation as the pivot equation and a ' 22 as the pivot element. So, to eliminate x2 in the third equation, one divides the second equation by a ' 22 (the pivot element) and then multiply it by a'32 . That is, same as multiplying the second equation by a'32 / a' 22 and subtracting from the third equation. This makes the coefficient of x2 zero in the third equation. The same procedure is now repeated for the fourth equation till the nth equation to give
a11 x1 + a12 x2 + a13 x3 + ... + a1n xn = b1
' ' ' ' a22 x2 + a23 x3 + ... + a2 n xn = b2 " " " a33 x3 + ... + a3n xn = b3
. . .
. . .
" " " an 3 x3 + ... + ann xn = bn
The next steps of forward elimination are conducted by using the third equation as a pivot equation and so on. That is, there will be a total of (n-1) steps of forward elimination. At the end of (n-1) steps of forward elimination, we get a set of equations that look like
a11 x1 + a12 x2 + a13 x3 + ... + a1n xn = b1
' ' ' ' a22 x2 + a23 x3 + ... + a2 n xn = b2 " " " a33 x3 + ... + an xn = b3
. . .
. . .
(n -1 ) (n ann-1) xn = bn
Back Substitution: Now the equations are solved starting from the last equation as it has only one unknown.
( bnn -1) x n = ( n -1) a nn
109
Then the second last equation, that is the (n-1)th equation, has two unknowns - xn and xn-1, but xn is already known. This reduces the (n-1)th equation also to one unknown. Back substitution hence can be represented for all equations by the formula
xi = and xn =
( bi(i -1) - aiji -1) x j ( aiii -1)
j =i +1
n
for i = n 1, n 2,..., 1
( bnn -1) (n a nn -1)
_________________________________
Example
The upward velocity of a rocket is given at three different times in the following table
Time, t
s 5 8 12
Velocity, v
m/s 106.8 177.2 279.2
The velocity data is approximated by a polynomial as
v(t ) = a1t 2 + a 2 t + a3 ,
by
5 t 12.
The coefficients a1 , a 2 , a3 for the above expression were found in Chapter 5 to be given
25 5 1 a 1 106.8 64 8 1 a = 177.2 2 144 12 1 a 3 279.2
Find the values of a1, a2, a3 using Nave Guass Elimination. t = 6, 7.5, 9, 11 seconds.
Find the velocity at
Solution
110
Forward Elimination of Unknowns: Since there are three equations, there will be two steps of forward elimination of unknowns. First step: Divide Row 1 by 25 and then multiply it by 64 Row1 25 (64) = [Row 1] 2.56 gives Row 1 as
[64
12.8 2.56]
[273.408]
Subtract the result from Row 2
5 1 a 1 106.81 25 0 - 4.8 - 1.56 a = - 96.21 2 144 12 1 a 3 279.2
Divide Row 1 by 25 and then multiply it by 144 Row 1 25 (144) = [Row 1] 5.76 gives Row 1 as
[144
28.8 5.76]
[615.2256]
Subtract the result from Row 3
5 1 a 1 106.8 25 0 - 4.8 - 1.56 a = - 96.21 2 0 - 16.8 - 4.76 a 3 - 336.0
Second step: We now divide Row 2 by 4.8 and then multiply by 16.8 Row2 - 4.8 (-16.8) = [Row 2] 3.5 gives Row 2 as
[0
- 16.8 - 5.46]
[- 336.735]
Subtract the result from Row 3
5 1 a 1 106.8 25 0 - 4.8 - 1.56 a = - 96.21 2 0 0 0.7 a 3 0.735
Back substitution: From the third equation
0.7 a 3 = 0.735
111
a3 =
0.735 0.7
= 1.050
Substituting the value of a3 in the second equation,
- 4.8a 2 - 1.56a3 = -96.21
a2 = - 96.21 + 1.56a 3 - 4 .8
=
- 96.21 + 1.56(1.050) - 4.8
= 19.70
Substituting the value of a 2 and a3 in the first equation,
25a1 + 5a 2 + a 3 = 106.8 a1 = 106.8 - 5a 2 - a3 25
=
106.8 - 5(19.70 ) - 1.050 25
= 0.2900
Hence the solution vector is
a1 0.2900 a = 19.70 2 a3 1.050
The polynomial that passes through the three data points is then
v(t ) = a1t 2 + a 2 t + a 3
= 0.2900t 2 + 19.70t + 1.050, 5 t 12 Since we want to find the velocity at t = 6, 7.5, 9 and 11 seconds, we could simply substitute each value of t in v(t ) = 0.2900t 2 + 19.70t + 1.050 and find the corresponding velocity. For example, at t = 6
v(6 ) = 0.2900(6 ) + 19.70(6 ) + 1.050 = 129.69 m / s.
2
112
However we could also find all the needed values of velocity at t = 6, 7.5, 9 and 11 seconds using matrix multiplication.
v(t ) = [0.29 19.7
t 2 1.05] t 1
So if we want to find v(6 ), v(7.5), v(9 ), v(11), it is given by
[v(6) v(7.5) v(9) v(11)]
6 2 = [0.29 19.7 1.05] 6 1
= [0.2900 = [129.7 19.70 165.1
7.5 2 7.5 1
9 2 112 9 11 1 1
36 56.25 81 121 1.050] 6 7.5 9 11 1 1 1 1 201.8 252.8]
v(6 ) = 129.7 m / s v(7.5) = 165.1 m / s v(9 ) = 201.8 m / s v(11) = 252.8 m / s
_________________________________
Example
Use Nave Gauss Elimination to solve
10 x1 - 7 x 2 = 7
- 3x1 + 2.099 x 2 + 6 x3 = 3.901 5 x1 - x2 + 5 x3 = 6 Use six significant digits with chopping in your calculations.
Solution
Working in the matrix form
113
- 7 0 x1 10 7 - 3 2.099 6 x = 3.901 2 5 6 - 1 5 x3
Forward Elimination of Unknowns Dividing Row 1 by 10 and multiplying by 3, that is, multiplying Row 1 by -0.3, and subtract it from Row 2 would eliminate a21,
-7 0 x1 7 10 0 - 0.001 6 x = 6.001 2 5 -1 5 x3 6
Again dividing Row 1 by 10 and multiplying by 5, that is, multiplying Row 1 by 0.5, and subtract it from Row 3 would eliminate a31,
-7 0 x1 7 10 0 - 0.001 6 x = 6.001 2 0 2 .5 5 x 3 2 .5
This is the end of the first step of forward elimination. Now for the second step of forward elimination, we would use Row 2 as the pivot equation and eliminate Row 3 Column 2. Dividing Row 2 by 0.001 and multiplying by 2.5, that is multiplying Row 2 by 2500, and subtracting from Row 3 gives
-7 0 10 0 - 0.001 6 0 0 15005 x1 x = 2 x3 7 6.001 15005
This is the end of the forward elimination steps. Back substitution We can now solve the above equations by back substitution. From the third equation, 15005 x3 = 15005
x3 =
15005 15005
= 1.
Substituting the value of x3 in the second equation - 0.001x 2 + 6 x3 = 6.001
114
x2 =
6.001 - 6 x3 - 0.001 6.001 - 6(1) = - 0.001 = = 6.001 - 6 - 0.001 0.001 - 0.001
= -1 Substituting the value of x3 and x2 in the first equation, 10 x1 - 7 x 2 + 0 x3 = 7
x1 = 7 + 7 x 2 - 0 x3 10
=
7 + 7(- 1) - 0(1) 10
=0
Hence the solution is
x1 0 [ X ] = x 2 = - 1 x3 1
_________________________________
Are there any pitfalls of Nave Gauss Elimination Method?
Yes, there are two pitfalls of Nave Gauss Elimination method. Division by zero: It is possible that division by zero may occur during forward elimination steps. For example for the set of equations 10 x 2 - 7 x3 = 7 6 x1 + 2.099 x 2 - 3x3 = 3.901 5 x1 - x 2 + 5 x3 = 6 during the first forward elimination step, the coefficient of x1 is zero and hence normalization would require division by zero.
115
Round-off error: Nave Gauss Elimination Method is prone to round-off errors. This is true when there are large numbers of equations as errors propagate. Also, if there is subtraction of numbers from each other, it may create large errors. See the example below.
_________________________________
Example
Remember the previous example where we used Nave Gauss Elimination to solve
10 x1 - 7 x 2 = 7
- 3x1 + 2.099 x 2 + 6 x3 = 3.901 5 x1 - x 2 + 5 x3 = 6 using six significant digits with chopping in your calculations. Repeat the problem, but now use five significant digits with chopping in your calculations.
Solution
Writing in the matrix form
- 7 0 x1 7 10 - 3 2.099 6 x = 3.901 2 5 - 1 5 x3 6
Forward Elimination of Unknowns Dividing Row 1 by 10 and multiplying by 3, that is, multiplying Row 1 by -0.3, and subtract it from Row 2 would eliminate a21,
-7 0 x1 7 10 0 - 0.001 6 x = 6.001 2 5 -1 5 x3 6
Again dividing Row 1 by 10 and multiplying by 5, that is, multiplying the Row 1 by 0.5, and subtract it from Row 3 would eliminate a31,
-7 0 x1 10 0 - 0.001 6 x 2 0 2 .5 5 x3 7 6.001 2. 5
=
This is the end of the first step of forward elimination.
116
Now for the second step of forward elimination, we would use Row 2 as the pivoting equation and eliminate Row 3 Column 2. Dividing Row 2 by 0.001 and multiplying by 2.5, that is, multiplying Row 2 by 2500, and subtract from Row 3 gives
-7 0 10 0 - 0.001 6 0 0 15005 x1 7 x = 6.001 2 x3 15004
This is the end of the forward elimination steps. Back substitution We can now solve the above equations by back substitution. From the third equation, 15005 x3 = 15004 x3 = 15004 15005
= 0.99993
Substituting the value of x3 in the second equation - 0.001x 2 + 6 x3 = 6.001 x2 = 6.001 - 6 x3 - 0.001 6.001 - 6(0.99993) = - 0.001 6.001 - 5.9995 = - 0.001 = 0.0015 - 0.001
= -1 . 5
Substituting the value of x3 and x2 in the first equation, 10 x1 - 7 x 2 + 0 x3 = 7
x1 = 7 + 7 x 2 - 0 x3 10
=
7 + 7(- 1.5) - 0(1) 10
117
= =
7 - 10.5 - 0 10 - 3.5 10
= - 0.3500 Hence the solution is
x1 [X ] = x 2 x3 - 0.35 = - 1.5 0.99993
Compare this with the exact solution of
x1 [X ] = x2 x3 0 = - 1 1
_________________________________
What are the techniques for improving Nave Gauss Elimination Method?
As seen in the example, round off errors were large when five significant digits were used as opposed to six significant digits. So one way of decreasing round off error would be to use more significant digits, that is, use double or quad precision. However, this would not avoid division by zero errors in Nave Gauss Elimination. To avoid division by zero as well as reduce (not eliminate) round off error, Gaussian Elimination with partial pivoting is the method of choice.
How does Gaussian elimination with partial pivoting differ from Nave Gauss elimination?
The two methods are the same, except in the beginning of each step of forward elimination, a row switching is done based on the following criterion. If there are n
118
equations, then there are (n - 1) forward elimination steps. At the beginning of the kth step of forward elimination, one finds the maximum of
a kk , a k +1,k , ............, a nk
Then if the maximum of these values is a pk in the pth row, kp n, then switch rows p and k. The other steps of forward elimination are the same as Nave Gauss elimination method. The back substitution steps stay exactly the same as Nave Gauss Elimination method.
_________________________________
Example
In the previous two examples, we used Nave Gauss Elimination to solve
10 x1 - 7 x 2 = 7
- 3x1 + 2.099 x 2 + 6 x3 = 3.901 5 x1 - x 2 + 5 x3 = 6 using five and six significant digits with chopping in the calculations. significant digits with chopping, the solution found was
x1 [X ] = x 2 x3 - 0.35 = - 1 .5 0.99993
Using five
This is different from the exact solution
x1 [X ] = x 2 x3 0 = - 1 1
119
Find the solution using Gaussian elimination with partial pivoting using five significant digits with chopping in your calculations.
Solution
- 7 0 x1 10 7 - 3 2.099 6 x = 2.901 2 5 6 - 1 5 x3
Forward Elimination of Unknowns Now for the first step of forward elimination, the absolute value of first column elements are
10 , - 3 , 5
or 10, 3, 5 So the largest absolute value is in the Row 1. So as per Gaussian Elimination with partial pivoting, the switch is between Row 1 and Row 1 to give
7 0 x1 10 7 - 3 2.099 6 x = 3.901 2 5 6 - 1 5 x3
Dividing Row 1 by 10 and multiplying by 3, that is, multiplying the Row 1 by -0.3, and subtract it from Row 2 would eliminate a21,
0 x1 7 -7 10 0 - 0.001 6 x = 6.001 2 5 5 x3 6 -1
Again dividing Row 1 by 10 and multiplying by 5, that is, multiplying the Row 1 by 0.5, and subtract it from Row 3 would eliminate a31,
0 x1 -7 10 0 - 0.001 6 x 2 0 2 .5 5 x3
7 6.001 2 .5
=
This is the end of the first step of forward elimination. Now for the second step of forward elimination, the absolute value of the second column elements below the Row 2 is
120
- 0.001 , 2.5
or 0.001, 2.5 So the largest absolute value is in Row 3. So the Row 2 is switched with the Row 3 to give
-7 0 x1 10 7 0 x = 2. 5 2 .5 5 2 0 - 0.001 6 x3 6.001
Dividing row 2 by 2.5 and multiplying by 0.001, that is multiplying by 0.001/2.5=0.0004, and then subtracting from Row 3 gives
0 10 - 7 0 2 .5 5 0 0 6.002
x1 7 x = 2 .5 2 x3 6.002
Back substitution 6.002 x3 = 6.002
x3 =
6.002 6.002
=1 Substituting the value of x3 in Row 2 2.5 x 2 + 5 x3 = 2.5
x2 =
2 .5 - 5 x 2 2 .5
= = =
2.5 - 5(1) 2.5 2.5 - 5 2.5 - 2.5 2.5
= -1 Substituting the value of x3 and x 2 in Row 1
121
10 x1 - 7 x 2 + 0 x 2 = 7
x1 =
7 + 7 x 2 - 0 x3 10
= = =
7 + 7(- 1) - 0(1) 10 7-7-0 10 0 10
=0
So the solution is
x1 0 [X ] = x2 = - 1 x3 1
This, in fact, is the exact solution. By coincidence only, in this case, the round off error is fully removed.
_________________________________
Can we use Nave Gauss Elimination methods to find the determinant of a square matrix?
One of the more efficient ways to find the determinant of a square matrix is by taking advantage of the following two theorems on a determinant of matrices coupled with Nave Gauss elimination.
Theorem 1: Let [A] be a nxn matrix. Then, if [B] is a matrix that results from adding or subtracting a multiple of one row to another row, then det (B) = det (A). (The same is true for column operations also).
Theorem 2: Let [A] be a nxn matrix that is upper triangular, lower triangular or diagonal, then det(A) = a11* a22*........ * ann = aii
i =1 n
This implies that if we apply the forward elimination steps of Naive Gauss Elimination method, the determinant of the matrix stays the same according the Theorem 1. Then since at the end of the forward elimination steps, the resulting matrix is upper triangular, the determinant will be given by Theorem 2.
122
_________________________________
Example
Find the determinant of
25 5 1 [A]= 64 8 1 144 12 1
Solution Remember earlier in this chapter, we conducted the steps of forward elimination of unknowns using Nave Gauss Elimination method on [A] to give
5 1 25 0 - 4.8 - 1.56 [B] = 0 0 0 .7
According to Theorem 2 det (A) = det (B)
= (25)(-4.8)(0.7) = -84.00
_________________________________
What if I cannot find the determinant of the matrix using Naive Gauss Elimination method, for example, if I get division by zero problems during Nave Gauss Elimination method? Well, you can apply Gaussian Elimination with partial pivoting. However, the determinant of the resulting upper triangular matrix may differ by a sign. The following theorem applies in addition to the previous two to find determinant of a square matrix.
Theorem 3: Let [A] be a nxn matrix. Then, if [B] is a matrix that results from switching one row with another row, then det (B) = - det (A).
_________________________________
Example
Find the determinant of
123
- 7 0 10 - 3 2.099 6 [A] = 5 - 1 5
Remember from that at the end of the forward elimination steps of Gaussian elimination with partial pivoting, we obtained
0 10 - 7 0 2 .5 [B] = 5 0 0 6.002 det (B ) = (10)(2.5)(6.002) = 150.05
Since rows were switched once during the forward elimination steps of Gaussian elimination with partial pivoting,
det (A ) = - det (B)
= -150.05 .
Prove det (A) = Proof
[A][A]-1 = [I]
1 . det A -1
( )
) = det (I ) det ( A) det (A ) = 1
det A A
-1 -1
(
det ( A) =
1 . det A -1
( )
If [A] is a nxn matrix and det (A) 0, what other statements are equivalent to it? 1. 2. 3. 4. 5. [A] is invertible. [A]-1 exists. [A] [X] = [C] has a unique solution. r [A] [X] = [0] solution is [X] = 0 . [A] [A]-1 = [I] = [A]-1 [A].
Key Terms
Nave Gauss Elimination
Partial Pivoting
Determinant
124
Homework
1.
The goal of forward elimination steps in Nave Gauss elimination method is to reduce the coefficient matrix A. to a diagonal matrix B. to an upper triangular matrix C. to a lower triangular matrix D. to an identity matrix
2. Using a computer with four significant digits with chopping, use Nave Gauss elimination to solve 0.0030 x1 + 55.23x 2 = 58.12 6.239 x1 - 7.123x 2 = 47.23
3. Using a computer with four significant digits with chopping, use Gaussian Elimination with partial pivoting to solve 0.0030 x1 + 55.23x 2 = 58.12 6.239 x1 - 7.123x 2 = 47.23 Answer:
4. Using a computer with four significant digits with chopping, use Nave Gauss elimination to solve 4 x1 + x 2 - x3 = -2 5 x1 + x 2 + 2 x3 = 4 6 x1 + x 2 + x3 = 6 Answer: (3, -13, 1)
125
Using a computer with four significant digits with chopping, use Gaussian Elimination with partial pivoting to solve 4 x1 + x 2 - x3 = -2 5 x1 + x 2 + 2 x3 = 4 6 x1 + x 2 + x3 = 6 Answer: (2.995, -12.98, 1.001)
5. For
- 7 0 10 - 3 2.099 6 [A] = 5 - 1 5
Find the determinant of [A] using forward elimination step of Nave Gauss Elimination method. Answer: -150.05
6.
One of the drawbacks of Naive-Gauss Elimination method is
Answer: Division by zero.
7.
Division by zero during forward elimination steps in Gaussian elimination with partial pivoting method implies the coefficient matrix [A] is Answer: It implies that the co-efficient matrix is singular, that is, its determinant is zero and it is not invertible.
8.
One of the advantages of Gaussian elimination with Partial Pivoting over Naive Guass Elimination method is Answer: It overcomes the problem of Division by zero.
126
9.
Show and explain clearly the first step of forward elimination of Nave Gauss elimination method for
1 2 3 a 6 4 6 7 b = 18 6 10 13 c 30
Answer: Dividing Row 1 by 1; multiplying it by 4 and subtracting the result from
Row 2
3 a 6 1 2 0 - 2 - 5 b = - 6 6 10 13 c 30
Dividing Row 1 by 1; multiplying it by 6 and subtracting the result from Row 3
3 a 6 1 2 0 - 2 - 5 b = - 6 0 - 2 - 5 c - 6 10.
Show and explain clearly the first step of forward elimination of Gaussian elimination method with partial pivoting for
1 2 3 a 6 4 6 7 b = 18 6 10 13 c 30
Answer: Comparing |a11|, |a21| and |a31| equals 1, 4, 6. Since 6 is greater, switch Row 3 and Row 1.
6 10 13 a 30 4 6 7 b = 18 1 2 3 c 6
Dividing Row 1 by 6; multiplying it by 4 and subtracting the result from Row 2
13 a 30 6 10 0 - 2 - 5 b = - 2 3 3 1 2 3 c 6
Dividing Row 1 by 6; multiplying it by 1 and subtracting the result from Row 3
6 10 0 - 2 3 0 1 3
13 a 30 - 5 b = - 2 3 5 c 1 6
127
11. At the end of forward elimination steps using Nave Gauss elimination method on the
coefficient matrix
5 1 25 c 1 25 64 a 1 , [A] reduces to [B] = 0 - 4.8 - 1.56 . [A]= 144 b 1 0 0 0.7
determinant of [A]? Answer: det(A) = det(B) = (25)(-4.8)(0.7) = -84
What is the
12.
At the end of Gauss Elimination steps on a set of three equations, I obtain the following system of equations. Now using a computer that uses only three significant digits with chopping, what is the value of unknowns using back substitution? Show all your intermediate work.
0 10 - 7 0 2.567 5 0 0 6.022
Answer:
x1 7 x = 2.5 2 x3 6.012
6.022 x3 = 6.012
x3 = 6.012 6.022 = 0.998
2.567 x2 + 5 x3 = 2.5 x2 = 10 x1 + (-7) x2 = 7 x1 = 7 + 7(-0.972) = 0.020 10 2.5 - 5(0.998) = -0.972 2.567
128
Chapter 7 LU Decomposition
_________________________________
After reading this chapter, you will be able to
Learn when LU Decomposition is numerically more efficient than Gaussian Elimination Decompose a nonsingular matrix into LU Show how LU decomposition is used to find matrix inverse.
_________________________________
I hear about LU Decomposition used as a method to solve a set of simultaneous linear equations? What is it and why do we need to learn different methods of solving a set of simultaneous linear equations?
We already studied two numerical methods of finding the solution to simultaneous linear equations Nave Gauss Elimination and Gaussian Elimination with Partial Pivoting. Then, why do we need to learn another method? To appreciate why LU Decomposition could be a better choice than the Gauss Elimination techniques in some cases, let us discuss first what LU Decomposition is about. For any nonsingular matrix [A] on which one can conduct Nave Gauss Elimination forward elimination steps, one can always write it as
[A] = [L][U ]
where [L] = lower triangular matrix [U] = upper triangular matrix Then if one is solving a set of equations [A] [X] = [C], then
129
[L][U ][X ] = [C ]
Multiplying both side by [L] ,
-1
([ A] = [L][U ])
[L]-1 [L][U ][X ] = [L]-1 [C ]
[I ][U ][X ] = [L]-1 [C ]
([L]
-1
[L] = [ I ])
[U ][X ] = [L]-1 [C ]
Let
([I ][U ] = [U ])
[L]-1 [C ] = [Z ]
then
[L][Z] = [C ]
and
(1)
[U ][X] = [Z ]
(2)
So we can solve equation (1) first for [ Z ] and then use equation (2) to calculate [X ] . This is all exciting but this looks more complicated than the Gaussian elimination techniques!! I know but I cannot tease you any longer. So here we go! Without proof, the computational time required to decompose the [A] matrix to [L] [U] form is proportional to
n3 , where n is the number of equations (size of [A] matrix). 3 n2 Then to solve the [L ][Z] = [C ] , the computational time is proportional to . Then to 2 n2 . So the total solve the [U ][X ] = [C ] , the computational time is proportional to 2 computational time to solve a set of equations by LU decomposition is proportional to n3 + n2 . 3
In comparison, Gaussian elimination is computationally more efficient. It takes a n3 n 2 computational time proportional to + , where the computational time for forward 3 2 n3 elimination is proportional to and for the back substitution the time is proportional 3 2 n to . 2
130
This has confused me further! Gaussian elimination takes less time than LU Decomposition method and you are trying to convince me then LU Decomposition has its place in solving linear equations! Yes, it does. Remember in trying to find the inverse of the matrix [A] in Chapter 5, the problem reduces to solving `n' sets of equations with the `n' columns of the identity matrix as the RHS vector. For calculations of each column of the inverse of the [A] matrix, the coefficient matrix [A] matrix in the set of equation [A][X ] = [C ] does not change. So if we use LU Decomposition method, the [A] = [L ][U ] decomposition needs to be done only once and the use of equations (1) and (2) still needs to be done `n' times. So the total computational time required to find the inverse of a matrix using LU 4n 3 n3 decomposition is proportional to + n(n 2 ) = . 3 3 In comparison, if Gaussian elimination method were applied to find the inverse of a matrix, the time would be proportional to
n3 n 2 n 4 n3 . n + = + 3 2 3 2
For large values of n
n 4 n 3 4n 3 + 3 2 3
Are you now convinced now that LU decomposition has its place in solving systems of equations? We are now ready to answer other questions - how do I find LU matrices for a nonsingular matrix [A] and how do I solve equations (1) and (2).
How do I decompose a non-singular matrix [A], that is, how do I find [A] = [L ][U ] ?
If forward elimination steps of Nave Gauss elimination methods can be applied on a nonsingular matrix, then [A] can be decomposed into L U as
a11 a [ A] = 21 M a n1 a12 a 22
M an2
K a1n L a2n L M L a nn
131
u 1 0 L 0 11 0 l M 21 1 = l 31 O 0 M M 1 l n1 L L 0
u 12 L u 1n O O 0 L u nn 0
1. The elements of the [U ] matrix are exactly the same as the coefficient matrix one obtains at the end of the forward elimination steps in Nave Gauss Elimination. 2. The lower triangular matrix [L] has 1 in its diagonal entries. The non zero elements on the non-diagonal elements in [L] are multipliers that made the corresponding entries zero in the upper triangular matrix [U ] during forward elimination. Let us look at this using the same example as used in Nave Gaussian elimination.
_________________________________
Example
Find the LU decomposition of the matrix
25 5 1 [A] = 64 8 1 144 12 1
Solution
1 [A] = [L][U ] = l 21 l 31
0 1
l 32
0 u11 0 0 1 0
u12 u 22
0
u13 u 23 u 33
The [U ] matrix is the same as found at the end of the forward elimination of Nave Gauss elimination method, that is
5 1 25 0 - 4.8 - 1.56 [U ] = 0 0 0 .7
To find l 21 and l 31 , what multiplier was used to make the a 21 and a 31 elements zero in the first step of forward elimination of Nave Gauss Elimination Method It was
132
l 21 =
64 25 144 25
= 2.56
l 31 =
= 5.76 To find l 32 , what multiplier was used to make a 32 element zero. Remember a 32 element was made zero in the second step of forward elimination. The [A] matrix at the beginning of the second step of forward elimination was
5 1 25 0 - 4.8 - 1.56 0 - 16.8 - 4.76
So
l 32 =
- 16.8 - 4.8
= 3.5 Hence
0 0 1 2.56 1 0 [L] = 5.76 3.5 1
Confirm
[L][U ] = [A].
5 1 0 0 25 1 2.56 1 0 0 - 4.8 - 1.56 [L][U ] = 5.76 3.5 1 0 0 0 .7
25 5 1 = 64 8 1 144 12 1
________________________________
133
Example
Use LU decomposition method to solve the following simultaneous linear equations.
25 5 1 a 1 106.8 64 8 1 a = 177.2 2 144 12 1 a 3 279.2
Solution
Recall that
[A][X ] = [C ]
and if
[A] = [L][U ]
then first solving
[L][Z ] = [C ]
and then
[U ][X ] = [Z ]
gives the solution vector [X ] . Now in the previous example, we showed
5 1 0 0 25 1 2.56 1 0 0 - 4.8 - 1.56 [A] = [L][U ] = 5.76 3.5 1 0 0 0 .7
First solve
[L][Z ] = [C ]
0 0 z1 106.8 1 2.56 1 0 z = 177.2 2 5.76 3.5 1 z 3 279.2
to give z1 = 106.8 2.56z1 +z2 = 177.2 5.76 z1 + 3.5 z2 + z3 = 279.2
134
Forward substitution starting from the first equation gives z1 =106.8 z2 = 177.2 2.56 z1 = 177.2 2.56 (106.8) = -96.21 z3 = 279.2 5.76 z1 3.5z2 = 279.2 5.76(106.8) 3.5(-96.21) = 0.735 Hence
z1 [Z ] = z 2 z3 106.8 = - 96.21 0.735
This matrix is same as the right hand side obtained at the end of the forward elimination steps of Nave Gauss elimination method. Is this a coincidence? Now solve
[U ][X ] = [Z ]
5 1 a1 106.8 25 0 - 4.8 - 1.56 a = - 96.21 2 0 0 0.7 a 3 0.735 25a1 + 5a 2 + a 3 = 106.8
- 4.8a 2 - 1.56a3 = -96.21
0.7 a 3 = 0.735
From the third equation
0.7 a 3 = 0.735
a3 =
0.735 0.7
135
= 1.050
Substituting the value of a3 in the second equation,
- 4.8a 2 - 1.56a3 = -96.21 a2 = - 96.21 + 1.56a 3 - 4 .8
=
- 96.21 + 1.56(1.050) - 4.8
= 19.70
Substituting the value of a 2 and a3 in the first equation,
25a1 + 5a 2 + a 3 = 106.8
a1 =
106.8 - 5a 2 - a3 25
=
106.8 - 5(19.70 ) - 1.050 25
= 0.2900
The solution vector is
a1 0.2900 a = 19.70 2 a3 1.050
_________________________________
How do I find the inverse of a square matrix using LU Decomposition?
A matrix [B ] is the inverse of [A] if [ A][B ] = [I ] = [B ][ A]. How can we use LU decomposition to find inverse of the matrix? Assume the first column of [B ] (the inverse of [A] is [b 11 b 12 .............b n1 ] T then from the above definition of inverse and definition of matrix multiplication.
136
b11 1 b 0 [A] 21 = M M bn1 0
Similarly the second column of [B ] is given by
b12 0 b 1 [A] 22 = M M bn 2 0
Similarly, all columns of [B ] can be found by solving n different sets of equations with the column of the right hand sides being the n columns of the identity matrix.
_________________________________
Example
Use LU decomposition to find the inverse of
25 5 1 [A] = 64 8 1 144 12 1
Solution
Knowing that
0 0 25 5 1 1 2.56 1 0 0 - 4.8 - 1.56 [A] = [L][U ] = 5.76 3.5 1 0 0 0.7
We can solve for the first column of [B]= [A] by solving for
-1
25 5 1 b11 1 64 8 1 b = 0 21 144 12 1 b31 0
First solve
[L][Z ] = [C ] , that is
137
0 0 z1 1 1 2.56 1 0 z = 0 2 5.76 3.5 1 z 3 0
to give
z1 = 1
2.56 z1 + z 2 = 0
5.76 z1 + 3.5 z 2 + z 3 = 0 Forward substitution starting from the first equation gives
z1 = 1 z 2 = 0 - 2.56z1
= 0 - 2.56(1) = -2.56
z 3 = 0 - 5.76 z1 - 3.5 z 2
= 0 - 5.76(1) - 3.5(- 2.56 ) = 3 .2
Hence
z1 [Z ] = z 2 z3 1 = - 2.56 3 .2
Now solve
[U ][X ] = [Z ]
that is
138
5 1 b11 1 25 0 - 4.8 - 1.56 b = - 2.56 21 0 0 0.7 b31 3.2
25b11 + 5b21 + b31 = 1 - 4.8b21 - 1.56b31 = -2.56 0.7b31 = 3.2 Backward substitution starting from the third equation gives
b31 =
3.2 0.7
- 2.56 + 1.560b31 - 4 .8
= 4.571
b21 =
=
- 2.56 + 1.560(4.571) - 4.8
1 - 5b21 - b31 25
= -0.9524
b11 =
=
1 - 5(-0.9524) - 4.571 25
= 0.04762 Hence the first column of the inverse of [A] is
b11 0.04762 b = - 0.9524 21 b31 4.571
Similarly by solving
25 5 1 b12 0 64 8 1 b = 1 22 144 12 1 b32 0
139
b12 - 0.08333 b = 1.417 22 b32 - 5.000
and solving
25 5 1 b13 0 64 8 1 b = 0 23 144 12 1 b 33 1
gives
b13 0.03571 b = - 0.4643 23 b33 1.429
Hence
[A]
-1
0.4762 0.08333 0.0357 = - 0.9524 1.417 - 0.4643 4.571 1.429 - 5.050
-1 -1
Can you confirm the following for the above example? [A][ A] = [I ] = [A]
[A]
_________________________________
Key Terms
LU decomposition Inverse
_________________________________
Homework
1. Show that LU decomposition is computationally more efficient way of finding the inverse of a square matrix than using Gaussian elimination.
2. LU decomposition method is computationally
140
more efficient than Nave Gauss elimination for A. Solving a single set of simultaneous linear equations B. Solving multiple sets of simultaneous linear equations with different coefficient matrices. C. Solving multiple sets of simultaneous linear equations with same coefficient matrix but different right hand sides. D. Solving less than ten simultaneous linear equations.
3. It one decomposes a symmetric matrix [A] to a LU form, then A. [L] = [U]T B. [U]T = [L]T A. [L] = [U] D. [L] = [I]
4. Use LU decomposition to solve 4 x1 + x 2 - x3 = -2 5 x1 + x 2 + 2 x3 = 4 6 x1 + x 2 + x3 = 6
Answer: (3, -13, 1)
5. Find the inverse of
141
1 3 4 [A] = 2 - 7 - 1 8 1 5 using LU decomposition
Answer: 2.931x10-1 1.638x10-1 [A]-1 = 1.552x10-1 - 6.034x10- 2 - 5.000x10-1 - 2.500x10-1
- 2.586x10-2 - 4.310x10- 2 2.500x10-1
6. 0 2 Show that the nonsingular matrix [A] = 2 0 cannot be decomposed into LU form.
Hint: Try to find the unknowns in 1 0 u11 u12 0 2 l = 21 1 0 u22 2 0 Do you see any inconsistencies? Understand 0 2 that is nonsingular. 2 0
7. For large values of n for a n n square matrix [A], the computational time taken to find the inverse of [A] is n4 n3 proportional to + if found by using Nave 3 2 Gaussian Elimination Method and is proportional to 4n 3 if found by using [L][U] decomposition method. 3 If it takes 15 seconds to find the inverse of a 2500 2500 matrix by using [L][U] decomposition method, estimate the time it would take to find the inverse of a 2500 2500 matrix by using Gaussian Elimination Method.
142
143
Chapter 8 Gauss-Siedel Method _________________________
After reading this chapter, you will be able to
Solve a set of equations using Gauss-Siedel method Learn the advantages and pitfalls of Gauss-Siedel method Understand under what conditions Gauss-Siedel method always converges
_________________________________
Why do we need another method to solve a set of simultaneous linear equations?
In certain cases, such as when a system of equations is large, iterative methods of solving equations such as Gauss-Siedel method are more advantageous. Elimination methods such as Gaussan elimination, are prone to round off errors for a large set of equations. Iterative methods, such as Guass-Siedel method, allow the user the control of the roundoff error. Also if the physics of the problem are well known for faster convergence, initial guesses needed in iterative methods can be made more judiciously. You convinced me, so what is the algorithm for Gauss-Siedel method? Given a general set of n equations and n unknowns, we have
a11 x1 + a12 x 2 + a13 x3 + ... + a1n x n = c1 a 21 x1 + a 22 x 2 + a 23 x3 + ... + a 2 n x n = c 2
. . . . . .
a n1 x1 + a n 2 x 2 + a n 3 x3 + ... + a nn x n = c n
If the diagonal elements are non-zero, each equation is rewritten for the corresponding unknown, that is, the first equation is rewritten with x1 on the left hand side, second equation is rewritten with x 2 on the left hand side and so on as follows
144
x1 =
c1 - a12 x 2 - a13 x3 KK - a1n x n a11
x2 = M M
c 2 - a 21 x1 - a 23 x3 KK - a 2 n x n a 22
x n -1 = xn =
c n -1 - a n -1,1 x1 - a n -1, 2 x 2 KK - a n -1,n - 2 x n - 2 - a n -1,n x n a n -1,n -1 a nn
c n - a n1 x1 - a n 2 x 2 - KK - a n ,n -1 x n -1
These equations can be rewritten in the summation form as
c1 - a1 j x j x1 =
j =1 j 1 n
a11 c2 - a2 j x j
n
x2 =
j =1 j 2
a 22
. . .
c n -1 - x n -1 =
n n
j =1 j n -1
a
n -1, j
xj
a n -1,n -1
c n - a nj x j xn =
j =1 j n
a nn
Hence for any row `i',
145
ci - aij x j xi =
j =1 j i
n
aii
, i = 1,2,K , n.
Now to find xi's, one assumes an initial guess for the xi's and then use the rewritten equations to calculate the new guesses. Remember, one always uses the most recent guesses to calculate xi. At the end of each iteration, one calculates the absolute relative approximate error for each xi as
a i =
xinew - xiold x100 xinew
where xinew is the recently obtained value of xi, and xiold is the previous value of xi. When the absolute relative approximate error for each xi is less than the prespecified tolerance, the iterations are stopped.
___________________________
Example
The upward velocity of a rocket is given at three different times in the following table
Time, t Velocity, v
S 5 8 12
m/s 106.8 177.2 279.2
The velocity data is approximated by a polynomial as
v(t ) = a1t 2 + a 2 t + a3 ,
5 t 12.
The coefficients a1, a2, a3 for the above expression were found in Chapter 5 to be given by
25 5 1 a 1 106.8 64 8 1 a = 177.2 2 144 12 1 a 3 279.2
146
Find the values of a1, a2, a3 using Guass-Seidal Method. Assume an initial guess of the solution as
[a
1
a2
a 3 ] = [1 2 5] .
Solution
Rewriting the equations gives
a1 = a2 = a3 =
106.8 - 5a 2 - a3 25 177.2 - 64a1 - a3 8 279.2 - 144a1 - 12a 2 1
Iteration #1
Given the initial guess of the solution vector as
a1 1 a = 2 2 a 3 5
we get
a1 =
106.8 - 5(2) - (5) 25 177.2 - 64(3.6720 ) - (5) 8 279.2 - 144(3.6720 ) - 12(- 7.8510 ) 1
= 3.6720
a2 =
= -7.8510
a3 =
= -155.36 The absolute relative approximate error for each xi then is 3.6720 - 1.0000 x100 3.6720
a 1 =
=72.76%
147
a 2 =
- 7.8510 - 2.0000 x100 - 7.8510
=125.47%
a 3 =
- 155.36 - 5.0000 x100 - 155.36
= 103.22% At the end of the first iteration, the guess of the solution vector is
a1 3.6720 a = - 7.8510 2 a3 - 155.36
and the maximum absolute relative approximate error is 125.47%.
Iteration #2
The estimate of the solution vector at the end of iteration #1 is
a1 3.6720 a = - 7.8510 2 a3 - 155.36
Now we get
a1 =
106.8 - 5(- 7.8510 ) - 155.36 25 177.2 - 64(12.056) - 155.36 8 279.2 - 144(12.056) - 12(- 54.882) 1
= 12.056
a2 =
= -54.882
a3 =
=-798.34 The absolute relative approximate error for each xi then is
a 1 =
12.056 - 3.6720 x100 12.056
= 69.542%
148
a
2
=
- 54.882 - (- 7.8510) x100 - 54.882 - 798.34 - (- 155.36 ) x100 - 798.34
= 85.695%
a
3
=
= 80.54%. At the end of second iteration the estimate of the solution is
a1 12.056 a = - 54.882 2 a3 - 798.34
and the maximum absolute relative approximate error is 85.695%. Conducting more iterations gives the following values for the solution vector and the corresponding absolute relative approximate errors. Iteration 1 2 3 4 5 6 a1 3.672 12.056 47.182 193.33 800.53 3322.6
a 1 %
a2 -7.8510 -54.882 -255.51 -1093.4 -4577.2 -19049
a 2 %
a3 -155.36 -798.34 -3448.9 -14440 -60072 -249580
a 3 %
72.767 67.542 74.448 75.595 75.850 75.907
125.47 85.695 78.521 76.632 76.112 75.971
103.22 80.540 76.852 76.116 75.962 75.931
As seen in the above table, the solution is not converging to the true solution of a1 a2 a3 = 0.29048 = 19.690 = 1.0858
___________________________
The above system of equations does not seem to converge? Why? Well, a pitfall of most iterative methods is that they may or may not converge. However, certain class of systems of simultaneous equations do always converge to a solution using
149
Gauss-Seidal method. This class of system of equations is where the coefficient matrix [A] in [A][X] = [C] is diagonally dominant, that is
aii aij
j =1 j i n n
for all `i'
and aii aij for at least one `i'.
j =1 j i
If a system of equations has a coefficient matrix that is not diagonally dominant, it may or may not converge. Fortunately, many physical systems that result in simultaneous linear equations have diagonally dominant coefficient matrices, which then assures convergence for iterative methods such as Gauss-Seidal method of solving simultaneous linear equations.
___________________________
Example
Given the system of equations. 12 x1 + 3x 2 - 5 x3 = 1
x1 + 5 x 2 + 3x3 = 28
3x1 + 7 x 2 + 13x3 = 76 find the solution. Given
x1 1 x = 0 2 x 3 1
as the initial guess.
Solution
The coefficient matrix
12 3 - 5 [A] = 1 5 3 3 7 13
is diagonally dominant as
150
a11 = 12 = 12 a12 + a13 = 3 + - 5 = 8 a 22 = 5 = 5 a 21 + a 23 = 1 + 3 = 4 a 33 = 13 = 13 a 31 + a32 = 3 + 7 = 10
and the inequality is strictly greater than for at least one row. Hence the solution should converge using Gauss-Seidal method. Rewriting the equations, we get
x1 = x2 = x3 =
1 - 3 x 2 + 5 x3 12 28 - x1 - 3x3 5 76 - 3x1 - 7 x 2 13
Assuming an initial guess of
x1 1 x = 0 2 x 3 1
Iteration 1: 1 - 3(0) + 5(1) 12 0.50000 28 - (0.5) - 3(1) 5 4.9000 76 - 3(0.50000 ) - 7(4.9000) 13 3.0923 0.50000 - 1.0000 x100 0.50000
x1 =
=
x2 =
=
x3 =
=
The absolute relative approximate error at the end of first iteration is
a 1 =
151
= 67.662%
a
2
=
4.9000 - 0 x100 4.9000
= 100.000%
a
3
=
3.0923 - 1.0000 x100 3.0923
= 67.662% The maximum absolute relative approximate error is 100.000% Iteration 2:
x1 =
1 - 3(4.9000) + 5(3.0923) 12 0.14679 28 - (0.14679) - 3(3.0923) 5 3.7153 76 - 3(0.14679) - 7(4.900) 13 3.8118 0.14679 - 0.50000 100 0.14679
=
x2 =
=
x3 =
=
At the end of second iteration, the absolute relative approximate error is
a 1 =
= 240.62%
a
2
=
3.7153 - 4.9000 100 3.7153
= 31.887%
a
3
=
3.8118 - 3.0923 100 3.8118
= 18.876%.
152
The maximum absolute relative approximate error is 240.62%. This is greater than the value of 67.612% we obtained in the first iteration. Is the solution diverging? No, as you conduct more iterations, the solution converges as follows. Iteration 1 2 3 4 5 6 a1 0.50000 0.14679 0.74275 0.94675 0.99177 0.99919
a 1
67.662 240.62 80.23 21.547 4.5394 0.74260
a2 4.900 3.7153 3.1644 3.0281 3.0034 3.0001
a
2
a3 3.0923 3.8118 3.9708 3.9971 4.0001 4.0001
a
3
100.00 31.887 17.409 4.5012 0.82240 0.11000
67.662 18.876 4.0042 0.65798 0.07499 0.00000
This is close to the exact solution vector of
x1 1 x = 3 2 x3 4
___________________________
Example
Given the system of equations
3x 1 + 7x 2 + 13x 3 = 76 x 1 + 5x 2 + 3x 3 = 28 12x 1 + 3x 2 - 5x 3 = 1
find the solution using Gauss-Seidal method. Use [x1 , x 2 , x3 ] = [1 0 1] as the initial guess.
Solution
Rewriting the equations, we get
x1 = x2 =
76 - 7 x 2 - 13 x3 3 28 - x1 - 3x3 5
153
x3 =
1 - 12 x1 - 3x 2 -5
Assuming an initial guess of
x1 1 x = 0 2 x 3 1
the next six iterative values are given in the table below Iteration 1 2 3 4 5 6 a1 21.000 -196.15 -1995.0 -20149 2.0364x105 -2.0579x105
a
1
a2 0.80000 14.421 -116.02 1204.6 -12140 1.2272x105
a
2
a3 5.0680 -462.30 4718.1 -47636 4.8144x105 -4.8653x106
a
3
110.71 109.83 109.90 109.89 109.90 1.0990
100.00 94.453 112.43 109.63 109.92 109.89
98.027 110.96 109.80 109.90 109.89 109.89
You can see that this solution is not converging and the coefficient matrix is not diagonally dominant. The coefficient matrix
3 7 13 [A] = 1 5 3 12 3 - 5
is not diagonally dominant as
a11 = 3 = 3 a12 + a13 = 7 + 13 = 20
Hence Gauss-Seidal metho...