Numerical Linear Algebra, CME 302
E. Darve
Autumn 2015
CME 302: Homework 1
Due Wednesday, October 7th, 2015
Problem 1: (Givens Rotation on Sparse Matrices)
20 points. Let a matrix A Rnn be of the form
1 2 3 . . . n
2 2
3
A = 3
.
.
.
.
.
n
n
Please desi

Numerical Linear Algebra, CME 302
E. Darve
Autumn 2014
CME 302: Homework 2
Due Wednesday, October 8th, 2014
Problem 1: (Givens Rotation on Sparse Matrices)
20 points. Let a matrix A Rnn be of the form
1 2 3 . . . n
2 2
3
A = 3
.
.
.
.
.
n
n
Please desi

Numerical Linear Algebra, CME 302
E. Darve
Autumn 2015
CME 302 Schedule
GVL: Golub, Van Loan
TB: Trefethen, Bau
D: Davis
H: homework
Date
Topic
Reading
Week 1 Sep 2125
Introduction; range and null space, rank; vector and matrix norms; singular
value decom

Numerical Linear Algebra, CME 302
E. Darve
Autumn 2014
CME 302: Homework 1
Due Wednesday, October 1st, 2014
Problem 1: (Norm Equivalencies)
In a nite dimensional space, all norms are equivalent. In this problem, you will be asked to verify
this theorem fo

CME 302: NUMERICAL LINEAR ALGEBRA
FALL 2005/06
ASSIGNMENT 3 SOLUTIONS
GENE H. GOLUB
1. (a) Let a and b be vectors such that a 2 = b 2 = 1. Construct a Householder matrix
H = I 2uu , u 2 = 1 such that H a = b.
Solution: We want
(I 2uu )a = b.
In other word

Numerical Linear Algebra, CME 302
E. Darve
Autumn 2014
CME 302: Homework 3
Due Wednesday, October 15, 2014
Problem 1: (Floating Point)
For this problem, make sure to review the denition of oating point numbers that we saw in
class. You will be asked to de

CME302 Homework 7 (1) Problem 33.2 from T.B. (2) Problem 36.1 from T.B. (3) We have seen the use of shifts in inverse iterations in Lecture 27. The section `Connection with Shifted Inverse Iteration' on pg.220 of T.B. claims that the QR algorithm which se

CME302 Homework 6 (1) Problem 28.2 (a) (b) (c) (2) This problem is based on Problem 29.1 from Trefethen. We would like you to put together a MATLAB program that finds all the eigenvalues of a real symmetric matrix, using only elementary building blocks. U

CME 302: Homework 4 Solutions
Problem 1.
1. The code simply builds a diagonal matrix with the given eigenvalues and then performs a
similatrity transform.
Matlab code:
n=8;
A=diag ( 3 . ( 0 : ( n 1 ) ) ) ;
B=randn ( n ) ;
A=B\ (AB ) ;
Python code:
import

14
14.1
Lecture 14
Krylov Subspace Method for Solving Ax = b
We are using iteration to generate a series of x(k) k (A, b), s.t.
x(k) x
(167)
There are basically four motivations:
We can construct a nice set of basis (using Lanczos iteration)
Each iterat

13
13.1
Lecture 13
Classical Iteration for Solving Ax = b
In solving Ax = b, iterative method may be better when:
A is sparse;
Ax can be computed quickly
13.1.1
Splitting method
If we have A = M N where M is non-singular,
Mx = Nx + b
(153)
x = M1 Nx +

16
16.1
16.1.1
Lecture 16
MINRES/GMRES
MINRES
Run Lanczos for A with v1 =
b
b
At each iteration solve
min rk
xk
2
min b AVk y
y
2
Let x(k) = Vk y
Even though, we need to x cost for each iteration (with respect to k). In fact, we can go from x(k) to
x(

CME 302, Fall 2015
Homework 1
Problem 1.
Solution:
To simplify notation, we denote Gij as the Givens rotation between the i-th and the j-th row of A. We get
the following algorithm.
.
. .
.
.
.
.
. .
.
G
.
n1,n .
0
+ +
+
+ +
.
. .
.
G

12
12.1
Lecture 12
Arnoldi iteration: Two Perspective
There are two perspective to understand Arnoldi process intuitively.
12.1.1
QR of Krylov matrix
As mentioned, Arnoldi iteration can be viewed as construing orthogonal basis of Krylov subspace. Specific

11
11.1
Lecture 11
Iteration Algorithm for Sparse Matrix
Sometimes, e.g. when we tries to use matrix to simulate continuous problems, we encounter matrices of
huge dimensions. But the good thing is, they are sparse; that is to say there are bunch of zeros

CME 302: NUMERICAL LINEAR ALGEBRA
FALL 2005/06
LECTURE 3
GENE H. GOLUB
1. Singular Value Decomposition
Suppose A is an m n real matrix with m n. Then we can write
A = U V ,
where
U U = Im ,
V V = In ,
1
=
.
.
.
n
0
The diagonal elements i , i = 1, . . .

Numerical Linear Algebra, CME 302
E. Darve
Autumn 2015
CME 302: Homework 2
Due Wednesday, October 14, 2015
Problem 1: (Floating Point)
For this problem, make sure to review the denition of oating point numbers that we saw in
class. You will be asked to de

CME 302: Homework 3
Problem 1
Solution: Let y = Xx, then
xT X T AXx = y t Ay 0.
So we just need to show that equality only holds if x = 0. Since X is rank r we know that
the kernel of X is just the zero vector. Since y T Ay = 0 if and only if y = 0 and we

Numerical Linear Algebra, CME 302
E. Darve
Autumn 2014
CME 302: Homework 6
Due Wednesday, November 12, 2014
Problem 1: (Convergence rate of shifted QR algorithm)
In this problem we consider the convergence rate of the QR algorithm with a single-shift stra

1
Lecture 1
1.1
Matrix
Matrix is a linear Map, and its a linear system:
A(x + y) = Ax + Ay
(1)
A(x) = Ax
1.2
(2)
Rank-nullity Theorem
Dene Null(A)
cfw_x|Ax = 0
n = rank(A) + dim(Null(A)
(3)
or could be write as:
rank(T) + dim(Null(T) = dim(V ),
1.3
T : V

Numerical Linear Algebra, CME 302
E. Darve
Autumn 2014
CME 302: Homework 5
Due Wednesday, November 5, 2014
Problem 1: (Convergence of the orthogonal iteration algorithm)
1. 5 points. Write some Python or Matlab code to create a matrix in Rnn of size n = 8

Numerical Linear Algebra, CME 302
E. Darve
Autumn 2014
CME 302: Homework 4
Due Wednesday, October 22, 2014
Problem 1: (Positive denite matrices)
10 points. Show that for any matrix A Rnn and X Rnr , if A is positive denite and X is
rank r, then X T AX is

15
15.1
15.1.1
Lecture 15
Conjugate Gradient
Termination
We can look into the residual,
rk = b Ax(k)
(182)
But it involves a matrix-vector multiplication. Instead, we can just compare the coecients of x(k) on
Vk , i.e. x(k) = Vk y(k) :
rk = b AVk y(k)
= V

17
Lecture 17
17.1
Sparse Matrix Data Structure
17.1.1
Coordinate format (matrix market format)
This format stores entries in triplet (i[], j[], v[]).
17.1.2
Compressed sparse column format(CSC) and (CSR)
Now we requires the entries lies column by column.

CME 302, Fall 2015
Assignment 2
Problem 1.
Solution.
d
1. First, observe that we may write a oating point number as t e with t xed, t1 d t 1, and
L e U. Now, WLOG we may consider x to be positive and e = 1 because for a xed e the space
between all the oat

10
10.1
10.1.1
Lecture 10
QR Iteration
Hessenberg form
Its impossible to use nite Householder to diagonalize the matrix A. However, we can still apply n 2
Householder transformations to change it into an upper triangular matrix (and for symmetric matrix,

7
7.1
Lecture 7
LU Factorization with Partial Pivoting
We want to get a LU factorization where the order of L, U is not larger than A signicantly. Since,
lik =
aik
akk
(86)
, rst we need to nd the row imax , s.t. aimax ,k is the largest in the column k. T

8
Lecture 8
8.1
Cholesky Factorization
We denotes
A=
a
c
c
B
=
c
1
0
0
I
0
B cc
a
0
c
I
= L1 A1 L1
(98)
Then,
A1 = L1 AL
1
1
(99)
x A1 x = (L x) A(L x) > 0
1
1
(100)
x R,
Thus, A1 is s.p.d., and B
8.1.1
cc
a
is also s.p.d.
Sensitivity and Accuracy
Suppos

9
Lecture 9
9.1
Orthogonal Iteration
yk : q0 Ak q0 x1
(112)
zk : q1 Ak q1 x2
(113)
Like the power iteration, the problem here is how can we force the second vector converge to the second
eigenvector. The approach is, while in the iteration, forcing the se

CS265/CME309, Problem Set [ 1 ]
SUNet IDs: kailaix jinnyxie
Names: Kailai Xu Jin Xie
ID Numbers: 06118115 06118109
By turning in this assignment, I agree by the Stanford honor code and declare that all
of this is the work of my partner and I.
Problem 1: R

Chapter 1
Introduction
1.1
Systems of Linear Equations
The main content of the course will be concerned with solving problems of
the form
Ax = b
where A is an m n matrix of rank r. The most common sources of such
problems are:
Solution of problems arisin

Numerical Linear Algebra
Draft
Eric Darve
Mary Wootters
Stanford University
October 3, 2016
4
Chapter 2
Linear Systems
1
LU factorization
Solving linear systems Ax = b is one of the most important topics in linear algebra.
This problem is at the core of t

CME 302: NUMERICAL LINEAR ALGEBRA
FALL 2005/06
LECTURE 0
GENE H. GOLUB
1. What is Numerical Analysis?
In the 1973 edition of the Websters New Collegiate Dictionary, numerical analysis is defined to
be the study of quantitative approximations to the soluti

1
Review of basic linear algebra facts
Eric Darve, September 26, 2016
This is a short review of important algebra facts to keep in mind. We list the main results with minimal
discussion. For brevity, proofs are omitted.
1.1
Range and null space of a matri