D
EFINITION
For
u
and
v
in
R
n
, the
distance between u and v
, written as dist(
u
,
v
), is the
length of the vector
u

v
. That is,
dist(
u
,
v
)
= 
u

v

D
EFINITION
Two vectors
u
and
v
in
R
n
are
orthogonal
to each other if
u
·
v
=
0.
T
HEOREM
2: T
HE
P
YTHAGOREAN
T
HEOREM
Two vectors
u
and
v
are orthogonal if and only
if

u
+
v

2
= 
u

2
+
v

2
T
HEOREM
3
Let
A
be an mxn matrix. The orthogonal complement of the row space of
A
is the null space of
A
, and the orthogonal complement of the column space of
A
is the null
space of
A
T
:
(Row
A
)
⊥
=
Nul (
A
) and (Col
A
)
⊥
=
Nul (
A
T
)
26
6.2 O
RTHOGONAL
S
ETS
D
EFINITION
A set of vectors
u
1
,...,
u
p
in
R
n
is said to be an
orthogonal set
if each pair of
distinct vectors from the set is orthogonal, that is, if
u
i
·
u
j
=
0 whenever
i
6=
j
T
HEOREM
4
If
S
=
u
1
,...,
u
p
is an orthogonal set of nonzero vectors in
R
n
, then
S
is linearly
independent and hence is a basis for the subspace spanned by
S
D
EFINITION
An
orthogonal basis
for a subspace
W
of
R
n
is a basis for
W
that is also an
orthogonal set.
T
HEOREM
5
Let
u
1
,...,
u
p
be an orthogonal basis for a subspace
W
of
R
n
. For each
y
in
W
,
the weights in the linear combination
y
=
c
1
u
1
+
...
+
c
p
u
p
are given by
c
j
=
y
·
u
j
u
j
·
u
j
D
EFINITION
A set
u
1
,...,
u
p
is an
orthonormal set
if it is an orthogonal set of unit vectors. If
W
is the subspace spanned by such a set, then
u
1
,...,
u
p
is an
orthonormal basis
for
W
, since
the set is automatically independent.
T
HEOREM
6
An
m
*
n
matrix
U
has orthonormal columns
⇐⇒
U
T
U
=
I
.
T
HEOREM
7
Let
U
be an
m
*
n
matrix with orthonormal columns, and let
x
and
y
be in
R
n
.
Then,
1.

Ux
 = 
x

2. (
Ux
)
·
(
U y
)
=
x
·
y
3. (
Ux
)
·
(
U y
)
=
0 if and only if
x
·
y
=
0
D
EFINITION
An
orthogonal matrix
U
is a useful SQUARE matrix that has the property
U
T
=
U

1
. By definition, the columns AND rows are orthonormal.
27
6.3 O
RTHOGONAL
P
ROJECTIONS
T
HEOREM
8: T
HE
O
RTHOGONAL
D
ECOMPOSITION
T
HEOREM
Let
W
be a subspace of
R
n
.
Then each
y
in
R
n
can be written uniquely in the form
y
=
ˆ
y
+
z
where ˆ
y
is in
W
and
z
is in
W
⊥
. In fact, if
u
1
,...,
u
p
is any orthogonal basis in
W
, then
ˆ
y
=
y
·
u
1
u
1
·
u
1
u
1
+
...
+
y
·
u
p
u
p
·
u
p
and
z
=
y

ˆ
y
P
ROPERTY OF
O
RTHOGONAL
P
ROJECTIONS
If
y
is in
W
=
Span(
u
1
,...,
u
p
), then proj
W
y
=
y
.
T
HEOREM
9: T
HE
B
EST
A
PPROXIMATION
T
HEOREM
Let
W
be a subspace of
R
n
. let
y
be any
vector in
R
n
, and let ˆ
y
be the orthogonal projection of
y
onto
W
. Then ˆ
y
is the closest point
in
W
to
y
, in the sense that

y

ˆ
y
 < 
y

v

for all
v
in
W
distinct from ˆ
y
.
T
HEOREM
10
If
u
1
,...
u
p
is an orthonormal basis for a subspace
W
of
R
n
, then
proj
W
y
=
(
y
·
u
1
)
u
1
+
(
y
·
u
2
)
u
2
+
...
+
(
y
·
u
p
)
u
p
If
U
=
£
u
1
u
2
...
u
p
/
, then
proj
W
y
=
UU
T
y
for all y in
R
n
28
6.4 T
HE
G
RAM
S
CHMIDT
P
ROCESS
T
HEOREM
11: T
HE
G
RAM
S
CHMIDT
P
ROCESS
Given a basis
x
1
,...,
x
p
or a nonzero subspace
W
of
R
n
, define
v
1
=
x
1
v
2
=
x
2

x
2
·
v
1
v
1
·
v
1
v
1
v
3
=
x
3

x
3
·
v
1
v
1
·
v
1
v
1

x
3

x
3
·
v
2
v
2
·
v
2
v
2
......
v
p
=
x
p

x
p
·
v
1
v
1
·
v
1
v
1

x
p

x
p
·
v
2
v
2
·
v
2
v
2

...

x
p
·
v
p

1
v
p

1
·
v
p

1
v
p

1
Then, {
v
1
,...,
v
p
} is an orthogonal basis for
W
. In addition,
Span{
v
1
,...,
v
k
}
=
Span{
x
1
,...,
x
k
} for 1
≤
k
≤
p
T
HEOREM
12: T
HE
QR F
ACTORIZATION
If
A
is an
m
x
n
matrix with linearly independent
columns, then
A
can be factored as
A
=
QR
where
Q
is an
m
x
n
You've reached the end of your free preview.
Want to read all 33 pages?
 Spring '08
 Chorin
 Differential Equations, Linear Algebra, Algebra, Equations, HEOREM