18.335 Problem Set 1 Solutions
Problem 1: Gaussian elimination
The inner loop of LU, the loop over rows, subtracts from each row a different multiple of the pivot
row. But this is exactly a rank1 update
U
→
U

xy
T
, where
x
is the columnvector of multipliers
and
y
T
is the pivot row. More explicitly, we can rewrite Gaussian elimination without row swaps
(pivoting) as:
U
=
A
for
k
= 1
to
m

1
x
=
u
k
+1:
m,k
/u
kk
U
k
+1:
m,k
:
m
=
U
k
+1:
m,k
:
m

xu
k,k
:
m
Note that I have used Matlab notation
n
:
n
0
to denote ranges (from
n
to
n
0
) of rows or columns.
In particular, note that we only have to do a rank1 update of a submatrix of
U
, and that
u
k,k
:
m
is a row vector of the
k
th row of
U
from column
k
to column
m
.
Problem 2: Asymptotic notation
(a)
Θ
means both
O
and
Ω
.
Let’s do one at a time.
By the definition of
O
,
f
(
n
)
≤
C
1
F
(
n
)
for
n > N
1
and
g
(
n
)
≤
C
2
G
(
n
)
for
n > N
2
(no absolute values since the functions were
given to be nonnegative), for some constants
C
1
,
2
and
N
1
,
2
. If we let
C
= max(
C
1
, C
2
)
and
N
= max(
N
1
, N
2
)
, then
f
(
n
) +
g
(
n
)
≤
C
1
F
(
n
) +
C
2
G
(
n
)
≤
C
[
F
(
n
) +
G
(
n
)]
for
n > N
,
hence
f
+
g
is
O
(
F
+
G
)
. Similarly for
Ω
, replacing
≤
with
≥
and
max
with
min
, so
f
+
g
is
Ω(
F
+
G
)
. Hence
f
+
g
is
Θ(
F
+
G
)
.
(b)
f
(
n
)
∈
O
[
g
(
n
)]
⇔ 
f
(
n
)
 ≤
C

g
(
n
)

for some
C >
0
and
n > N
⇔ 
g
(
n
)
 ≥
C

1

f
(
n
)

for
C

1
>
0
and
n > N
⇔
g
(
n
)
∈
Ω[
g
(
n
)]
. Q.E.D.
(c)
f
(
n
)
∈
O
[
F
(
n
)]
⇔ 
f
(
n
)
 ≤
C

F
(
n
)

for
n > N
.
If
h
(
n
)
∈
O
[
f
(
n
) +
c F
(
n
)]
then for
n > N
0
we have

h
(
n
)
 ≤
C
0

f
(
n
) +
c F
(
n
)
 ≤
C
0

f
(
n
)

+
C
0

c
 
F
(
n
)

for some
C
0
>
0
.
For
n >
max(
N
0
, N
)
,

h
(
n
)
 ≤
(
C
0
C
+
C
0

c

)

F
(
n
)

where
C
0
C
+
C
0

c

>
0
, and thus
h
(
n
)
∈
O
[
F
(
n
)]
. However, the same inference is not true if we replace
O
with
Θ
; as a simple example,
consider
f
(
n
) =
n
3
and
F
(
n
) =
n
3

n
2
with
c
=

1
: in this case
n
3
∈
O
(
n
3

n
2
)
, but
f
(
n
) +
c F
(
n
) =
n
2
and
Θ(
n
2
)
is not a subset of
Θ(
n
3

n
2
) = Θ(
n
3
)
.
(d) If the running time is
O
(
n
2
)
, that means that the time is
≤
n
2
multiplied by a constant,
asymptotically. If it is “
O
(
n
2
)
or worse”, that would mean that the time is bounded above by
any function
≥
n
2
, which is true of
every
function! Usually, when you hear things like this,
what people really mean is “
Θ(
n
2
)
or worse,” or equivalently “
Ω(
n
2
)
”.
Problem 3: Caches and matrix multiplications
(a) See figure 1. For discussion of the results, see part (d).
This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
This is the end of the preview.
Sign up
to
access the rest of the document.
 Spring '06
 Prof.CarolLivermore
 Multiplication, CPU cache, Cache algorithms

Click to edit the document details