2. If we want to increase the degree of the approximating polynomial we
need to start over again and solve a larger set of normal equations.
That is, we cannot use the
a
0
, a
1
, . . . , a
n
we already found.
It is more efficient and easier to solve the Least Squares Approximation
problem using orthogonality, as we did with approximation by trigonometric
polynomials. Suppose that we have a set of polynomials defined on an interval
[
a, b
],
{
φ
0
, φ
1
, ..., φ
n
}
,
such that
φ
j
is a polynomial of degree
j
. Then, we can write any polyno
mial of degree at most
n
as a linear combination of these polynomials. In
particular, the Least Square Approximating polynomial
p
n
can be written as
p
n
(
x
) =
a
0
φ
0
(
x
) +
a
1
φ
1
(
x
) +
...
+
a
n
φ
n
(
x
) =
n
X
j
=0
a
j
φ
j
(
x
)
,
We have textbook solutions for you!
The document you are viewing contains questions related to this textbook.
The document you are viewing contains questions related to this textbook.
Expert Verified
84
CHAPTER 5.
LEAST SQUARES APPROXIMATION
for some coefficients
a
0
, . . . , a
n
to be determined. Then
E
(
a
0
, ..., a
n
) =
Z
b
a
[
f
(
x
)

n
X
j
=0
a
j
φ
j
(
x
)]
2
dx
=
Z
b
a
f
2
(
x
)
dx

2
n
X
j
=0
a
j
Z
b
a
φ
j
(
x
)
f
(
x
)
dx
+
n
X
j
=0
n
X
k
=0
a
j
a
k
Z
b
a
φ
j
(
x
)
φ
k
(
x
)
dx
(5.13)
and
0 =
∂E
∂a
m
=

2
Z
b
a
φ
m
(
x
)
f
(
x
)
dx
+ 2
n
X
j
=0
a
j
Z
b
a
φ
j
(
x
)
φ
m
(
x
)
dx.
for
m
= 0
,
1
, . . . , n
, which gives the normal equations
n
X
j
=0
a
j
Z
b
a
φ
j
(
x
)
φ
m
(
x
)
dx
=
Z
b
a
φ
m
(
x
)
f
(
x
)
dx,
m
= 0
,
1
, ..., n.
(5.14)
Now if the set of approximating functions
{
φ
0
,
....
φ
n
}
is
orthogonal
, i.e.
Z
b
a
φ
j
(
x
)
φ
m
(
x
)
dx
= 0
if
j
6
=
m
(5.15)
then the coefficients of the least squares approximation are explicitly given
by
a
m
=
1
α
m
Z
b
a
φ
m
(
x
)
f
(
x
)
dx,
α
m
=
Z
b
a
φ
2
m
(
x
)
dx,
m
= 0
,
1
, ..., n.
(5.16)
and
p
n
(
x
) =
a
0
φ
0
(
x
) +
a
1
φ
1
(
x
) +
...
+
a
n
φ
n
(
x
)
.
Note that if the set
{
φ
0
,
....
φ
n
}
is orthogonal, (5.16) and (5.13) imply the
Bessel inequality
n
X
j
=0
α
j
a
2
j
≤
Z
b
a
f
2
(
x
)
dx.
(5.17)
5.2. LINEAR INDEPENDENCE AND GRAMSCHMIDT ORTHOGONALIZATION
85
This inequality shows that if
f
is square integrable, i.e. if
Z
b
a
f
2
(
x
)
dx <
∞
,
then the series
∞
X
j
=0
α
j
a
2
j
converges.
We can consider the Least Squares approximation for a class of linear
combinations of orthogonal functions
{
φ
0
, ..., φ
n
}
not necessarily polynomials.
We saw an example of this with Fourier approximations
1
. It is convenient
to define a weighted
L
2
norm associated with the Least Squares problem
k
f
k
w,
2
=
Z
b
a
f
2
(
x
)
w
(
x
)
dx
1
2
,
(5.18)
where
w
(
x
)
≥
0 for all
x
∈
(
a, b
)
2
. A corresponding inner product is defined
by
h
f, g
i
=
Z
b
a
f
(
x
)
g
(
x
)
w
(
x
)
dx.
(5.19)
Definition 7.
A set of functions
{
φ
0
, ..., φ
n
}
is orthogonal, with respect to
the weighted inner product (5.19), if
h
φ
j
, φ
k
i
= 0
for
j
6
=
k
.
5.2
Linear Independence and GramSchmidt
Orthogonalization
Definition 8.
A set of functions
{
φ
0
(
x
)
, ..., φ
n
(
x
)
}
defined on an interval
[
a, b
]
is said to be linearly independent if
a
0
φ
0
(
x
) +
a
1
φ
1
(
x
) +
. . . a
n
φ
n
(
x
) = 0
,
for all
x
∈
[
a, b
]
(5.20)
then
a
0
=
a
1
=
. . .
=
a
n
= 0
. Otherwise, it is said to be linearly dependent.
1
For complexvalued functions orthogonality means
Z
b
a
φ
j
(
x
)
¯
φ
m
(
x
)
dx
= 0 if
j
6
=
m
,
where the bar denotes the complex conjugate
2
More precisely, we will assume
w
≥
0,
Z
b
a
w
(
x
)
dx >
0, and
Z
b
a
x
k
w
(
x
)
dx <
+
∞
for
k
= 0
,
1
, . . .
. We call such a
w
an admissible weight function.
86
CHAPTER 5.
LEAST SQUARES APPROXIMATION
Example 12.
The set of functions
{
φ
0
(
x
)
, ..., φ
n
(
x
)
}
, where
φ
j
(
x
)
is a poly
nomial of degree
j
for
j
= 0
,
1
, . . . , n
is linearly independent on any interval