3. Leastsquares methods
Special classes of optimization problems are leastsquare problems. In this type of
problems we are most interested in problems that involve modeling some dataset for
example as is done with the resistivity sounding case. Typically we would have a dataset,
consisting of measurements
y
and possibly also independent (or control) variables
x
(such
as our electrode spacing:
a dataset (
x
i
, y
i
)
i
=1 …
N
and we have a physical model that relates these variable, for example in the form of a set
of partial differential equations
f
(
x
i
, y
i
,
θ
) = 0
where the physical model has physical parameters
θ
. In the more favorable case, this
model can be written as
y
=
y
(
x
i
,
θ
)
In addition we have measurement errors on the measurements of
y
i
(we assume that the
errors on
x
are much less or negligible), hence we need a framework to account for these.
That framework is formulated within probability theory, hence is may be useful to refresh
our memory with some basic concepts.
3.1 Probability concepts: a review
To describe errors we do not know exactly/deterministically how large they are we use
the concept of random variable. For example in the case or measurements with errors, we
could have statistical information about these errors in terms of bias of the measurement
or variation/variance of the measurement. Essentially all these statistics would be
available directly if we would be able to repeat an experiment many times, each time we
record a different measurements, hence the total set of measurements determine the
probability distribution of outcomes of these measurements. However, in many case we
cannot afford the luxury of repeated measurements, so we have either to rely on
experience of using the measurement device or provide an expert guess. The
experimental error, being the difference between the true but unknown measurement and
the actual measurement is therefore modeled as a random variable. Such random variable
is fully described by a probability function. For errors we often use a Gaussian
probability function
)
)
(
2
1
exp(
2
1
)
(
2
2
μ
σ
σ
π


=
x
x
f
X
This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
∫
+∞
∞

=
=
dx
x
xf
X
E
X
)
(
]
[
μ
∫
+∞
∞


=
=
dx
x
f
x
X
Var
X
)
(
)
(
]
[
2
2
μ
σ
Where
X
is the random variable describing the amount of error made. If
μ
=0 then we say
the measurement is unbiased, i.e. we are not making any systematic errors.
In other cases, when we have bounds on the error we often use a traingular distribution
function.
If we measure different variables, we will have a measurement error for each variable.
This is the case for resistivity sounding, where we measure at different spacing, so each
measurement can have a different kind of error. Moreover, it may not be appropriate to
look at these errors as being independent one from each other, that is if you measure
something too high for one variable, then it may be such that you will also measure too
high for another variable (you may have been making the same mistake). Hence, we
cannot simply state the probability distribution for the errors independently. We need a
This is the end of the preview.
Sign up
to
access the rest of the document.
 Spring '10
 .
 Linear Algebra, Normal Distribution, Yi, Singular value decomposition, Linear least squares

Click to edit the document details