ERROR
TRUNCATION There are instances where a computation is represented by an
infinite series. For example, the series:
1 - 1/2 + 1/4 - 1/8 + 1/16 - ...
Is an infinite series. Often we need can represent a function by its
Taylor expansion (another infinite series):
f(x) = f(c) + f'(c)*(x - c) + f''(c)/2! * (x-c)^2 + ...
However, computers cannot perform infinite calculations. We approximate
our answers by choosing where to terminate (or "truncate") the series.
So, for example, the above Taylor series could have been truncated as
follows:
(n)
f(x) = f(c) + f'(c)*(x-c) + ... + f (x)/n! * (x-n)^2
Truncation error occurs when you only use a finite number of terms in
an infinite series. The larger n (i.e. the more terms in the series), the
more accurate the result.
REAL-LIFE EXAMPLES OF FLOATING POINT ERRORS
- The Patriot missile failures in Desert Storm were due to roundoff
error because the system was designed to be on alert for only a few
minutes at a time. Instead, they ran continuously for *days*.
Thus, the clock, which was storing time in floating-point, began to
have too many digits to the left of the decimal place and not
enough after the decimal place to do accurate timings of incoming
SCUD velocities.
- The Vancouver Stock Exchange ran for 22 months with a bad algorithm
for computing the index. The initial value of the index was
normalized to be 1000.000. Instead of the obvious algorithm of
adding up all the current values of the stocks, somebody thought it
would be more efficient (bonehead!) if only the stocks that
actually changed that day were included in the computation, since
some stocks don't change price on some days. So, instead of the
obvious "add all the stock prices up" algorithm, they used
for each stock S
if S changed in price today then
index := index + change in S
end if
end for
Of course, the changes are almost always small compared to
1000.000, and we're adding a bunch of small numbers to a big
number. Result: loss of precision, and after 22 months, when the
actual value of the VSE had become about 1098.892, the accumulated
roundoff error of 22 months had the index valued at 520.
- Another interesting fact from the software engineering point of view
is that they had decided to "cut costs" and re-use alot of code from
the Ariane 4 launcher. This is fine and dandy, but they decided to cut
costs *further* by not even reviewing some of the code to ensure that
it would still work on the Ariane 5. The overflow mentioned in the
article occurred simply because, during the initial launch phase, the
Ariane 4 was a slower rocket and never had a horizontal velocity
greater than 32768, and nobody noticed this fact during the truncated
code reviews that were performed.
- An excellent page with many more examples is at:
http://dutita0.twi.tudelft.nl/users/vuik/wi211/disasters.html
MORE EXAMPLES OF UNSTABLE COMPUTATIONS
(1)
A first example of a stable and unstable computation is:
Unstable:
[[...((...(1+eps)^(1/2))^(1/2))...)^1/2)]^2]^2]..]^2
Stable:
[[...((...(1+eps)^2)^2)...)^2)]^(1/2)]^(1/2)]...]^(1/2)
That is for small eps if you compute 1+eps, take K square roots
followed by K squarings you would recover 1+eps in exact arithmetic.
An easy anlysis of the round-off error bound predicts growth exponential
in the error (exponential in K) and this can be observed in
computation. Reversing the order (squaring first, followed by
K square roots) yields a uniform bound on the error (as a function
of K) and this is observed in practice. The example is probably better
if what is computed are the above expressions minus 1.0 (as then the
exact answer should be eps in both cases.
(2)
A second example of a stable and unstable computation is
the following Taylor's series expansion for approximating the
value of e^x
e^x = 1 + x + x^2/2! + x^3/3! + ... + x^n/n! + ...
Obviously, the more terms we eliminate from the end, the less
accurate our result will be due to truncation error. But recall
that even if we did implement this equation in a program, it
would be subject to roundoff error.
What is the best way to see this? Let's compress the above
equation into a sum as follows:
n
e^x = SUM { x^i / i! }
i=0
As 'i' increases, you need to determine what will be increasing
or decreasing faster -> the numerator or denominator of each term.
It should be apparent that i! grows much more quickly than x^i.
Initially, we could propose to sum the series *backwards* since
the rightmost numbers will most likely be smaller due to relatively
larger denominators. This may decrease the problem of overflow
and roundoff error since we begin summing with smaller numbers
instead of starting with larger numbers.
But what if the numerators x^i of the end terms are very large?
If we summed backwards or forwards, we would run into the same
problem of roundoff error and possible overflow.
A better solution is to use what is referred to as "Horner's Rule".
By inspecting the equation, you should realize that you can take
out common factors of 'x' since it continuously increases its
exponent.
So, for example, if we truncated our equation for e^x to 10 terms
as follows:
1 + x + x^2/2! + ... + x^9/9!
It can be rewritten a...