AND
OVERFLOW UNDERFLOW
----------------------
Consider the following floating point number system
s = 3, Beta = 10, E_min = -9, E_max = 9
(this system is 3-digit, base 10, 1-digit exponent)
The largest possible absolute value that can be represented is
the *overflow* value. This would be:
9.99 x 10^9
Now, we must consider what is the lowest possible absolute value.
If our system was restricted to normalized numbers, this would be:
1.00 x 10^-9 <-- "normalized min"
NOTE: Normalized means that the lefmost digit is non-zero. In scientific
notation, all numbers are normalized.
However, a floating point number system need not be restricted to
normalized numbers. So, if our system was not subject to this restriction,
the lowest absolute value (or underflow value) would be
0.01 x 10^-9 (which is 10^-11)
REPRESENTATION ERROR
--------------------
The number 1.01 can be reprsented as 1.01 x 10^1
The number 1200 can be represented as 1.20 x 10^3
What about 1.001? This cannot be represented exactly in our floating
point number system because it would require 4 digits in the mantissa.
What about 5.23x10^23. Not possible because our system only has 1 digit
for the exponent.
It is important to realize from this example that floating point number
systems are *limited*. Thus, not all numbers can be reprsented exactly
(even very large or very small ones).
ROUNDOFF ERROR
--------------
Consider the following operation:
1 + 0.001 = ???
Under our floating point number system, our operation would become:
1.00 x 10^1 + 1.00 x 10^-3
We know that the correct mathematical solution is 1.001. But, according
to the discussion of representation error, this cannot be represented exactly
in our floating point number system.
So in our system
fl(1 + 0.001) = 1.00 <-- result is "rounded"
This introduces error because we didn't get an exact result.
Some more examples of rounding:
fl (1.00 + 0.004) = fl (1.004) = 1.00
fl (1.00 + 0.006) = fl (1.006) = 1.01
Now what about 1.00 + 0.005? If it results in 1.01, this introduces bias
for since more numbers will be rounded up. If we chose to round down for
1.005, this would still introduce bias.
One solution, implemented by certain floating point number systems, is
to round to the nearest *even* digit. This eliminates bias because half
of the .5's will be rounded up, while the other half will be rounded down.
So our example will become
fl (1.00 + 0.005) = fl (1.005) = 1.00
fl (1.00 + 0.015) = fl (1.015) = 1.02
MORE ON LOSS OF PRECISION
-------------------------
We previously defined the machine epsilon as the smallest floating point
number such that:
fl(1 + e_mach) > 1
How would you approximately compute this. One algorithm would be to
start at 1 and continue iterating until the above rule is broken.
e_mach := 1
while (1 + e_mach > 1) do
prev = e_mach
e_mach := e_mach / 10
end while
e_mach := prev
- Why do we need the "prev" value? Notice that our "while" loop will only
exit *after* our e_mach is too small.
- Instead of divising by 10, we divide by a smaller number to get a more
accurate estimate e_mach of (but at the cost of more iterations).
- NOTE: The machine epsilon is *not* the same as the underflow value!!!
Loss of precision can often be observed in what is termed "catastrophic
cancellation" or "cancellation error".
e.g.1 (use our 3-digit floating point system again)
1.00 x 10^3 + 1 = 1.00 x 10^3
= fl(1000 + 1) - 1000
= 1000 - 1000
= 0
But the true answer is 1. Our relative error becomes abs(1-0)/1 = 1 which
is very large (catastrophic).
e.g.2
100 + 3.14 - 100
= fl(100 + 3.14) - 100
= 103 - 100
= 3
But true value is 3.14
INTRO TO INTERVAL ARITHMETIC
----------------------------
- All of the methods of error estimates talked about in Numerical
Integration and Zero Finding assume that our computer does the actual
arithmetic exactly (ie., they were not concerned with roundoff
errors). But that's not true. We know, for example, that if our
computer used a base 10 number system, then when we compute 1/3 =
0.333333..., the answer is not exactly representable. We can only
store a finite list of 3's. Even though our computers actually store
numbers in base-2 rather than base-10, the same principle holds: the
the result of most computations, even if it's just one single addition
or multiplication, is not exact. We lose a little bit of precision on
each operation. As we perform more and more operations, the little bit
of precision we lose on each step adds up. It would be good if we
could automatically keep track of a bound on the error we make.
Luckily, it is not hard to do so.
* Rounding modes
- When performing one of the four basic operations (+-*/), we can choose
a rounding mode: round up, round down, or round-to-nearest. Usually
we choose round-to-nearest. So, for example, in base-10 4-digit
arithmetic, 1.000 + 0.0009 = 1.0009 which when rounded to the nearest
representable number, gives 1.001. If we want to...