computer arithmetic v8

# computer arithmetic v8 - Computer Arithmetic CEE3804:...

This preview shows pages 1–9. Sign up to view the full content.

01/02/12 Copyright, 2000 1 Computer Arithmetic CEE3804: Computer Applications for Civil and Environmental Engineers

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
01/02/12 Copyright, 2000 2 Learning Objectives Define: bit, byte, machine epsilon, exponent, significand, mantissa, overflow, underflow, Contrast integer vs floating point storage. Describe how range and precision varies between single and double precision. Perform simple addition and subtraction in base 2. Convert integers from base 10 to base 2 and vice versa.
01/02/12 Copyright, 2000 3 How computers store numbers: Computer arithmetic is not the same as pencil and paper arithmetic or math class arithmetic. Hand calculations usually short. Small errors negligible. Computer calculations longer, may accumulate errors over millions of steps to catastrophic results. Software itself can be buggy.

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
01/02/12 Copyright, 2000 4 Errors in scientific computing A. machine hardware malfunctions Very rare, but possible. Recall Pentium floating point error. B. software errors More common than you might think. see calc.exe Windows 3.1 calculator. Subtract 3.11 - 3.1 = 0.00. (Note the answer is calculated correctly but displayed incorrectly. You can check this by multiplying the answer above 0.00 * 100 = 1.) See http://www.wired.com/news/technology/bugs/0,2924,693
01/02/12 Copyright, 2000 5 Errors, continued C. blunders - programming the wrong formula Depending on the QA/QC implemented, can be very common. These errors can arise from typos or other outright errors. experimental error - data acquired by machine with limited precision D. Truncation error A floating point number often cannot be represented exactly by the computer. Only a fixed storage length is available. Often a portion of the number must be truncated or rounded. Example: sums of a series of numbers vary depending on the order in which they are added.

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
01/02/12 Copyright, 2000 6 Sorting Error Example
01/02/12 Copyright, 2000 7 Truncation Error Example

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
01/02/12 Copyright, 2000 8 Errors, continued E. numerical or rounding error 1. ill conditioning or sensitivity of problem For example, finding the intersection of 2 nearly parallel lines. 2. stability of algorithm Can also use inappropriate algorithm. Example: Taylor series expansion to evaluate exp(x).
This is the end of the preview. Sign up to access the rest of the document.

## This note was uploaded on 01/01/2012 for the course CEE 3804 taught by Professor Aatrani during the Spring '07 term at Virginia Tech.

### Page1 / 32

computer arithmetic v8 - Computer Arithmetic CEE3804:...

This preview shows document pages 1 - 9. Sign up to view the full document.

View Full Document
Ask a homework question - tutors are online