02 ComputerArithmetic

# 02 ComputerArithmetic - Computer Arithmetic CEE3804...

This preview shows pages 1–7. Sign up to view the full content.

CEE 3804 - Rojiani 1 Computer Arithmetic CEE3804: Computer Applications for Civil and Environmental Engineers

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
CEE 3804 - Rojiani 2 How Computers Store Numbers z Computer arithmetic is not the same as pencil and paper arithmetic or math class arithmetic. z Hand calculations are usually short and small errors are negligible. z Computer calculations are longer z Errors may accumulate over millions of steps with catastrophic results. z Computer software itself can be buggy.
CEE 3804 - Rojiani 3 Errors In Scientific Computing z A. Machine hardware malfunctions Very rare, but possible. Recall Pentium floating point error. z B. Software errors More common than you might think. see calc.exe Windows 3.1 calculator. Subtract 3.11 - 3.1 = 0.00. (Note the answer is calculated correctly but displayed incorrectly. You can check this by multiplying the answer above 0.00 * 100 = 1.) See www.bugnet.com

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
CEE 3804 - Rojiani 4 Errors z C. Blunders - Programming the Wrong Formula Depending on the QA/QC implemented, can be very common. These errors can arise from typos or other outright errors. Experimental error - data acquired by machine with limited precision z D. Truncation Errors A floating point number often cannot be represented exactly by the computer. Only a fixed storage length is available. Often a portion of the number must be truncated or rounded. Example: sums of a series of numbers vary depending on the order in which they are added.
CEE 3804 - Rojiani 5 Sorting Error Example Normal Lognormal Descending Ascending Sum of sorting 10000 numbers 4.300136 73.70984886 1.23283E+19 7.58267E-19 Order Sum 28.55013 2.50706E+12 1.44752E+18 1.39587E-16 Random (Sum of Column B) 13809803823504100000 -14.143 7.20759E-07 2.51877E+16 1.68166E-16 Descending (Sum of Column C) 13809803823504000000 0.283012 1.327121106 5.29302E+15 2.07875E-15 Ascending (Sum of Column D) 13809803823504200000 2.058755 7.83620722 2.9574E+15 3.42929E-15 4.515789 91.4496902 9.64917E+13 6.08422E-15 4.274511 71.84502808 7.80226E+13 1.00957E-14 9.762903 17376.99813 6.32725E+13 1.03636E-14 -17.0228 4.04649E-08 5.78981E+13 1.3311E-14 7.784729 2403.614561 4.19756E+13 3.40282E-14

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
CEE 3804 - Rojiani 6 Truncation Error Example Truncation example Calculate variance (s 2 ) of 3 numbers Group 1 Group 2 Group 3 Difference Col C squared Col D squared 0 9999999 99999999 0 99999980000001 9999999800000000 1 10000000 100000000 1 100000000000000 10000000000000000 2 10000001 100000001 2 100000020000001 10000000200000000 varianc e 1 1 0 1 All the columns should have the same variance.
This is the end of the preview. Sign up to access the rest of the document.

## This note was uploaded on 11/14/2008 for the course CEE 3804 taught by Professor Aatrani during the Fall '07 term at Virginia Tech.

### Page1 / 29

02 ComputerArithmetic - Computer Arithmetic CEE3804...

This preview shows document pages 1 - 7. Sign up to view the full document.

View Full Document
Ask a homework question - tutors are online