This preview shows page 1. Sign up to view the full content.
Unformatted text preview: CIS 450 Computer Architecture and Organization
Lecture 4: Floating Point Representation Mitch Neilsen neilsen@ksu.edu 219D Nichols Hall Topics
Integer Representation
Quick Quiz Floating Point Representation
IEEE Floating Point Standard Rounding Floating Point Operations Mathematical Properties 2 Quick Quiz: Integer Representation
Consider a 5bit two's complement representation for signed integers. Fill in the empty boxes in the following table: 3 Floating Point Puzzles
For each of the following C expressions, either:
Argue that it is true for all argument values Explain why not true 2/3 == 2/3.0 int x = ...; float f = ...; double d = ...; x == (int)(float) x x == (int)(double) x f == (float)(double) f d == (float) d Assume neither d nor f is NaN f == (f); d < 0.0 d > f ((d*2) < 0.0) f > d d * d >= 0.0 (d+f)d == f
4 IEEE Floating Point
IEEE Standard 754
Established in 1985 as uniform standard for floating point arithmetic
Before that, many idiosyncratic formats Supported by all major CPUs Driven by Numerical Concerns
Nice standards for rounding, overflow, underflow Hard to make go fast
Numerical analysts predominated over hardware types in defining standard 5 Fractional Binary Numbers
2i 2i1 4 2 1 b2 b1 b0 . b1 b2 b3 1/2 1/4 1/8 2j bj bi bi1 Representation
Bits to right of "binary point" represent fractional powers of 2 Represents rational number:
6 k = j bk 2 i k Frac. Binary Number Examples
Value
53/4 27/8 63/64 Representation
101.112 10.1112 0.1111112 Observations
Divide by 2 by shifting right Multiply by 2 by shifting left Numbers of form 0.111111...2 just below 1.0
1/2 + 1/4 + 1/8 + ... + 1/2i + ... 1.0 Use notation 1.0 7 Representable Numbers
Limitation
Can only exactly represent numbers of the form x/2k Other numbers have repeating bit representations Value
1/3 1/5 1/10 Representation
0.0101010101[01]...2 0.001100110011[0011]...2 0.0001100110011[0011]...2 8 Floating Point Representation
Numerical Form
1s M 2E
Sign bit s determines whether number is negative or positive Significand M normally a fractional value in range [1.0,2.0). Exponent E weights value by power of two Encoding
s exp frac MSB is sign bit exp field encodes E frac field encodes M 9 Floating Point Precisions
Encoding
s exp frac MSB is sign bit exp field encodes E frac field encodes M Sizes
Single precision: 8 exp bits, 23 frac bits
32 bits total Double precision: 11 exp bits, 52 frac bits
64 bits total Extended precision: 15 exp bits, 63 frac bits
Only found in Intelcompatible machines Stored in 80 bits 1 bit wasted 10 "Normalized" Numeric Values
Condition
exp 000...0 and exp 111...1 Exponent coded as biased value
E = Exp Bias
Exp : unsigned value denoted by exp Bias : Bias value Single precision: 127 (Exp: 1...254, E: 126...127) Double precision: 1023 (Exp: 1...2046, E: 1022...1023) in general: Bias = 2e1  1, where e is number of exponent bits Significand coded with implied leading 1
M = 1.xxx...x2
xxx...x: bits of frac Minimum when 000...0 (M = 1.0) Maximum when 111...1 (M = 2.0 ) Get extra leading bit for "free" 11 Normalized Encoding Example
Value
Float f = 15213.0; 1521310 = 111011011011012 = 1.11011011011012 X 213 Significand
M = frac = 1.11011011011012 110110110110100000000002 13 127 140 = Exponent
E = Bias = Exp = 100011002 Floating Point Representation: Hex: Binary: 140: 15213: 12 4 6 6 D B 4 0 0 0100 0110 0110 1101 1011 0100 0000 0000 100 0110 0 1110 1101 1011 01 Denormalized Values
Condition
exp = 000...0 Value
Exponent value E = Bias + 1 Significand value M = 0.xxx...x2
xxx...x: bits of frac Cases
exp = 000...0, frac = 000...0
Represents value 0 Note that have distinct values +0 and 0 exp = 000...0, frac 000...0
Numbers very close to 0.0 Lose precision as get smaller "Gradual underflow" 13 Special Values
Condition
exp = 111...1 Cases
exp = 111...1, frac = 000...0
Represents value (infinity) Operation that overflows Both positive and negative E.g., 1.0/0.0 = 1.0/0.0 = +, 1.0/0.0 =  exp = 111...1, frac 000...0
NotaNumber (NaN) Represents case when no numeric value can be determined E.g., sqrt(1),  , 0 14 Summary of Floating Point Real Number Encodings
 Normalized Denorm 0 +Denorm +Normalized +
NaN NaN +0 15 Tiny Floating Point Example
8bit Floating Point Representation
the sign bit is in the most significant bit. the next four bits are the exponent, with a bias of 7. the last three bits are the frac Same General Form as IEEE Format
normalized, denormalized representation of 0, NaN, infinity
7 6 3 2 0 s exp frac 16 Values Related to the Exponent
Exp 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 17 exp 0000 0001 0010 0011 0100 0101 0110 0111 1000 1001 1010 1011 1100 1101 1110 1111 E 6 6 5 4 3 2 1 0 +1 +2 +3 +4 +5 +6 +7 n/a 2E 1/64 1/64 1/32 1/16 1/8 1/4 1/2 1 2 4 8 16 32 64 128 (denorms) (inf, NaN) Dynamic Range
s exp 0 0 Denormalized 0 ... numbers 0 0 0 0 ... 0 0 Normalized 0 numbers 0 0 ... 0 0 0 18 frac E 6 6 6 6 6 6 6 1 1 0 0 0 7 7 n/a Value 0 1/8*1/64 = 1/512 2/8*1/64 = 2/512 6/8*1/64 7/8*1/64 8/8*1/64 9/8*1/64 14/8*1/2 15/8*1/2 8/8*1 9/8*1 10/8*1 = = = = = = = = = 6/512 7/512 8/512 9/512 14/16 15/16 1 9/8 10/8 0000 000 0000 001 0000 010 0000 0000 0001 0001 0110 0110 0111 0111 0111 110 111 000 001 110 111 000 001 010 closest to zero largest denorm smallest norm closest to 1 below closest to 1 above 1110 110 1110 111 1111 000 14/8*128 = 224 15/8*128 = 240 inf largest norm Distribution of Values
6bit IEEElike format
e = 3 exponent bits f = 2 fraction bits Bias is 3 Notice how the distribution gets denser toward zero. 15 10 5 Denormalized 0 Normalized 5 Infinity 10 15 19 Distribution of Values (closeup view)
6bit IEEElike format
e = 3 exponent bits f = 2 fraction bits Bias is 3 1 0.5
Denormalized 0
Normalized 0.5
Infinity 1 20 Interesting Numbers
Description Zero Smallest Pos. Denorm. exp frac Numeric Value 0.0 2 {23,52} X 2 {126,1022} 00...00 00...00 00...00 00...01 Single 1.4 X 1045 Double 4.9 X 10324
Largest Denormalized 00...00 11...11 (1.0 ) X 2 {126,1022} Single 1.18 X 1038 Double 2.2 X 10308
Smallest Pos. Normalized 00...01 00...00 1.0 X 2 {126,1022} 1.0 (2.0 ) X 2{127,1023} Just larger than largest denormalized
One Largest Normalized 01...11 00...00 11...10 11...11 Single 3.4 X 1038 Double 1.8 X 10308 21 Special Properties of Encoding
FP Zero Same as Integer Zero
All bits = 0 Can (Almost) Use Unsigned Integer Comparison
Must first compare sign bits Must consider 0 = 0 NaNs problematic
Will be greater than any other values What should comparison yield? Otherwise OK
Denorm vs. normalized Normalized vs. infinity 22 Floating Point Operations
Conceptual View
First compute exact result Make it fit into desired precision
Possibly overflow if exponent too large Possibly round to fit into frac Rounding Modes (illustrate with $ rounding)
$1.40 $1.60 $1.50 $2.50 $1.50 Zero Round down () Round up (+) Nearest Even (default) $1 $1 $2 $1 $1 $1 $2 $2 $1 $1 $2 $2 $2 $2 $3 $2 $1 $2 $1 $2 Note: 1. Round down: rounded result is close to but no greater than true result. 2. Round up: rounded result is close to but no less than true result. 23 Closer Look at RoundToEven
Default Rounding Mode
Hard to get any other kind without dropping into assembly All others are statistically biased
Sum of set of positive numbers will consistently be over or under estimated Applying to Other Decimal Places / Bit Positions
When exactly halfway between two possible values
Round so that least significant digit is even E.g., round to nearest hundredth
1.2349999 1.2350001 1.2350000 1.2450000 1.23 1.24 1.24 1.24 (Less than half way) (Greater than half way) (Half wayround up) (Half wayround down) 24 Rounding Binary Numbers
Binary Fractional Numbers
"Even" when least significant bit is 0 Half way when bits to right of rounding position = 100...2 Examples
Round to nearest 1/4 (2 bits right of binary point) Value 2 3/32 2 3/16 2 7/8 2 5/8 Binary Rounded Action 10.000112 10.002 (<1/2down) 10.001102 10.012 10.111002 11.002 10.101002 10.102 (>1/2up) (1/2up) (1/2down) Rounded Value 2 2 1/4 3 2 1/2 25 FP Multiplication
Operands
(1)s1 M1 2E1
* (1)s2 M2 2E2 Exact Result
(1)s M 2E Sign s: s1 ^ s2 Significand M: Exponent E: M1 * M2 E1 + E2 Fixing
If M 2, shift M right, increment E If E out of range, overflow Round M to fit frac precision Implementation
Biggest chore is multiplying significands 26 FP Addition
Operands
(1)s1 M1 2E1 (1)s2 M2 2E2 Assume E1 > E2
+ (1)s M (1)s1 M1 (1)s2 M2
E1E2 Exact Result
(1)s M 2E Sign s, significand M:
Result of signed align & add Exponent E: E1 Fixing
If M 2, shift M right, increment E if M < 1, shift M left k positions, decrement E by k Overflow if E out of range Round M to fit frac precision 27 Mathematical Properties of FP Add
Compare to those of Abelian Group
Closed under addition?
But may generate infinity or NaN YES YES NO YES ALMOST Commutative? Associative?
Overflow and inexactness of rounding 0 is additive identity? Every element has additive inverse
Except for infinities & NaNs Monotonicity
a b a+c b+c?
Except for infinities & NaNs ALMOST 28 Math. Properties of FP Mult
Compare to Commutative Ring
Closed under multiplication?
But may generate infinity or NaN YES YES NO YES Multiplication Commutative? Multiplication is Associative? 1 is multiplicative identity? Possibility of overflow, inexactness of rounding Multiplication distributes over addition? NO
Possibility of overflow, inexactness of rounding Monotonicity
a b & c 0 a *c b *c?
Except for infinities & NaNs ALMOST 29 Creating Floating Point Number
Steps
Normalize to have leading 1 Round to fit within fraction Postnormalize to deal with effects of rounding
7 6 3 2 0 s exp frac Case Study
Convert 8bit unsigned numbers to tiny floating point format Example Numbers 128 10000000 15 33 35 138 30 00001101 00010001 00010011 10001010 00111111 63 Normalize
7 6 3 2 0 s exp frac Requirement
Set binary point so that numbers of form 1.xxxxx Adjust all to have leading one
Decrement exponent as shift left Value 128 15 17 19 138 63 31 Binary 10000000 00001101 00010001 00010011 10001010 00111111 Fraction 1.0000000 1.1010000 1.0001000 1.0011000 1.0001010 1.1111100 Exponent 7 3 5 5 7 5 Rounding
1.BBGRXXX
Guard bit: LSB of result Round bit: 1st bit removed Sticky bit: OR of remaining bits Round up conditions
Round = 1, Sticky = 1 Value 128 15 17 19 138 63 32 > 0.5 Round to even Incr? Rounded N 1.000 N N Y Y Y 1.101 1.000 1.010 1.001 10.000 GRS 000 100 010 110 111 111 Guard = 1, Round = 1, Sticky = 0 Fraction 1.0000000 1.1010000 1.0001000 1.0011000 1.0001010 1.1111100 Postnormalize
Issue
Rounding may have caused overflow Handle by shifting right once & incrementing exponent Value 128 15 17 19 138 63 Rounded 1.000 1.101 1.000 1.010 1.001 10.000 Exp 7 3 4 4 7 5 1.000/6 Adjusted Result 128 15 16 20 134 64 33 Floating Point in C
C Guarantees Two Levels
float double single precision double precision Conversions
Casting between int, float, and double changes numeric values Double or float to int
Truncates fractional part Like rounding toward zero Not defined when out of range or NaN Generally sets to TMin int to double
Exact conversion, as long as int has 53 bit word size int to float
Will round according to rounding mode 34 Curious Excel Behavior Default Format Currency Format Number Subtract 16 Subtract .3 Subtract .01 16.31 0.31 0.01 1.2681E15 $16.31 $0.31 $0.01 ($0.00) Spreadsheets use floating point for all computations Some imprecision for decimal arithmetic Can yield nonintuitive results to an accountant! 35 Summary
IEEE Floating Point Has Clear Mathematical Properties
Represents numbers of form M X 2E Can reason about operations independent of implementation
As if computed with perfect precision and then rounded Not the same as real arithmetic
Violates associative and distributive properties Associative: (x + (y+z)) == ((x+y)+z) Distributive: x*(y+z) == x*y + x*z Makes life difficult for compilers and serious numerical applications programmers 36 ...
View
Full
Document
This note was uploaded on 04/09/2008 for the course CIS 450 taught by Professor Neilsen,mitch during the Spring '08 term at Kansas State University.
 Spring '08
 Neilsen,Mitch
 Computer Architecture

Click to edit the document details