8
Advanced Lecture Pipeline
Pipeline Complications
CS510 Computer Architectures
Lecture 8 - 1
Extending the DLX to Handle Multi-cycle Operations
EX
int unit EX FP/int EX Multiply MEM
IF
ID IF
ID
WB MEM
WB
EX FP adder
EX FP/Int divider
DLX pipeline with 3 additional unpipelined, floadting-point functional units
Pipeline Complications CS510 Computer Architectures Lecture 8 - 2
Multicycle Operations
integer unit EX
FP/integer multiply M1 M2 M3 M4 M5 M6 M7
IF
ID A1
FP adder A2 A3 A4
MEM
WB
FP/integer divider DIV
24 clock cycles
Pipeline Complications
CS510 Computer Architectures
Lecture 8 - 3
Latency and Initiation Interval
MULTD ADDD LD* SD* IF ID IF M1 ID IF M2 M3 M4 M5 M6 M7 MEM AI A2 A3 A4 MEM WB ID EX MEM WB IF ID EX MEM WB Data needed Result available WB * FP LD and ST are
same as integer by having 64-bit path to memory.
Latency: Number of intervening cycles between an instruction that produces a result and an instruction that uses the result Initiation Interval: number of cycles that must elapse between issuing of two operations of a given type Example
Integer ALU Load FP add FP mul FP div
Pipeline Complications
latency 0 1 3 6 24
initiation interval 1 1 1 1 25
Lecture 8 - 4
CS510 Computer Architectures
Floating Point Operations
Floating Point: long execution time Also, pipeline FP execution unit may initiate new instructions without waiting full latency
Reality: MIPS R4000
FP Instruction Add, Subtract Multiply Divide Square root Negate Absolute value FP compare
Latency 4 8 36 112 2 2 3
Cycles before using result
Initiation Interval (MIPS R4000) 3 4 35 111 1 1 2
Cycles before issuing instr of the same type Lecture 8 - 5
Pipeline Complications
CS510 Computer Architectures
Complications Due to FP Operations (in DLX)
Because the divide unit is not fully pipelined, structural hazards can ocur WAW hazards are possible, since instructions no longer reach WB in order. (WAR hazards are not possible, since register reads always occur in ID) Instructions can complete in a different order than they were issued, causing problems with exceptions Because of longer latency of operations, stalls for RAW hazards will be more frequent
Pipeline Complications
CS510 Computer Architectures
Lecture 8 - 6
Summary of Pipelining Basics
Hazards limit performance Structural: need more HW resources Data: need forwarding, compiler scheduling Control: early evaluation of PC, delayed branch, prediction Increasing length of pipe increases impact of hazards; pipelining helps instruction bandwidth, not latency Interrupts, FP Instruction Set makes pipelining harder Compilers reduce cost of data and control hazards Load delay slots Branch delay slots Branch prediction
Pipeline Complications
CS510 Computer Architectures
Lecture 8 - 7
Case Study: MIPS R4000 and Introduction to Advanced Pipelining
Pipeline Complications
CS510 Computer Architectures
Lecture 8 - 8
Case Study:
MIPS R4000 Pipeline
8 Stage Pipeline:
IF
First half of fetching of instruction
PC selection Initiation of instruction cache access
IS - Second half of fetching of instruction
Access to instruction cache
RF Instruction decode, register fetch, hazard checking, and also instruction cache hit detection(tag check) EX Execution
Effective address calculation ALU operation Branch target computation and condition evaluation
Pipeline Complications
DF - First half of access to data cache DS - Second half of access to data cache TC - Tag check for data cache hit WB -Write back for loads and register-register operations
CS510 Computer Architectures
Lecture 8 - 9
The Pipeline Structure of the R4000
IF IS RF
REG
EX
ALU
DF
DS
Data Memory
TC
WB
REG
Instruction Memory
Instruction is available Tag check
load data available
Pipeline Complications
CS510 Computer Architectures
Lecture 8 - 10
Case Study: MIPS R4000
LOAD Latency
Load data available with forwarding LD R1, X IF IS IF RF IS IS IF EX RF RF IS IS IF 2 Stall Cycles DF EX EX RF RF IS DS DF DF EX EX RF TC DS DS DF DF EX WB TC . . WB . TC DS . . . DS DF ...
ADD R3, R1, R2
Load data needed
2 Cycle Load Latency
Pipeline Complications
CS510 Computer Architectures
Lecture 8 - 11
Case Study: MIPS R4000
LOAD Followed by ALU Instructions
IF LW R1 ADD R2, R1 SUB R3, R1 OR R4, R1 IS IF RF IS IF EX RF IS IF DF stall stall stall DS stall stall stall TC EX RF IS WB DF ... EX ... RF ...
Forwarding 2 cycle Load Latency with Forwarding Circuit
Pipeline Complications
CS510 Computer Architectures
Lecture 8 - 12
Case Study: MIPS R4000
Branch Latency
Branch target address available after EX stage R4000 uses Predict NOT TAKEN TAKEN Br IF Delay Slot Stall Stall Br Target instr IS IF RF IS EX RF DF EX IF DS DF IS TC DS RF WB TC ... EX ...
Delay Slot plus 2 stall cycles NOT TAKEN Br IF IS RF DF DS WB EX TC Delay Slot IF IS RF EX DF TC ... DS Br instr +2 IF IS RF EX DS ... DF Br instr +3 IF IS RF DF ... EX Br instr +4 IS EX ... IF RF Predict NOT TAKEN strategy NOT TAKEN: one-cycle delayed slot TAKEN: one-cycle delayed slot followed by two stalls - 3 cycle latency
CS510 Computer Architectures
Pipeline Complications
Lecture 8 - 13
Extending DLX to Handle Floating Point Operations
Integer Unit(EX)
FP/integer multiply FP Multiplier
IF
ID FP Adder
MEM
WB
FP Divider
Pipeline Complications
CS510 Computer Architectures
Lecture 8 - 14
MIPS R4000 FP Unit
FP Adder, FP Multiplier, FP Divider Last step of FP Multiplier/Divider uses FP Adder HW 8 kinds of stages in FP units: (single copy of each)
Stage A D E M N R S U
Pipeline Complications
Functional unit FP adder FP divider FP multiplier FP multiplier FP multiplier FP adder FP adder
Description Mantissa ADD stage Divide pipeline stage Exception test stage First stage of multiplier Second stage of multiplier Rounding stage Operand shift stage Unpack FP numbers
Lecture 8 - 15
CS510 Computer Architectures
MIPS R4000 FP Pipe Stages
FP Instr Add, Subtract Multiply Divide Square root Negate Absolute value FP compare Stages: M 1 U U U U U U U 2 S+A E+M A E S S A 3 4 A+R R+S M M R D27 (A+R)108 5 6 7 8 ... M N N+A R ... D+AD+R, D+A, D+R, A, R ... A R latency 4 8 36 112 2 2 3
R N Second stage of multiplier
First stage of multiplier
R S U
Rounding stage Operand shift stage Unpack FP numbers
A D E
Mantissa ADD stage Divide pipeline stage Exception test stage
Pipeline Complications
CS510 Computer Architectures
Lecture 8 - 16
Latency and Initiation Intervals
FP Instruction Add, Subtract Multiply Divide Square root Negate Absolute value FP compare Latency 4 8 36 112 2 2 3 Initiation Interval 3 4 35 111 1 1 2
Pipeline Complications
CS510 Computer Architectures
Lecture 8 - 17
MIPS R4000 FP Pipe Stages
clock cycle Operation Multiply Add Add Add Add Add Add Add Issue/stall Issue Issue Issue Issue Stall Stall Stall Stall Issue Issue 0 U 1 M U 2 M 3 M 4 M 5 N 6 A N+ A 7 R 8 9 10 11 12
S+A A+R R+S U S+A A+R R+S U S+A A+R U R+S R +S R +S A+R R+S S+A A+R R+S
S+A A + R A U
A S + A A +R U S+A U
ADD issued at 4 cycles after Multiply will stall 2 cycles. ADD issued at 5 cycles after Multiply will stall 1 cycle.
Pipeline Complications CS510 Computer Architectures Lecture 8 - 18
R4000 Performance
Not an ideal pipeline CPI of 1:
Pipeline CPI
eqntott
espresso
Integer programs
Base Load stalls Branch stalls
Floating Point programs
FP result stalls FP structural stalls
Pipeline Complications
CS510 Computer Architectures
spice2g6
tomcatv
doduc
su2cor
nasa7
gcc
ora
4.5 4 3.5 3 2.5 2 1.5 1 0.5 0
Load stalls Branch stalls: (2 cycles for taken br. + unfilled branch slots or cancelled branch delay slots) FP result stalls: RAW data hazard (latency) FP structural stalls: Not enough FP hardware (parallelism)
li
Lecture 8 - 19
Advanced Pipeline And Instruction Level Parallelism
Pipeline Complications
CS510 Computer Architectures
Lecture 8 - 20
Advanced Pipelining and Instruction Level Parallelism
gcc 17% control transfer 5 instructions + 1 branch Beyond single block to get more instruction level parallelism Loop level parallelism is one opportunity, SW and HW
Block of Code
... Branch Target ... Branch instruction ...
Pipeline Complications
... Any instruction ... Branch instruction ...
Lecture - 8 21
CS510 Computer Architectures
Advanced Pipelining and Instruction Level Parallelism
Technique
Loop unrolling Basic pipeline scheduling Dynamic scheduling with scoreboarding Dynamic scheduling with register renaming Dynamic branch prediction Issuing multiple instructions per cycle Compiler dependence analysis Software pipelining and trace scheduling Speculation Dynamic memory disambiguation
Reduces
Control stalls RAW stalls RAW stalls WAR and WAW stalls Control stalls Ideal CPI Ideal CPI and data stalls Ideal CPI and data stalls All data and control stalls RAW stalls involving memory
Pipeline Complications
CS510 Computer Architectures
Lecture 8 - 22
Basic Pipeline Scheduling and Loop Unrolling
FP unit latencies Instruction producing result FP ALU op FP ALU op Load double* Load double* Instruction using result Another FP ALU op Store double FP ALU op Store double Latency in clock cycles 3 2 1 0
* Same as integer Load since there is a 64-bit data path from/to memory.
Fully pipelined or replicated --- no structural hazards, issue on every clock cycle for ( i =1; i <= 1000; i++) x[i] = x[i] + s;
Pipeline Complications
CS510 Computer Architectures
Lecture 8 - 23
FP Loop Hazards
Loop: LD ADDD SD SUBI BNEZ NOP F0,0(R1) F4,F0,F2 0(R1),F4 R1,R1,8 R1,Loop ;R1 is the pointer to a vector ;F2 contains a scalar value ;store back result ;decrement pointer 8B (DW) ;branch R1!=zero ;delayed branch slot
Instruction using result Another FP ALU op Store double FP ALU op Store double Integer op Latency in clock cycles 3 2 1 0 0
Instruction producing result FP ALU op FP ALU op Load double Load double Integer op
Where are the stalls?
Pipeline Complications CS510 Computer Architectures Lecture 8 - 24
FP Loop Showing Stalls
1 Loop: LD 2 stall 3 ADDD 4 stall 5 stall 6 SD 7 SUBI 8 stall 9 BNEZ 10 stall F0,0(R1) F4,F0,F2 ;F0=vector element ;add scalar in F2
0(R1),F4 R1,R1,8 R1,Loop
;store result ;decrement pointer 8B (DW) ;branch R1!=zero ;delayed branch slot
Rewrite code to minimize stalls?
Pipeline Complications CS510 Computer Architectures Lecture 8 - 25
Reducing Stalls
1 Loop: 2 3 4 5 6 7 8 9 10 LD stall ADDD stall stall SD SUBI stall BNEZ stall F0,0(R1) F4,F0,F2 For Load-ALU latency
Consider moving SUBI into this Load Delay Slot. Reading R1 by LD is done before Writing R1 by SUBI. Yes we can. When we do this, we need to change the immediate value 0 to 8 in SD
For ALU-ALU latency
There is only one instruction left, i.e., BNEZ. When we do that, SD instruction fills the delayed branch slot.
8 0(R1),F4
R1,R1,#8 R1,Loop
Pipeline Complications
CS510 Computer Architectures
Lecture 8 - 26
Revised FP Loop to Minimize Stalls
1 Loop: LD 2 SUBI 3 ADDD 4 stall 5 BNEZ 6 SD F0,0(R1) R1,R1,#8 F4,F0,F2 R1,Loop 8(R1),F4 ;delayed branch ;altered when move past SUBI Latency in clock cycles 3 2 1
Instruction producing result FP ALU op FP ALU op Load double
Instruction using result Another FP ALU op Store double FP ALU op
Unroll loop 4 times to make the code faster
Pipeline Complications CS510 Computer Architectures Lecture 8 - 27
Unroll Loop 4 Times
1 Loop: LD 2 ADDD 3 SD 4 LD 5 ADDD 6 SD 7 LD 8 ADDD 9 SD 10 LD 11 ADDD 12 SD 13 SUBI 14 BNEZ 15 NOP Rewrite loop to minimize the stalls
Pipeline Complications CS510 Computer Architectures Lecture 8 - 28
F0,0(R1) F4,F0,F2 0(R1),F4 F6,-8(R1) F8,F6,F2 -8(R1),F8 F10,-16(R1) F12,F10,F2 -16(R1),F12 F14,-24(R1) F16,F14,F2 -24(R1),F16 R1,R1,#32 R1,Loop
;drop SUBI & BNEZ ;drop SUBI & BNEZ ;drop SUBI & BNEZ
;alter to 4*8
Unrolled Loop to Minimize Stalls
1 Loop: 2 3 4 5 6 7 8 9 10 11 12 13 14 LD LD LD LD ADDD ADDD ADDD ADDD SD SD SUBI SD BNEZ SD F0,0(R1) F6,-8(R1) F10,-16(R1) F14,-24(R1) F4,F0,F2 F8,F6,F2 F12,F10,F2 F16,F14,F2 0(R1),F4 -8(R1),F8 R1,R1,#32 16(R1),F12 R1,LOOP 8(R1),F16
Assumptions - OK to move SD past SUBI even though SUBI changes R1 SUBI IF RF EX MEM WB SD IF ID EX MEM WB BNEZ IF ID EX MEM WB - OK to move loads before stores(Get right data) - When is it safe for compiler to do such changes?
; 16-32= -16 ; 8-32 = -24
14 clock cycles, or 3.5 per iteration
Pipeline Complications CS510 Computer Architectures Lecture 8 - 29
Compiler Perspectives on Code Movement
Definitions: Compiler is concerned about dependencies in the program, whether this causes a HW hazard or not depends on a given pipeline Data dependencies (RAW if a hazard for HW): Instruction j is data dependent on instruction i if either
Instruction i produces a result used by instruction j, or Instruction j is data dependent on instruction k, and instruction k is data dependent on instruction i.
Easy to determine for registers (fixed names) Hard for memory:
Does 100(R4) = 20(R6)? From different loop iterations, does 20(R6) = 20(R6)?
Pipeline Complications
CS510 Computer Architectures
Lecture 8 - 30
Compiler Perspectives on Code Movement
Name Dependence: Two instructions use the same name(register or memory location) but they do n...