Six
1998 Chapter Morgan Kaufmann Publishers
1
Pipelining
Improve perfomance by increasing instruction throughput
Program execution Time order (in instructions) lw $1, 100($0) lw $2, 200($0) lw $3, 300($0) 2 4 6 8 10 12 14 16 18
Instruction Reg fetch
ALU
Data access
Reg Instruction Reg fetch ALU Data access Reg Instruction fetch
8 ns
8 ns
8 ns Program execution Time order (in instructions) lw $1, 100($0) lw $2, 200($0) lw $3, 300($0) 2 4 6 8 10 12 14
...
Instruction fetch
Reg Instruction fetch
ALU Reg Instruction fetch
Data access ALU Reg
Reg Data access ALU Reg Data access Reg
2 ns
2 ns
2 ns
2 ns
2 ns
2 ns
2 ns
Ideal speedup is number of stages in the pipeline. Do we achieve this?
1998 Morgan Kaufmann Publishers
2
Pipelining
What makes it easy all instructions are the same length just a few instruction formats memory operands appear only in loads and stores What makes it hard? structural hazards: suppose we had only one memory control hazards: need to worry about branch instructions data hazards: an instruction depends on a previous instruction Well build a simple pipeline and look at these issues Well talk about modern processors and what really makes it hard: exception handling trying to improve performance with out-of-order execution, etc.
1998 Morgan Kaufmann Publishers
3
Basic Idea
IF: Instruction fetch
0 M u x 1
ID: Instruction decode/ register file read
EX: Execute/ address calculation
MEM: Memory access
WB: Write back
Add 4 Shift left 2 Read register 1 Add Add result
PC
Address
Instruction Instruction memory
Read data 1 Read register 2 Registers Read Write data 2 register Write data
0 M u x 1
Zero ALU ALU result
Address Data memory Write data
Read data
1 M u x 0
16
Sign extend
32
What do we need to add to actually split the datapath into stages?
1998 Morgan Kaufmann Publishers
4
Pipelined Datapath
0 M u x 1
IF/ID Add 4
ID/EX
EX/MEM
MEM/WB
Add Add result Shift left 2 Instruction Read register 1
PC
Address Instruction memory
Read data 1 Read register 2 Registers Read Write data 2 register Write data
0 M u x 1
Zero ALU ALU result
Address Data memory Write data
Read data
1 M u x 0
16
Sign extend
32
Can you find a problem even if there are no dependencies? What instructions can we execute to manifest the problem?
1998 Morgan Kaufmann Publishers
5
Corrected Datapath
0 M u x 1
IF/ID Add 4
ID/EX
EX/MEM
MEM/WB
Add Shift left 2 Instruction Read register 1
Add result
PC
Address Instruction memory
Read data 1 Read register 2 Registers Read Write data 2 register Write data
0 M u x 1
Zero ALU ALU result
Address Data memory Write data
Read data
1 M u x 0
16
Sign extend
32
1998 Morgan Kaufmann Publishers
6
Graphically Representing Pipelines
Time (in clock cycles) Program execution order (in instructions) lw $10, 20($1) CC 1 CC 2 CC 3 CC 4 CC 5 CC 6
IM
Reg
ALU
DM
Reg
sub $11, $2, $3
IM
Reg
ALU
DM
Reg
Can help with answering questions like: how many cycles does it take to execute this code? what is the ALU doing during cycle 4? use this representation to help understand datapaths
1998 Morgan Kaufmann Publishers
7
Pipeline Control
PCSrc 0 M u x 1
IF/ID Add 4
RegWrite
ID/EX
EX/MEM
MEM/WB
Add Shift left 2
Add result
Branch
PC
Address Instruction memory
Instruction
Read register 1
MemWrite Read data 1
ALUSrc
Read register 2 Registers Read Write data 2 register Write data
0 M u x 1
Zero Zero ALU ALU result
MemtoReg Address Data memory Write data Read data 1 M u x 0
Instruction 16 [15 0]
Sign extend
32
6
ALU control
MemRead
Instruction [20 16] Instruction [15 11]
0 M u x 1
RegDst
ALUOp
1998 Morgan Kaufmann Publishers
8
Pipeline control
We have 5 stages. What needs to be controlled in each stage? Instruction Fetch and PC Increment Instruction Decode / Register Fetch Execution Memory Stage Write Back How would control be handled in an automobile plant? a fancy control center telling everyone what to do? should we use a finite state machine?
1998 Morgan Kaufmann Publishers
9
Pipeline Control
Pass control signals along just like the data
Execution/Address Calculation Memory access stage stage control lines control lines Reg ALU ALU ALU Mem Mem Dst Op1 Op0 Src Branch Read Write 1 1 0 0 0 0 0 0 0 0 1 0 1 0 X 0 0 1 0 0 1 X 0 1 0 1 0 0 Write-back stage control lines Reg Mem to write Reg 1 0 1 1 0 X 0 X
Instruction R-format lw sw beq
WB Instruction M EX WB M WB
Control
IF/ID
ID/EX
EX/MEM
MEM/WB
1998 Morgan Kaufmann Publishers
10
Datapath with Control
PCSrc ID/EX WB Control M EX EX/MEM WB M MEM/WB WB
0 M u x 1
IF/ID Add 4 RegWrite
Add Add result MemWrite Shift left 2 ALUSrc Branch
PC
Address Instruction memory
Read data 1 Read register 2 Registers Read Write data 2 register Write data
0 M u x 1
Zero ALU ALU result
Address Data memory Write data
Read data
Instruction 16 0]
Sign [15 extend
32
6
ALU control
MemRead
Instruction [20 16] Instruction [15 11]
0 M u x 1 RegDst
ALUOp
MemtoReg 1 M u x 0
Instruction
Read register 1
1998 Morgan Kaufmann Publishers
11
Dependencies
Problem with starting next instruction before first is finished dependencies that go backward in time are data hazards
Time (in clock cycles) CC 1 Value of register $2: 10 Program execution order (in instructions) sub $2, $1, $3 IM Reg DM Reg CC 2 10 CC 3 10 CC 4 10 CC 5 10/ 20 CC 6 20 CC 7 20 CC 8 20 CC 9 20
and $12, $2, $5
IM
Reg
DM
Reg
or $13, $6, $2
IM
Reg
DM
Reg
add $14, $2, $2
IM
Reg
DM
Reg
sw $15, 100($2)
IM
Reg
DM
Reg
1998 Morgan Kaufmann Publishers
12
Software Solution
Have compiler guarantee no hazards Where do we insert the nops ? sub and or add sw $2, $1, $3 $12, $2, $5 $13, $6, $2 $14, $2, $2 $15, 100($2)
Problem: this really slows us down!
1998 Morgan Kaufmann Publishers
13
Forwarding
Use temporary results, dont wait for them to be written register file forwarding to handle read/write to same register ALU forwarding
Time (in clock cycles) CC 1 Value of register $2 : 10 Value of EX/MEM : X Value of MEM/WB : X Program execution order (in instructions) sub $2, $1, $3 IM Reg DM Reg CC 2 10 X X CC 3 10 X X CC 4 10 20 X CC 5 10/ 20 X 20 CC 6 20 X X CC 7 20 X X CC 8 20 X X CC 9 20 X X
and $12, $2, $5
IM
Reg
DM
Reg
or $13, $6, $2
IM
Reg
DM
Reg
add $14, $2, $2
IM
Reg
DM
Reg
sw $15, 100($2)
IM
Reg
DM
Reg
what if this $2 was $13?
1998 Morgan Kaufmann Publishers
14
Forwarding
ID/EX WB Control IF/ID M EX EX/MEM WB M MEM/WB WB
Instruction
M u x Registers ALU M u x IF/ID.RegisterRs IF/ID.RegisterRt IF/ID.RegisterRt IF/ID.RegisterRd Rs Rt Rt Rd M u x Forwarding unit EX/MEM.RegisterRd Data memory
PC
Instruction memory
M u x
MEM/WB.RegisterRd
1998 Morgan Kaufmann Publishers
15
Can't always forward
Load word can still cause a hazard: an instruction tries to read a register following a load instruction that writes to the same register.
Time (in clock cycles) Program CC 1 execution order (in instructions) lw $2, 20($1) IM CC 2 CC 3 CC 4 CC 5 CC 6 CC 7 CC 8 CC 9
Reg
DM
Reg
and $4, $2, $5
IM
Reg
DM
Reg
or $8, $2, $6
IM
Reg
DM
Reg
add $9, $4, $2
IM
Reg
DM
Reg
slt $1, $6, $7
IM
Reg
DM
Reg
Thus, we need a hazard detection unit to stall the load instruction
1998 Morgan Kaufmann Publishers
16
Stalling
We can stall the pipeline by keeping an instruction in the same stage
Program Time (in clock cycles) execution CC 1 CC 2 order (in instructions) lw $2, 20($1) IM Reg CC 3 CC 4 CC 5 CC 6 CC 7 CC 8 CC 9 CC 10
DM
Reg
and $4, $2, $5
IM
Reg
Reg
DM
Reg
or $8, $2, $6
IM
IM
Reg
DM
Reg
bubble
add $9, $4, $2 IM Reg DM Reg
slt $1, $6, $7
IM
Reg
DM
Reg
1998 Morgan Kaufmann Publishers
17
Hazard Detection Unit
Stall by letting an instruction that wont write anything go forward
Hazar...