431 CSE Computer Architecture Fall 2005 Lecture 12: SS Front End (Fetch , Decode & Dispatch)
Mary Jane Irwin ( www.cse.psu.edu/~mji ) www.cse.psu.edu/~cg431
[Adapted from Computer Organization and Design, Patterson & Hennessy, 2005 and Superscalar Microprocessor Design, Johnson, 1992]
CSE431 L12 SS Front End.1 Irwin, PSU, 2005
FETCH
In Order Fetch multiple instructions
CSE431 L12 SS Front End.2
SS Pipeline
DECODE & DISPATCH
In Order Out of Order In Order 2nd RUU function (copy Result Bus data to matching src's and to RUU dst entry)
Decode instructions 1st RUU function (RUU allocation, src operand copying (or Tag field set), dst Tag field set)
ISSUE & EXECUTE WRITE BACK RESULT COMMIT 4th RUU function (for instr at RUU_Head (if executed), write dst Contents to RegFile)
3nd RUU function (when both src operands Ready and FU free, schedule Result Bus and issue instr for execution)
Irwin, PSU, 2005
Instruction Fetch Sequences
Instruction run number of instructions (run length) fetched between taken branches
q
Instruction fetcher operates most efficiently when processing long runs unfortunately runs are usually quite short 4-way Instr Fetcher
Time (cycles) Branch delay
The average run length is about six instructions Instruction bandwidth of only 1.125 instructions per cycle
q
S1 S4 S5
S2
S3
T1 T4
T2
T3
9 instructions in 8 cycles
Irwin, PSU, 2005
CSE431 L12 SS Front End.3
Instruction Fetch Inefficiencies
Fetcher can't provide adequate bandwidth to the decoder to exploit the available ILP because
q
Decoder is idle while the outcome of the branch is determined
- Can (mostly) fix with dynamic branch prediction
q
Instruction fetch misalignment prevents the decoder from operating at full capacity even when the decoder is processing valid instructions
- The fetcher can align fetched instructions to avoid wasted decoder slots - If supported by dynamic branch prediction, the fetcher can also merge instructions from different runs
Aligning and merging can only be done if the fetcher has the sufficient bandwidth (i.e., the fetch rate is faster than the decode rate)
Irwin, PSU, 2005
CSE431 L12 SS Front End.4
Speedups of Fetch Alternatives
4 3
From Johnson, 1992
Speedup
Low HM High
q
2 1
Base: no prediction and no alignment Pred: dynamic branch prediction
q
0 2base 4base 2pred 42-max 4-max pred
A 4-way instr fetcher out performs a 2-way instr fetcher
q q
It has twice the potential instruction bandwidth But it requires twice as much decoder hardware to keep up (e.g., in decoders and in ports and buses)
Irwin, PSU, 2005
CSE431 L12 SS Front End.5
4-Way Decoder Implementation
A 4-way instr fetcher has higher fetch bandwidth but at what cost ?
q
12 dependency checks between the 4 decode instruction
op rs rt rd
op rs rt rd
op rs rt rd
op rs rt rd
q
8 read ports on the RegFile and 8 write ports to the RUU and 8 buses to distribute those source operands (or their RegFile addr || LI)
Irwin, PSU, 2005
CSE431 L12 SS Front End.6
Reducing 4-Way Decoder Hardware
Limiting the number of RegFile read ports, buses and RUU write ports is acceptable since
q q q
Not all decoded instructions access two registers Not all decoded instruction are valid (because of misalignment) Some decoded instructions have dependences on one or more simultaneously decoded instructions
0.35 0.3
From Johnson, 1992
ccom troff
Fraction of total decoded parcels
The register demand means a 8 port capacity would be wasted 4 ports reduces average performance < 2%
0.25 0.2 0.15 0.1 0.05 0 0 1 2 3 4 5 6 7
8
CSE431 L12 SS Front End.7
# of Ports Used
Irwin, PSU, 2005
Arbitrating Read Port and Bus Usage
If only 4 read ports and 4 buses are provided, have to determine which instr's get first access to them
q
If more than 4 ports are needed to dispatch the decoded instructions, then instruction fetch and decode must stall
Prioritized register identifier selection for port usage must be accomplished within about half a processor cycle
q
If the first decoder position instr (i.e., the first instr in program order) requires register access, it is always enabled on the first and second, if it has two source operands, ports Such arbitration continues in sequence for the second, third, and fourth decoder position instr's
q
CSE431 L12 SS Front End.8
Irwin, PSU, 2005
SS Branch Prediction
Recall from Lecture 8 for branch prediction in a scalar pipeline we needed
q
A mechanism to predict the branch outcome: a BHT (branch history table) in the fetch stage A way to fetch two instructions the sequential instruction (I$) and the branch target instruction (BTB (branch target buffer)) A way to ensure that instructions active in the pipeline following the branch didn't change the machine state until the branch outcome was known
- Allowed to complete (in order commit) on correct prediction - Flushed on mispredict and restart
q
q
With a SS machine, it is possible to have many such instructions after predicted branches active in the pipeline
q
Flag instructions following branches as speculative until the branch outcome is known
Irwin, PSU, L12 2005
CSE431 SS Front End.9
Implementing Branches
A SS processor could have more than one branch per fetch set and could have several uncompleted branches pending I$ at any time Fetch (BHT/BTB)
Branch (check predict) Decode (Predict) Dispatch
Must access BHT/BTB for all branch instr's in the fetch set during fetch to reduced branch delay (i.e., need a 4 read-port BHT/BTB for 4-instr fetcher)
q q
Pass BHT information to decode stage After decode, choose between I$ set and BTB sets to determine the next fetch set
Irwin, PSU, 2005
CSE431 L12 SS Front End.10
Decoding & Dispatching Branches
While multiple branches could be dispatched per cycle, incur only a slight performance decrease (about 2%) from imposing a decoder limit of one branch per fetch set since typically only one branch per cycle can be executed (usually only have one branch FU) Having minimum branch delay is more important that decoding multiple branches per cycle
3 2.5
From Johnson, 1992
Low HM High
Speedup
2 1.5 1 0.5 0 4-sinpred 4-mulpred
CSE431 L12 SS Front End.11
Irwin, PSU, 2005
Speculative Instructions
Speculation The processor (or compiler) guesses the outcome of an instruction (e.g., branches, loads) so as to enable execution of other instructions that depend on the speculated instruction
q
One of the most important methods for finding more ILP in SS and VLIW processors
Producing correct results requires result checking, recovery and restart hardware mechanisms
q q
Checking mechanisms to see if the prediction was correct Recovery mechanisms to cancel the effects of instructions that were issued under false assumptions (e.g., branch misprediction) Restart mechanisms to reestablish the correct instruction sequence
- For branches the correct program counter restart value is known when the branch outcome is determined
q
CSE431 L12 SS Front End.12
Irwin, PSU, 2005
RUU Speculation Field Support
speculative src operand 1
src operand 2
destination
issued functional unit executed
For dependent speculative instr's, the speculative flag is set to Yes until the outcome of the driving instr (i.e., the branch) is determined. Then an associate comparison of that branch's PC addr and the RUU's SIA fields can be done.
PC
Spec Instr Addr
CSE431 L12 SS Front End.13
Yes/No Unit Number Yes/No
Content
Content
Content
Irwin, PSU, 2005
Address
Yes/No
Ready
Ready
Tag
Tag
Tag
Branch Execution
If the branch was not mispredicted, then the branch and its trailing instructions can commit when at RUU_Head If the branch was mispredicted, then all subsequent instr's must be discarded (even though subsequent branches may have been correctly predicted) When there is an exception, all of the RUU entries are discarded in a single cycle and instruction stream fetching restarts on the next cycle. Thus, the RUU provides an easy way to discard instructions coming after a mispredicted branch.
CSE431 L12 SS Front End.14
Irwin, PSU, 2005
Effects of RUU Size on Performance
Since instruction decoding must stall when there is no free RUU entry, the RUU should be large enough to accept all instructions during the expected dispatch-tocommit time period Performance decreases markedly with 8 and 4 entries For 2 entries the performance is worse than the scalar processor
q
3
From Johnson, 1992
Low HM High
Speedup
2
1
0 32 16 8 4 2
Why?
Number of IROB Entries (4-instr decoder)
Irwin, PSU, 2005
CSE431 L12 SS Front End.15
Effects of LSQ Size on Performance
Since instruction decoding must stall when there is no free LSQ entry, the LSQ should be large enough but the LSQ size has relatively little impact on performance
q
With a 4-way decoder, a 4-entry LSQ only incurs a 1% speedup loss over an 8-entry LSQ Smaller LSQ facilitate dependency checking
3
q
From Johnson, 1992
Speedup
2
1
Low HM High
0 8
CSE431 L12 SS Front End.16
4 # of Entries
2
Irwin, PSU, 2005
CSE431 L12 SS Front End.17
FETCH
SS Fetch and Decode Pipeline Stages
In Order DECODE & DISPATCH In Order Out of Order Decode 4 instructions 1st RUU function (RUU allocation, src operand copying (or Tag field set), dst Tag field set) If branch, use BHT info to determine next PC value and whether the I$ or BTB instr's are the next fetch set ISSUE & EXECUTE 3nd RUU function (when both src operands Ready and FU free, schedule Result Bus and issue instr for execution)
Fetch 4 instructions Align and merge Access BHT/BTB for PC, PC+4, PC+8, PC+12
Irwin, PSU, 2005
Next Lecture and Reminders
Next lecture
q
MIPS superscalar back end (instruction executed and commit)
- Reading assignment Sohi paper, Johnson Chapter 6, 7 & 8 (opt)
Reminders
q q
HW3, Part 1 due October 13th (due IN CLASS) HW3, Part 2 (the SimpleScalar experiments) due Friday, October 21th (email solution to the TA, [e-mail]) by 5:00pm Evening midterm exam scheduled (one week away)
- Tuesday, October 18th, 20:15 to 22:15, Location 113 IST - Only one student has contacted me about a conflict (you know who you are) so that is the only conflict exam that will be scheduled
q
CSE431 L12 SS Front End.18
Irwin, PSU, 2005