Architecture
Chapter Computer 5 Memory Hierarchy Jay Lamb, Jeff Meirhofer EE 466 Spring 2009
Overview
Introduction Eleven Advanced Optimizations of Cache Performance DRAM technology Fallacies and Pitfalls
Introduction
Memory Hierarchy Principle of locality CPU registers, CPU cache, memory, I/O Bus
Ideally one would desire an indefinitely large memory capacity such that any particular ... word would be immediately available... We are ...forced to recognize the possibility of constructing a hierarchy of memories, each of which has greater capacity than the preceding but which is less quickly accessible. A.W. Burks, H.H. Goldstine, and J. von Neumann
Introduction
Processor performance vs. Memory performance
Eleven optimizations of Cache Performance
Hit time = Time to hit in the cache Miss penalty = The time to replace a block from memory Miss rate = Number of cache accesses that miss divided by the number of cache accesses
Eleven optimizations of Cache Performance
First Optimization: Small and Simple Caches to Reduce Hit Time
smaller hardware is faster data is easier to find in a simple cache
Eleven optimizations of Cache Performance
Second Optimization: Way Prediction to Reduce Hit Time
predicts next block of data to be accessed saves pipeline stages more than 85% of the time used in Pentium 4
Eleven optimizations of Cache Performance
Third Optimization: Trace Caches to Reduce Hit Time
neither simple nor popular not predicted to be widely used
Eleven optimizations of Cache Performance
Fourth Optimization: Pipelined Cache Access to Increase Cache Bandwidth
pipelined cache access high bandwidth, but slow hits
Eleven optimizations of Cache Performance
Fifth Optimization: Nonblocking Caches to Increase Cache Bandwidth
lockupfree outoforder completion
Eleven optimizations of Cache Performance
Sixth Optimization: Caches Multibanked to Increase Cache Bandwidth
cache is not treated as a single block, but many banks simultaneous access to banks is allowed
Eleven optimizations of Cache Performance
Seventh Optimization: Critical Word First and Early Restart to Reduce Miss Penalty
Critical Word First sends requested word first Early Restart fetches words in normal order, but sends requested word to processor as soon as it is fetched works best with large cache blocks
Eleven optimizations of Cache Performance
Eighth Optimization: Merging Write Buffer to Reduce Miss Penalty
Widely used (Sun Niagara) Based on efficiency of multiword writes 5% to 10% of performance lost due to stalls in a four entry write buffer
Eleven optimizations of Cache Performance
Ninth Optimization: Compiler Optimizations to Reduce Miss Rate
code and data rearrangement loop interchange blocking
Eleven optimizations of Cache Performance
Tenth Optimization: Hardware Prefetching of Instructions and Data to Reduce Miss Penalty or Miss Rate
can be done for both instructions and data utilizes unused memory bandwidth...