cture13: Le C acheand Virtual Me roy Re w m vie
C acheoptim ization approache cachem s, iss classification,
Adapted from UCB CS252 S01
1
What I s Me ory Hie m rarchy
A typical m m hie e ory rarchy today:
Proc/Regs L1-Cache Bigger L2-Cache L3-Cache (optional) Memory Disk, Tape, etc. Faster
He wefocus on L1/L2/L3 cache and m m m re s ain e ory
2
Why Me ory Hie m rarchy?
1000 Performance
"Moore's Law"
CPU
Proc 60%/yr.
100 10 1
Processor-Memory Performance Gap: (grows 50% / year) DRAM DRAM 7%/yr.
1980 1981 1982 1983 1984 1985 1986 1987 1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000
1980: no cachein proc; 1995 2-le l cacheon chip ve (1989 first I nte proc with a cacheon chip) l
3
Ge rations of Microproce ne ssors
Tim of a full cachem in instructions e cute e iss xe d: 1st Alpha: 340 ns/5.0 ns = 68 clks x 2 or 2nd Alpha: 266 ns/3.3 ns = 80 clks x 4 or 3rd Alpha: 180 ns/1.7 ns =108 clks x 6 or 1/2X late x 3X clock ratex 3X I nstr/clock 4.5X ncy 136 320 648
4
Are C of C s a osts ache
%Are a % Transistors (-cost) (-powe r) I nte 80386 l 0% 0% Alpha 21164 37% 77% S trongArmS A110 61% 94% Pe ntiumPro 64% 88% 2 die pe package Proc/I $/D$ + L2$ s r : I tanium 92% C s storere ache dundant data only to closepe rform ancegap
5
Proce ssor
What I s C , Exactly? ache
S all, fast storageuse to im m d proveave acce tim to slow m m rage ss e e ory; usually m by S ade RAM Exploits locality: spatial and te poral m I n com r archite pute cture alm e rything is a cache , ost ve !
Re r fileis thefaste placeto cachevariable giste st s First-le l cachea cacheon se ve cond-le l cache ve S cond-le l cachea cacheon m m e ve e ory Me ory a cacheon disk (virtual m m m e ory) TLB a cacheon pagetable Branch-pre diction a cacheon pre diction inform ation? Branch-targe buffe can beim m nte as cache t r ple e d
Be yond archite cture filecache browse cache proxy cache : , r , He wefocus on L1 and L2 cache (L3 optional) as buffe to m m m re s rs ain e ory
6
Exam : 1 KB Dire Mappe C ple ct d ache
Assum a cacheof 2N byte 2K blocks, block sizeof 2M byte N = M+K (#block tim s e s, s; e block size ) (32 - N)-bit cachetag, K-bit cacheinde and M-bit cache x, Thecachestore tag, data, and valid bit for e block s ach C acheinde is use to se ct a block in S x d le RAM (Re BHT, BTB) call Block tag is com pare with theinput tag d A word in thedata block m bese cte as theoutput ay le d
31 Tag Block address Example: 0x50 Stored as part of the cache "state" Valid Bit Cache Tag 0x50 Cache Data Byte 31 Byte 63 : 9 Index Ex: 0x01 4 Block offset Ex: 0x00 0
Byte 1 Byte 33
Byte 0 Byte 32
0 1 2 3
:
:
:
:
Byte 1023
Byte 992
31
7
:
For Que stions About CacheDe sign
Block place e Whe can a block beplace m nt: re d? Block ide ntification: How to find a block in thecache ? Block re place e I f a ne block is to befe d, which of m nt: w tche e xisting blocks to re place (if the arem ? re ultiplechoice s Writepolicy: What happe on a write ns ?
8
Whe C A Block BePlace re an d
What is a block: dividem m spaceinto blocks as cacheis e ory divide d
A m m block is thebasic unit to becache e ory d
Dire m d cache the is only oneplacein thecacheto ct appe : re buffe a give m m block r n e ory N-way se associativecache N place for a give m m t : s n e ory block
LikeN dire m d cache ope ct appe s rating in paralle l Re ducing m rate with incre d com xity, cacheacce tim , iss s ase ple ss e and powe consum r ption
Fully associativecache a m m block can beput : e ory anywhe in thecache re
9
S t AssociativeC e ache
Exam : Two-way se associativecache ple t C acheinde se cts a se of two blocks x le t Thetwo tags in these arecom t pare to theinput in paralle d l Data is se cte base on thetag com le d d parison
S t associativeor dire m d? Discuss late e ct appe r
Valid Cache Tag Cache Index Cache Data Cache Data Cache Block 0 Cache Block 0 Cache Tag Valid
:
Adr Tag
:
:
:
:
:
Compare
Sel1 1
Mux
0 Sel0
Compare
OR
Hit
Cache Block
10
How to Find a C d Block ache
Dire m d cache thestore tag for thecacheblock ct appe : d m atche theinput tag s Fully associativecache any of thestore N tags m : d atche s theinput tag S t associativecache any of thestore K tags for thecache e : d se m t atche theinput tag s C achehit tim is de d by both tag com e cide parison and data acce C bede rm d by C Mode ss an te ine acti l
11
Which Block to Re place ?
Dire m d cache Not an issue ct appe : For se associativeor fully associative cache t * :
Random S le candidateblocks random fromthecachese : e ct ly t LRU (Le Re ntly Use Re ast ce d): placetheblock that has be n e unuse for thelonge tim d st e FI FO (First I n, First Out): Re placetheolde block st
Usually LRU pe rform thebe but hard (and e nsive s st, xpe ) to im m nt ple e
*Think fully associativecacheas a se associativeonewith a singlese t t
12
What Happe on Write ns s
Whe to writethedata if theblock is found in cache re ? Writethrough: ne data is writte to both thecacheblock and thelowe w n rle l m m ve e ory He to m lp aintain cacheconsiste ncy Writeback: ne data is writte only to thecacheblock w n Lowe r-le l m m is update whe theblock is re ve e ory d n place d A dirty bit is use to indicatethene ssity d ce He to re lp ducem m traffic e ory What happe if theblock is not found in cache ns ? Writeallocate Fe theblock into cache the writethedata (usually : tch , n com d with writeback) bine No-writeallocate Do not fe theblock into cache(usually com d with : tch writethrough)
bine 13
Re Exam : Alpha 21264 C s al ple ache
64KB 2-way associative instruction cache 64KB 2-way associative data cache
Icache
Dcache
14
Alpha 21264 Data C ache
D-cache 64K 2-way : associative
Use48-bit virtual addre to ss inde cache usetag from x , physical addre ss 48-bit Virtual=>44-bit addre ss 512 block (9-bit blk inde x) C acheblock size64 byte (6-bit s offse t)t Tag has 44-(9+6)=29 bits Write back and writeallocate d (Wewill study virtual-physical addre translation) ss
15
C achepe rform ance
C alculateave m m acce tim (AMAT) rage e ory ss e AMAT = hit time + Miss rate Miss penalty
Exam : hit tim = 1 cycle m tim = 100 cycle m rate= 4% ple e , iss e , iss , than AMAT = 1+100*4%= 5
C alculatecacheim pact on proce pe ssor rform ance
CPU time = (CPU execution cycles + Memory stall cycles) Cycle time Memory Stall Cycles CPU time = IC CPI execution + CycleTime Notecycle spe on cachehit is usually counte into e cution Instruction d s nt xe
cycle s
16
Disadvantageof S t AssociativeCache e
C paren-way se associativewith dire m d cache om t ct appe : Has n com parators vs. 1 com parator Has Extra MUX de lay for thedata Data com s afte hit/m de e r iss cision and se se ction t le I n a dire m d cache cacheblock is availablebe hit/m de ct appe , fore iss cision Usethedata assum ing theacce is a hit, re r if found othe ss cove rwise
Cache Index Cache Data Cache Data Cache Block 0 Cache Block 0
Valid
Cache Tag
Cache Tag
Valid
:
:
:
:
:
:
Adr Tag
Compare
Sel1 1
Mux
0 Sel0
Compare
OR
Hit
Cache Block
17
Virtual Me ory m
Virtual m m (VM) allows program to havetheillusion of a ve large e ory s ry m m that is not lim d by physical m m size e ory ite e ory
Makem m m (DRAM) acts likea cachefor se ain e ory condary storage(m tic disk) agne Othe rwise application program e haveto m data in/out m m m , m rs ove ain e ory That's how virtual m m was first propose e ory d Allowing m ultipleproce s sharethephysical m m in m sse e ory ultiprogram ing m e nvironm nt e Providing prote ction for proce s (com sse pareInte 8086: without VM applications l can ove rwriteOSke l) rne Facilitating programre location in physical m m space e ory
Virtual m m also provide thefollowing functions e ory s
VM Exam ple
19
Virtual Me ory and C m ache
VM addre translation a provide a m ss s apping fromthevirtual addre of theproce to thephysical addre in m m m and ss ssor ss ain e ory se condary storage . C achete s vs. VM te s rm rm C acheblock => page C acheMiss => pagefault
Tasks of hardwareand OS
TLB doe fast addre translations s ss OShandle le fre ntly e nts: s ss que ve
pagefault TLB m (whe softwareapproach is use iss n d)
20
Virtual Me ory and C m ache
Parameter Block (page) size Hit time Miss Penalty Miss rate Address mapping L1 Cache 16-128 bytes 1-3 cycles 8-300 cycles 0.1-10% Main Memory 4KB 64KB 50-150 cycles 1M to 10M cycles 0.00001-0.001%
25-45 bits => 13-21 32-64 bits => 25-45 bits bits
4 Qs for Virtual Me ory m
Q1: Whe can a block beplace in theuppe le l? re d r ve
Miss pe nalty for virtual m m is ve high => Full associativity is e ory ry de sirable(so allow blocks to beplace anywhe in them m d re e ory) Havesoftwarede rm te inethelocation whileacce ssing disk (10M cycle...