Hierarchy: Memory Introduction
Dr. Xiao Qin
Auburn University
http://www.eng.auburn.edu/~xqin
[e-mail]
Fall, 2010
1
Memory Systems - the Big
Picture with
Memory provides processor
Instructions
Data
Problem: memory is too slow and too small
Processor
Input
Control
Memory
Data
Datapath
Instructions
Five Classics Components Picture
2
Output
Technology Trends
Logic:
DRAM:
Disk:
Capacity Speed (latency)
2x in 3 years
2x in 3 years
4x in 3 years
2x in 10 years
4x in 3 years
2x in 10 years
DRAM
Year
Size
Cycle Time
1980
64 Kb
250 ns
1983
256 Kb
220 ns
1986
1 Mb
190 ns
1989
4 Mb
165 ns
1992
16 Mb
145 ns
1995
64 Mb
120 ns
1000:1!
3
2:1!
Why Cares About Memory Hierarchy?
Processor-DRAM Memory Gap (latency)
Performance
1000
CPU
Moores Law
100
4
Processor-Memory
Performance Gap:
(grows 50% / year)
10
DRAM
1980
1981
1982
1983
1984
1985
1986
1987
1988
1989
1990
1991
1992
1993
1994
1995
1996
1997
1998
1999
2000
1
Proc
60%/yr.
(2X/1.5yr)
Time
DRAM
9%/yr.
(2X/10 yrs)
Memory Hierarchy - the Big
Picture
Problem: memory is too slow and too small
Solution: memory hierarchy
Processor
Control
Size (bytes):
5
L1 On-Chip
Cache
Speed (ns):
Registers
Datapath
0.25-0.5
<1K
L2
Off-Chip
Cache
Main
Memory
(DRAM)
0.5-25
80-250
<16M
<16G
Secondary
Storage
(Disk)
5,000,000 (5ms)
>100G
Memory Configuration in Current PCs
Static RAM (SRAM) - used for L1,
L2 cache
Processor
L1 Cache
L2/L3
Cache
(SRAM)
System
Controller
(I/O Bus)
6
Fast - 0.5-25ns access time (less for
on-chip)
Larger, More Expensive
Higher power consumption
Main
Memory
(DRAM)
Dynamic RAM (DRAM) - used for
PC main memory
Slower - 80-250ns access time*
Smaller, Cheaper
Lower power consumption
CPU Core
1 GHz - 3.6 GHz
4-way Superscaler
RISC or RISC-core (x86):
Deep Instruction Pipelines
Dynamic scheduling
Multiple FP, integer FUs
Dynamic branch prediction
Hardware speculation
SDRAM
(possibly
PC100/PC133
on-chip)
100-133MHz
64-128 bits wide
2-way inteleaved
~ 900 MBYTES/SEC )64bit)
Double Date Rate (DDR)
SDRAM
PC3200
200 MHz DDR
64-128 bits wide
4-way interleaved
~3.2 GBYTES/SEC (64bit)
DDR2 SDRAM
667MHZ
8~16 bit wide
System Components
L1
CPU
L2
L3
Caches
All Non-blocking caches
L1 16-128K
1-2 way set associative (on chip), separate or unified
L2 256K- 2M 4-32 way set associative (on chip) unified
L3 2-16M
8-32 way set associative (on or off chip) unified
(FSB)
System Bus
Bus Adapter
Memory
Controller
Memory Bus
I/O Controllers
Memory
Chipset
North
Bridge
Examples: Alpha, AMD K7: EV6, 200-400 MHz
Intel PII, PIII: GTL+ 133 MHz
Intel P4
800 MHz
NICs
Disks
Displays
Keyboards
Chipset
South
Bridge
I/O Devices
Main I/O Bus
Example:
PCI, 33-66MHz
32-64 bits wide
133-528 MB/s
PCI-X 133MHz 64-bits wide
1066 MB/s
Networks
I/O Subsystem
Important issue: Which component creates a system performance bottleneck?
7
Levels of Memory Hierarchy
Upper Level
Capacity
Access Time
Cost
CPU Registers
100s Bytes
1s ns
Cache
K Bytes
4 ns
1-0.1 cents/bit
Main Memory
M Bytes
100ns- 300ns
$.0001-.00001 cents /bit
Disk
G Bytes, 10 ms
(10,000,000 ns)
-5 -6
10 - 10 cents/bit
Tape
infinite
sec-min
-8
10
8
Staging
Xfer Unit
faster
Registers
Instr. Operands
prog./compiler
1-8 bytes
Cache
Blocks
cache cntl
8-128 bytes
Memory
Pages
OS
512-4K bytes
Files
user/operator
Mbytes
Disk
Tape
Larger
Lower Level
COMP 4300 Computer Architecture
Block Placement
Dr. Xiao Qin
Auburn University
http://www.eng.auburn.edu/~xqin
[e-mail]
Fall, 2010
9
The Principle of Locality
The Principle of Locality:
Program access a relatively small portion of the
address space at any instant of time.
Two Different Types of Locality:
Temporal Locality (Locality in Time): If an item is
referenced, it will tend to be referenced again soon
(e.g., loops, reuse)
Spatial Locality (Locality in Space): If an item is
referenced, items whose addresses are close by tend
to be referenced soon
(e.g., straightline code, array access)
Last 15 years, HW relied on locality for speed
10
Why Hierarchy Works
The principle of locality
Programs access a relatively small portion of the
address space at any instant of time.
Probability
of reference
0
Address Space
2n - 1
Temporal locality: recently accessed data is likely
to be used again
Spatial locality: data near recently accessed data
is likely to be used soon
Result: the illusion of large, fast memory
11
Cache Operation
Insert between CPU, Main
Mem.
Implement with fast Static
RAM
Holds some of a programs
data
instructions
Operation:
Hit: Data in Cache (no penalty)
Miss: Data not in Cache (miss penalty)
12
CPU
addr
Processor
data
Cache
Memory
addr
data
DRAM
Memory
Cache Performance
Measures
Hit rate: fraction found in the cache
high So that we usually talk about Miss rate = 1 - Hit Rate
Hit time: time to access the cache
Miss penalty: time to replace a block from lower
level, including time to replace in CPU
access time: time to acccess lower level
transfer time: time to transfer block
Average memory-access time (AMAT)
= Hit time + Miss rate x Miss penalty (ns or
clocks)
13
Fundamental Questions
Q1: Where can a block be placed in the upper level?
(Block placement)
Q2: How is a block found if it is in the upper level?
(Block identification)
Q3: Which block should be replaced on a miss?
(Block replacement)
Q4: What happens on a write?
(Write strategy)
14
Q1: Block Placement
Where can block be placed in cache?
In one predetermined place - direct-mapped
Use fragment of address to calculate block location
in cache
Compare cache block with tag to test if block
present
Anywhere in cache - fully associative
Compare tag to every block in cache
In a limited set of places - set-associative
Use address fragment to calculate set (like directmapped)
Place in any block in the set
Compare tag to every block in set
Hybrid of direct mapped and fully associative
15
Direct Mapped Block Placement
Cache
*0 *4 *8 *C
address maps to block:
location = (block address MOD # blocks in cache)
00 04 08 0C 10 14 18 1C 20 24 28 2C 30 34 38 3C 40 44 48 4C
Memory
16
Example: Accessing A Direct-Mapped
Cache
DM cache contains 4 1-word blocks. Find the #
Misses for each cache given this sequence of
memory block accesses: 0, 8, 0, 6, 8
DM Memory Access 1: Mapping: 0 modulo 4 = 0
Mem Block
0
17
DM Hit/Miss
Block 0
Block 1
Block 2
Block 3
Example: Accessing A Direct-Mapped
Cache
DM cache contains 4 1-word blocks. Find the #
Misses for each cache given this sequence of
memory block accesses: 0, 8, 0, 6, 8
DM Memory DM Hit/Miss Mapping: 0 mod 4 = 0
Access 1:
Mem Block
0
miss
Block 0
Block 1
Block 2
Block 3
Mem[0]
Set 0 is empty: write Mem[0]
18
Example: Accessing A Direct-Mapped
Cache
DM cache contains 4 1-word blocks. Find the #
Misses for each cache given this sequence of
memory block accesses: 0, 8, 0, 6, 8
DM Memory Access 2: Mapping: 8 mod 4 = 0
Mem Block
DM Hit/Miss
Block 0
Mem[0]
0
miss
Block 1
8
Block 2
Block 3
19
Example: Accessing A Direct-Mapped
Cache
DM cache contains 4 1-word blocks. Find the #
Misses for each cache given this sequence of
memory block accesses: 0, 8, 0, 6, 8
DMMem Block Access 2: Mapping: 8 mod 4 = 0
Memory DM Hit/Miss
Block 0
Mem[8]
0
miss
Block 1
8
miss
Block 2
Block 3
Set 0 contains Mem[0]. Overwrite with Mem[8]
20
Example: Accessing A Direct-Mapped
Cache
DM cache contains 4 1-word blocks. Find the #
Misses for each cache given this sequence of
memory block accesses: 0, 8, 0, 6, 8
DMMem Block Access 3: Mapping: 0 mod 4 = 0
Memory DM Hit/Miss
Block 0
Mem[8]
0
miss
Block 1
8
miss
0
Block 2
Block 3
21
Example: Accessing A Direct-Mapped
Cache
DM cache contains 4 1-word blocks. Find the #
Misses for each cache given this sequence of
memory block accesses: 0, 8, 0, 6, 8
DMMem Block Access 3: Mapping: 0 mod 4 = 0
Memory DM Hit/Miss
Block 0
Mem[0]
0
miss
Block 1
8
miss
0
miss
Block 2
Block 3
Set 0 contains Mem[8]. Overwrite with Mem[0]
22
Example: Accessing A Direct-Mapped
Cache
DM cache contains 4 1-word blocks. Find the #
Misses for each cache given this sequence of
memory block accesses: 0, 8, 0, 6, 8
DMMem Block Access 4: Mapping: 6 mod 4 = 2
Memory DM Hit/Miss
Block 0
Mem[0]
0
miss
Block 1
8
miss
0
miss
Block 2
6
Block 3
23
Example: Accessing A Direct-Mapped
Cache
DM cache contains 4 1-word blocks. Find the #
Misses for each cache given this sequence of
memory block accesses: 0, 8, 0, 6, 8
DMMem Block Access 4: Mapping: 6 mod 4 = 2
Memory DM Hit/Miss
Block 0
Mem[0]
0
miss
Block 1
8
miss
0
miss
Block 2
Mem[6]
6
miss
Block 3
Set 2 empty. Write Mem[6]
24
Example: Accessing A Direct-Mapped
Cache
DM cache contains 4 1-word blocks. Find the #
Misses for each cache given this sequence of
memory block accesses: 0, 8, 0, 6, 8
DMMem Block Access 5: Mapping: 8 mod 4 = 0
Memory DM Hit/Miss
Block 0
Mem[0]
0
miss
Block 1
8
miss
0
miss
Block 2
Mem[6]
6
miss
Block 3
8
25
Example: Accessing A Direct-Mapped
Cache
DM cache contains 4 1-word blocks. Find the #
Misses for each cache given this sequence of
memory block accesses: 0, 8, 0, 6, 8
DMMem Block Access 5: Mapping: 8 mod 4 = 0
Memory DM Hit/Miss
Block 0
Mem[8]
0
miss
Block 1
8
miss
0
miss
Block 2
Mem[6]
6
miss
Block 3
8
miss
Set 0 contains Mem[0]. Overwrite with Mem[8]
26
Direct-Mapped Cache with n one-word
blocks
Pros: find data fast
Con: What if access 00001 and 10001
repeatedly?
We always miss
00 00 1
0 0101
0100 1
0 1101
10 00 1
Me m ory
27
11
1
10
1
11
0
10
0
01
1
00
1
01
0
00
0
Ca ch e
1 01 01
1 1 00 1
11 1 01