284H JDEP Foundations of Computer Systems
Giving credit where credit is due
Most of slides for this lecture are based on slides created by Drs. Bryant and O'Hallaron, Carnegie Mellon University. I have modified them and added new slides.
The Memory Hierarchy
Dr. Steve Goddard [e-mail]
http://cse.unl.edu/~goddard/Courses/JDEP284
2
Topics
Storage technologies and trends Locality of reference Caching in the memory hierarchy
Random-Access Memory (RAM)
Key features
RAM is packaged as a chip. Basic storage unit is a cell (one bit per cell). Multiple RAM chips form a memory.
Static RAM (SRAM) (SRAM)
Each cell stores bit with a six-transistor circuit. Retains value indefinitely, as long as it is kept powered. Relatively insensitive to disturbances such as electrical noise. Faster and more expensive than DRAM.
Dynamic RAM (DRAM) (DRAM)
Each cell stores bit with a capacitor and transistor. Value must be refreshed every 10-100 ms. Sensitive to disturbances. Slower and cheaper than SRAM.
3 4
SRAM vs DRAM Summary
Conventional DRAM Organization
d x w DRAM:
dw total bits organized as d supercells of size w bits
Tran. per bit SRAM DRAM 6 1
Access time Persist? Sensitive? 1X 10X Yes No No Yes
16 x 8 DRAM chip
Cost 100x 1X
Applications
0 1
cols 2 3
2 bits /
cache memories Main memories, frame buffers
memory controller (to CPU)
0 1 rows 2 3 supercell (2,1)
addr
8 bits /
data
internal row buffer
5 6
Page 1
Reading DRAM Supercell (2,1)
Step 1(a): Row access strobe (RAS) selects row 2. (RAS) Step 1(b): Row 2 copied from DRAM array to row buffer.
16 x 8 DRAM chip cols RAS = 2
2 /
Reading DRAM Supercell (2,1)
Step 2(a): Column access strobe (CAS) selects column 1. (CAS) Step 2(b): Supercell (2,1) copied from buffer to data lines, and eventually back to the CPU.
16 x 8 DRAM chip cols 0 0 1 memory controller supercell (2,1)
8 /
0 0 1
1
2
3
CAS = 1
2 /
1
2
3
addr memory controller
8 /
To CPU
addr rows 2 3
rows 2 3
data
data
internal row buffer
7
supercell (2,1)
internal row buffer
8
Memory Modules
addr (row = i, col = j) : supercell (i,j)
DRAM 0
Enhanced DRAMs
All enhanced DRAMs are built around the conventional DRAM core.
Fast page mode DRAM (FPM DRAM)
Access contents of row with [RAS, CAS, CAS, CAS, CAS] instead of [(RAS,CAS), (RAS,CAS), (RAS,CAS), (RAS,CAS)].
DRAM 7
64 MB memory module consisting of eight 8Mx8 DRAMs
Extended data out DRAM (EDO DRAM)
Enhanced FPM DRAM with more closely spaced CAS signals.
Synchronous DRAM (SDRAM)
bits bits bits bits bits bits bits 56-63 48-55 40-47 32-39 24-31 16-23 8-15 bits 0-7
Driven with rising clock edge instead of asynchronous control signals.
Double data-rate synchronous DRAM (DDR SDRAM)
63 56 55 48 47 40 39 32 31 24 23 16 15 8 7 0
64-bit doubleword at main memory address A
Memory controller
Enhancement of SDRAM that uses both clock edges as control signals.
Video RAM (VRAM)
Like FPM DRAM, but output is produced by shifting row buffer Dual ported (allows concurrent reads and writes)
9 10
64-bit doubleword
Nonvolatile Memories
DRAM and SRAM are volatile memories
Lose information if powered off.
Typical Bus Structure Connecting CPU and Memory
A bus is a collection of parallel wires that carry address, data, and control signals. Buses are typically shared by multiple devices.
CPU chip register file ALU system bus memory bus
Nonvolatile memories retain value even if powered off.
Generic name is read-only memory (ROM). Misleading because some ROMs can be read and modified.
Types of ROMs
Programmable ROM (PROM) Eraseable programmable ROM (EPROM) Electrically eraseable PROM (EEPROM) Flash memory
Firmware
Program stored in a ROM
Boot time code, BIOS (basic input/ouput system) graphics cards, disk controllers.
11
bus interface
I/O bridge
main memory
12
Page 2
Memory Read Transaction (1)
CPU places address A on the memory bus.
Memory Read Transaction (2)
Main memory reads A from the memory bus, retreives word x, and places it on the bus.
register file %eax ALU main memory 0
x
register file %eax ALU
Load operation: movl A, %eax
Load operation: movl A, %eax
I/O bridge bus interface
A
main memory 0
x
I/O bridge bus interface
x
A
A
13
14
Memory Read Transaction (3)
CPU reads word x from the bus and copies it into register %eax. %eax.
register file %eax
x
Memory Write Transaction (1)
CPU places address A on bus. Main memory reads it and waits for the corresponding data word to arrive.
register file %eax
y
Load operation: movl A, %eax ALU main memory 0
x
Store operation: movl %eax, A ALU main memory 0 A
I/O bridge bus interface
I/O bridge bus interface
A
A
15
16
Memory Write Transaction (2)
CPU places data word y on the bus.
Memory Write Transaction (3)
Main memory reads data word y from the bus and stores it at address A.
register file %eax
y
register file %eax
y
Store operation: movl %eax, A ALU main memory 0 A
Store operation: movl %eax, A ALU main memory 0
y
I/O bridge bus interface
y
I/O bridge bus interface
A
17
18
Page 3
Disk Geometry
Disks consist of platters, each with two surfaces. platters, surfaces. Each surface consists of concentric rings called tracks. tracks. Each track consists of sectors separated by gaps. gaps.
tracks surface track k gaps
Disk Geometry (Multiple-Platter View)
Aligned tracks form a cylinder.
cylinder k surface 0 surface 1 surface 2 surface 3 surface 4 surface 5 platter 0 platter 1 platter 2
spindle spindle
sectors
19 20
Disk Capacity
Capacity: maximum number of bits that can be stored.
Vendors express capacity in units of gigabytes (GB), where 1 GB = 10^9.
Computing Disk Capacity
Capacity = (# bytes/sector) x (avg. # sectors/track) x (# tracks/surface) x (# surfaces/platter) x (# platters/disk) Example:
512 bytes/sector 300 sectors/track (on average) 20,000 tracks/surface 2 surfaces/platter 5 platters/disk
Capacity is determined by these technology factors:
Recording density (bits/in): number of bits that can be squeezed into a 1 inch segment of a track. Track density (tracks/in): number of tracks that can be squeezed into a 1 inch radial segment. Areal density (bits/in2): product of recording and track density.
Modern disks partition tracks into disjoint subsets called recording zones
Each track in a zone has the same number of sectors, determined by the circumference of innermost track. Each zone has a different number of sectors/track
Capacity = 512 x 300 x 20000 x 2 x 5 = 30,720,000,000 = 30.72 GB
21 22
Disk Operation (Single-Platter View)
The disk surface spins at a fixed rotational rate The read/write head is attached to the end of the arm and flies over the disk surface on a thin cushion of air. spindle spindle spindle spindle
Disk Operation (Multi-Platter View)
read/write heads move in unison from cylinder to cylinder
arm
spindle By moving radially, the arm can position the read/write head over any track.
23
24
Page 4
Disk Access Time
Average time to access some target sector approximated by :
Taccess = Tavg seek + Tavg rotation + Tavg transfer
Disk Access Time Example
Given:
Rotational rate = 7,200 RPM Average seek time = 9 ms. Avg # sectors/track = 400.
Seek time (Tavg seek)
Time to position heads over cylinder containing target sector. Typical Tavg seek = 9 ms
Derived:
Tavg rotation = 1/2 x (60 secs/7200 RPM) x 1000 ms/sec = 4 ms. Tavg transfer = 60/7200 RPM x 1/400 secs/track x 1000 ms/sec = 0.02 ms Taccess = 9 ms + 4 ms + 0.02 ms
Rotational latency (Tavg rotation)
Time waiting for first bit of target sector to pass under r/w head. Tavg rotation = 1/2 x 1/RPMs x 60 sec/1 min
Transfer time (Tavg transfer)
Time to read the bits in the target sector. Tavg transfer = 1/RPM x 1/(avg # sectors/track) x 60 secs/1 min.
Important points:
Access time dominated by seek time and rotational latency. First bit in a sector is the most expensive, the rest are free. SRAM access time is about 4 ns/doubleword, DRAM about 60 ns
Disk is about 40,000 times slower than SRAM, 2,500 times slower then DRAM.
25 26
Logical Disk Blocks
Modern disks present a simpler abstract view of the complex sector set geometry:
The of available sectors is modeled as a sequence of bsized logical blocks (0, 1, 2, ...)
I/O Bus
CPU chip register file ALU system bus memory bus main memory
Mapping between logical blocks and actual (physical) sectors
Maintained by hardware/firmware device called disk controller. Converts requests for logical blocks into (surface,track,sector) triples.
bus interface
I/O bridge
Allows controller to set aside spare cylinders for each zone.
Accounts for the difference in "formatted capacity" and "maximum capacity".
27
I/O bus USB controller mouse keyboard graphics adapter monitor disk disk controller
Expansion slots for other devices such as network adapters.
28
Reading a Disk Sector (1)
CPU chip register file ALU
Reading a Disk Sector (2)
CPU chip register file ALU
CPU initiates a disk read by writing a command, logical block number, and destination memory address to a port (address) associated with disk controller.
Disk controller reads the sector and performs a direct memory access (DMA) transfer into main memory.
bus interface
main memory
bus interface
main memory
I/O bus
I/O bus
USB controller mouse keyboard
graphics adapter monitor
disk controller
USB controller mouse keyboard disk
29
graphics adapter monitor
disk controller
disk
30
Page 5
Reading a Disk Sector (3)
CPU chip register file ALU
Storage Trends
metric 1980 19,200 300 1985 2,900 150 1990 320 35 1995 256 15 2000 100 2 2000:1980 190 100
When the DMA transfer completes, the disk controller notifies the CPU with an interrupt (i.e., asserts a special "interrupt" pin on the CPU)
SRAM
$/MB access (ns)
bus interface
main memory
metric
1980
1985 880 200 0.256
1990 100 100 4
1995 30 70 16
2000 1 60 64
2000:1980 8,000 6 1,000
DRAM
I/O bus
$/MB 8,000 access (ns) 375 typical size(MB) 0.064
metric USB controller mouse keyboard graphics adapter monitor disk
31
1980
1985 100 75 10
1990 8 28 160
1995 0.30 10 1,000
2000 0.05 8 9,000
2000:1980 10,000 11 9,000
disk controller
Disk
$/MB 500 access (ms) 87 typical size(MB) 1
(Culled from back issues of Byte and PC Magazine)
32
CPU Clock Rates
1980 8080 1 1,000 1985 286 6 166 1990 386 20 50 1995 Pent 150 6 2000 P-III 750 1.6 2000:1980 750 750
The CPU-Memory Gap
The increasing gap between DRAM, disk, and CPU speeds.
100,000,000 10,000,000 1,000,000 100,000 ns 10,000 1,000 100 10 1 1980 1985 1990 year 1995 2000 Disk seek time DRAM access time SRAM access time CPU cycle time
processor clock rate(MHz) cycle time(ns)
33
34
Locality
Principle of Locality:
Programs tend to reuse data and instructions near those they have used recently, or that were recently referenced themselves. Temporal locality: Recently referenced items are likely to be referenced in the near future. Spatial locality: Items with nearby addresses tend to be referenced close together in time.
Locality Example
Claim: Being able to look at code and get a qualitative sense of its locality is a key skill for a professional programmer.
Question: Does this function have good locality?
int sumarrayrows(int a[M][N]) { int i, j, sum = 0; for (i = 0; i < M; i++) for (j = 0; j < N; j++) sum += a[i][j]; return sum }
35 36
Locality Example:
Data sum += a[i]; Reference array elements in succession return sum; (stride-1 reference pattern): Spatial locality Reference sum each iteration: Temporal locality Instructions Reference instructions in sequence: Spatial locality Cycle through loop repeatedly: Temporal locality
sum = 0; for (i = 0; i < n; i++)
Page 6
Locality Example
Question: Does this function have good locality?
Locality Example
Question: Can you permute the loops so that the function scans the 3-d array a[] with a stride-1 3stridereference pattern (and thus has good spatial locality)?
int sumarray3d(int a[N][N][N]) { int i, j, k, sum = 0; for (i = 0; i < N; i++) for (j = 0; j < N; j++) for (k = 0; k < N; k++) sum += a[k][i][j]; return sum }
int sumarraycols(int a[M][N]) { int i, j, sum = 0; for (j = 0; j < N; j++) for (i = 0; i < M; i++) sum += a[i][j]; return sum }
37
38
Memory Hierarchies
Some fundamental and enduring properties of hardware and software:
Fast storage technologies cost more per byte and have less capacity. The gap between CPU and main memory speed is widening. Well-written programs tend to exhibit good locality.
An Example Memory Hierarchy
Smaller, faster, and costlier (per byte) storage devices L0: registers L1: on-chip L1 cache (SRAM) L2: off-chip L2 cache (SRAM) main memory (DRAM)
CPU registers hold words retrieved from L1 cache.
L1 cache holds cache lines retrieved from the L2 cache memory. L2 cache holds cache lines retrieved from main memory.
L3:
These fundamental properties complement each other beautifully.
They suggest an approach for organiz...