749 ELE Lecture 23. Disks & RAIDs
Ali Ziya Alkar ELE749 Computer Organization and Design
[Adapted from Computer Organization and Design, Patterson & Hennessy, 2005, UCB and Mary Jane Irwin ( www.cse.psu.edu/~mji ) ]
ELE 749 L23 RAIDs.1 Alkar 2007
Review: Major Components of a Computer
Processor Control
Devices Memory Output Input
Datapath
Main Memory
Cache
Secondary Memory (Disk)
ELE 749 L23 RAIDs.2
Alkar 2007
Magnetic Disk
Purpose
Long term, nonvolatile storage Lowest level in the memory hierarchy
- slow, large, inexpensive
Sector
General structure
Track
A rotating platter coated with a magnetic surface A moveable read/write head to access the information on the disk
Typical numbers
1 to 4 (1 or 2 surface) platters per disk of 1 to 5.25 in diameter (3.5 dominate in 2004) Rotational speeds of 5,400 to 15,000 RPM 10,000 to 50,000 tracks per surface
- cylinder - all the tracks under the head at a given point on all surfaces
100 to 500 sectors per track
- the smallest unit that can be read/written (typically 512B)
ELE 749 L23 RAIDs.3 Alkar 2007
Magnetic Disk Characteristic
Disk read/write components
proper track (3 to 14 ms avg)
due to locality of disk references the actual average seek time may be only 25% to 33% of the advertised number Cylinder Platter Controller + 1. Seek time: position the head over the Cache Track Sector
Head
2.
Rotational latency: wait for the desired sector to rotate under the head ( of 1/RPM converted to ms)
0.5/5400RPM = 5.6ms to 0.5/15000RPM = 2.0ms
3.
Transfer time: transfer a block of bits (one or more sectors) under the head to the disk controllers cache (30 to 80 MB/s are typical disk transfer rates)
the disk controllers cache takes advantage of spatial locality in disk accesses
cache transfer rates are much faster (e.g., 320 MB/s)
4.
Controller time: the overhead the disk controller imposes in performing a disk I/O access (typically < .2 ms)
Alkar 2007
ELE 749 L23 RAIDs.4
Typical Disk Access Time
The average time to read or write a 512B sector for a disk rotating at 10,000RPM with average seek time of 6ms, a 50MB/sec transfer rate, and a 0.2ms controller overhead
If the measured average seek time is 25% of the advertised average seek time, then
The rotational latency is usually the largest component of the access time
ELE 749 L23 RAIDs.5 Alkar 2007
Typical Disk Access Time
The average time to read or write a 512B sector for a disk rotating at 10,000RPM with average seek time of 6ms, a 50MB/sec transfer rate, and a 0.2ms controller overhead
Avg disk read/write = 6.0ms + 0.5/(10000RPM/(60sec/minute) )+ 0.5KB/(50MB/sec) + 0.2ms = 6.0 + 3.0 + 0.01 + 0.2 = 9.21ms
If the measured average seek time is 25% of the advertised average seek time, then
Avg disk read/write = 1.5 + 3.0 + 0.01 + 0.2 = 4.71ms
The rotational latency is usually the largest component of the access time
ELE 749 L23 RAIDs.6 Alkar 2007
Magnetic Disk Examples (www.seagate.com)
Characteristic Disk diameter (inches) Capacity (GB) # of surfaces (heads) Rotation speed (RPM) Transfer rate (MB/sec) Minimum seek (ms) Average seek (ms) MTTF (hours@25oC) Dimensions (inc hes ) GB/cu.inch Power: op/idle/sb (watts) GB/watt Weight (pounds)
ELE 749 L23 RAIDs.7
Seagate ST37 Seagate ST32 Seagate ST94 3.5 73.4 8 15,000 57-86 0.2r-0.4w 3.6r-3.9w 1,200,000 1x4x5.8 3 20?/12/4 1.9 3.5 200 4 7,200 32-58 1.0r-1.2w 8.5r-9.5w 600,000 1x4x5.8 9 12/8/1 16 1.4 2.5 40 2 5,400 34 1.5r-2.0w 12r-14w 330,000 0.4x2.7x3.9 10 2.4/1/0.4 17 0.2
Alkar 2007
Disk Latency & Bandwidth Milestones
CDC Wren SG ST 41 SG ST 15 SG ST 39 SG ST 37
RSpeed ( RPM) Year Capacity (Gbytes) Diameter (inches) Interface Bandwidth (MB/s) Latency (msec)
3600 1983 0.03 5.25 ST-412 0.6 48.3
5400 1990 1.4 5.25 SCSI 4 17.1
7200 1994 4.3 3.5 SCSI 9 12.7
10000 1998 9.1 3.0 SCSI 24 8.8
15000 2003 73.4 2.5 SCSI 86 5.7
Patterson, CACM Vol 47, #10, 2004
Disk latency is one average seek time plus the rotational latency. Disk bandwidth is the peak transfer time of formatted data from the media (not from the cache).
ELE 749 L23 RAIDs.8 Alkar 2007
Latency & Bandwidth Improvements
In the time that the disk bandwidth doubles the latency improves by a factor of only 1.2 to 1.4
100
Bandw idth (MB/s)
80 60 40 20 0 1983
ELE 749 L23 RAIDs.9
Latency (msec)
1990 1994 1998 Year of Introduction
2003
Alkar 2007
Aside: Media Bandwidth/Latency Demands
Bandwidth requirements
High quality video
- Digital data = (30 frames/s) (640 x 480 pixels) (24-b color/pixel) = 221 Mb/s (27.625 MB/s)
High quality audio
- Digital data = (44,100 audio samples/s) (16-b audio samples) (2 audio channels for stereo) = 1.4 Mb/s (0.175 MB/s)
Compression reduces the bandwidth requirements considerably
Latency issues
How sensitive is your eye (ear) to variations in video (audio) rates? How can you ensure a constant rate of delivery? How important is synchronizing the audio and video streams?
- 15 to 20 ms early to 30 to 40 ms late is tolerable
ELE 749 L23 RAIDs.10 Alkar 2007
Dependability, Reliability, Availability
Reliability measured by the mean time to failure (MTTF). Service interruption is measured by mean time to repair (MTTR) Availability a measure of service accomplishment
Availability = MTTF/(MTTF + MTTR)
To increase MTTF, either improve the quality of the components or design the system to continue operating in the presence of faulty components
1. 2.
Fault avoidance: preventing fault occurrence by construction Fault tolerance: using redundancy to correct or bypass faulty components (hardware)
Fault detection versus fault correction Permanent faults versus transient faults
ELE 749 L23 RAIDs.11
Alkar 2007
RAIDs: Disk Arrays
Redundant Array of Inexpensive Disks Arrays of small and inexpensive disks
Increase potential throughput by having many disk drives
- Data is spread over multiple disk - Multiple accesses are made to several disks at a time
Reliability is lower than a single disk But availability can be improved by adding redundant disks (RAID)
Lost information can be reconstructed from redundant information MTTR: mean time to repair is in the order of hours MTTF: mean time to failure of disks is tens of years
ELE 749 L23 RAIDs.12 Alkar 2007
RAID: Level 0 (No Redundancy; Striping)
blk1 blk2 blk3 blk4
Multiple smaller disks as opposed to one big disk
Spreading the blocks over multiple striping disks means that multiple blocks can be accessed in parallel increasing the performance
- A 4 disk system gives four times the throughput of a 1 disk system
Same cost as one big disk assuming 4 small disks cost the same as one big disk
No redundancy, so what if one disk fails?
Failure of one or more disks is more likely as the number of disks in the system increases
ELE 749 L23 RAIDs.13 Alkar 2007
RAID: Level 1 (Redundancy via Mirroring)
blk1.1 blk1.2 blk1.3 blk1.4 blk1.1 blk1.2 blk1.3 blk1.4
redundant (check) data
Uses twice as many disks as RAID 0 (e.g., 8 smaller disks with second set of 4 duplicating the first set) so there are always two copies of the data
# redundant disks = # of data disks so twice the cost of one big disk
- writes have to be made to both sets of disks, so writes would be only 1/2 the performance of RAID 0
What if one disk fails?
If a disk fails, the system just goes to the mirror for the data
ELE 749 L23 RAIDs.14
Alkar 2007
RAID: Level 0+1 (Striping with Mirroring)
blk1 blk2 blk3 blk4 blk1 blk2 blk3 blk4
redundant (check) data
Combines the best of RAID 0 and RAID 1, data is striped across four disks and mirrored to four disks
Four times the throughput (due to striping) # redundant disks = # of data disks so twice the cost of one big disk
- writes have to be made to both sets of disks, so writes would be only 1/2 the performance of RAID 0
What if one disk fails?
If a disk fails, the system just goes to the mirror for the data
ELE 749 L23 RAIDs.15 Alkar 2007
RAID: Level 2 (Redundancy via ECC)
blk1,b0 blk1,b1 blk1,b2 blk1,b3
Checks 4,5,6,7 Checks 2,3,6,7 Checks 1,3,5,7
1
3
0
5
1
6
0 0
7
1
4
0
2 ECC disks
1
1
ECC disks 4 and 2 point to either data disk 6 or 7, but ECC disk 1 says disk 7 is okay, so disk 6 must be in error
ECC disks contain the parity of data on a set of distinct overlapping disks
# redundant disks = log (total # of data disks) so almost twice the cost of one big disk
- writes require computing parity to write to the ECC disks - reads require reading ECC disk and confirming parity
Can tolerate limited disk failure, since the data can be reconstructed
ELE 749 L23 RAIDs.16 Alkar 2007
RAID: Level 3 (Bit-Interleaved Parity)
blk1,b0 blk1,b1 blk1,b2 blk1,b3
1
0
1
0
(odd) bit parity disk
Cost of higher availability is reduced to 1/N where N is the number of disks in a protection group
# redundant disks = 1 # of protection groups
- writes require writing the new data to the data disk as well as computing the parity, meaning reading the other disks, so that the parity disk can be updated
Can tolerate limited disk failure, since the data can be reconstructed
- reads require reading all the operational data disks as well as the parity disk to calculate the missing data that was stored on the failed disk
ELE 749 L23 RAIDs.17 Alkar 2007
RAID: Level 3 (Bit-Interleaved Parity)
blk1,b0 blk1,b1 blk1,b2 blk1,b3
1
0
1 disk fails
0
1
(odd) bit parity disk
Cost of higher availability is reduced to 1/N where N is the number of disks in a protection group
# redundant disks = 1 # of protection groups
- writes require writing the new data to the data disk as well as computing the parity, meaning reading the other disks, so that the parity disk can be updated
Can tolerate limited disk failure, since the data can be reconstructed
- reads require reading all the operational data disks as well as the parity disk to calculate the missing data that was stored on the failed disk
ELE 749 L23 RAIDs.18 Alkar 2007
RAID: Level 4 (Block-Interleaved Parity)
blk1 blk2 blk3 blk4
block parity disk
Cost of higher availability still only 1/N but the parity is stored as blocks associated with sets of data blocks
Four times the throughput (striping) # redundant disks = 1 # of protection groups Supports small reads and small writes (reads and writes that go to just one (or a few) data disk in a protection group)
- by watching which bits change when writing new information, need only to change the corresponding bits on the parity disk - the parity disk must be updated on every write, so it is a bottleneck for back-to-back writes
Can tolerate limited disk failure, since the data can be reconstructed
ELE 749 L23 RAIDs.19 Alkar 2007
Small Writes
RAID 3 small writes
New D1 data D1 D2 D3 D1 D2 D3 D4 P D4 P
3 reads and 2 writes involving all the disks
RAID 4 small writes
New D1 data D1 D2 D3 D4 D2 D3 D4 P
Alkar 2007
P
2 reads and 2 writes involving just two disks
ELE 749 L23 RAIDs.20
D1
RAID: Level 5 (Distributed Block-Interleaved Parity)
one of these assigned as the block parity disk
Cost of higher availability still only 1/N but the parity block can be located on any of the disks so there is no single bottleneck for writes
Still four times the throughput (striping) # redundant disks = 1 # of protection groups Supports small reads and small writes (reads and writes that go to just one (or a few) data disk in a protection group) Allows multiple simultaneous writes as long as the accompanying parity blocks are not located on the same disk
Can tolerate limited disk failure, since the data can be reconstructed
ELE 749 L23 RAIDs.21 Alkar 2007
Distributing Parity Blocks
RAID 4
1 5 9 13 2 6 10 14 3 7 11 15 4 8 12 16 P0 P1 P2 P3 1 5 9 13 2 6 10 P3
RAID 5
3 7 P2 14 4 P1 11 15 P0 8 12 16
By distributing parity blocks to all disks, some small writes can be performed in parallel
ELE 749 L23 RAIDs.22 Alkar 2007
Summary
Four components of disk access time:
Seek Time: advertised to be 3 to 14 ms but lower in real systems Rotational Latency: 5.6 ms at 5400 RPM and 2.0 ms at 15000 RPM Transfer Time: 30 to 80 MB/s Controller Time: typically less than .2 ms
RAIDS can be used to improve availability
RAID 0? and RAID 5 widely used in servers, one estimate is that 80% of disks in servers are RAIDs RAID 1 (mirroring) EMC, Tandem, IBM RAID 3 Storage Concepts RAID 4 Network Appliance
RAIDS have enough redundancy to allow continuous operation, but not hot swapping
ELE 749 L23 RAIDs.23 Alkar 2007
Next Lecture and Reminders
Next lecture
- Reading assignment PH 8.4-8.10
Reminders
Hw3 Do: Exercises: 8.5, 8.6 and 8.8 due on.. You should have started your Mt project topics already. Your presentations are due on Monday December, 24th Your final reports are due on Last day of exams.
ELE 749 L23 RAIDs.24
Alkar 2007