- CSC228H File Structures and Data Management
File structure design principles...
Ideal case: get all the information we want with just one access.
Is this possible in practice?
Whenever we can't get the information we need in just one access, we would like to do as few disk operations as possible.
1
Francisco J. Estrada
CSC228H - File Structures and Data Management
Problem:
The contents of a disk change often, maintaining the structure of the data contained in the disk is hard in practice.
Hence, how do we maintain efficiency when we are allowed to edit / insert / delete data from our files?
2
Francisco J. Estrada
CSC228H - File Structures and Data Management
What we can use to help us deal with these problems:
Files whose internal structures allow quick access and ease of maintenance. Files with no internal organization require the longest time to search! Auxiliary files that allow us to locate information within a file.
3
Francisco J. Estrada
1
CSC228H - File Structures and Data Management
Physical files & logical files Physical file, the information that is actually stored in the disk, tape, CD, etc. Logical file, what your program actually uses, a `pipe' though which information can be extracted, or sent. Why is the distinction useful? Why not allow our programs to deal directly with physical files?
4
Francisco J. Estrada
CSC228H - File Structures and Data Management
Basic file operations... Opening a file - basically, links a logical file to a physical file. On open, the O/S performs a series operations that end in the program that is trying to open the file being assigned a file descriptor. Additionally, the O/S will perform particular operations on the file at the request of the calling program, these operations are intended to `initialize' the file for use by the program. What happens when the O/S detects an error?
5
Francisco J. Estrada
CSC228H - File Structures and Data Management
Basic file operations... Closing a file - cuts the link between the physical and logical files. Upon closing, the O/S takes care of `synchronizing' the contents of the file. In general, files are automatically closed when the program ends. So, why do we need to worry about closing files?
6
Francisco J. Estrada
2
CSC228H - File Structures and Data Management
Basic file operations... Reading and writing - basic I/O operations. Usually require three parameters: a logical file, an address, and the amount of data that is to be read or written. What is the use of the address parameter?
7
Francisco J. Estrada
CSC228H - File Structures and Data Management
Additional operations...
Seeking Detecting the end of a file Detecting I/O errors
8
Francisco J. Estrada
CSC228H - File Structures and Data Management
Storage organization... Two types of devices:
- Those that provide direct access (such as Hard Disks).
- Those that support only serial access.
9
Francisco J. Estrada
3
CSC228H - File Structures and Data Management
Disk organization... Divided in sectors, tracks, and cylinders. All the information contained in a cylinder can be accessed with one seek. Capacity of a disk: - One platter has 2 sides, hence #tracks / cylinder = 2 * # platters - # of cylinders = # of tracks in a single surface - Disk capacity = (# bytes / sector) * (# sectors / track) * (# tracks / cylinder) * (# of cylinders)
10
Francisco J. Estrada
CSC228H - File Structures and Data Management
Example...
For a disk with: - 512 bytes/sector - 63 sectors / track - 8 platters - 16,383 cylinders Capacity = (512) * (63) * (8*2) * (16,383) = 8,455,200,768
11
Francisco J. Estrada
CSC228H - File Structures and Data Management
Example (2)... For the same disk, how many cylinders would be required to store a 200 Mb file? 200 Mb = (200 * 1024 * 1024) = 209,715,200 bytes Cylinder capacity = (512) * (63) * (8*2) = 526,096 bytes Hence, we need: 399 cylinders.
12
Francisco J. Estrada
4
CSC228H - File Structures and Data Management
Other units of file storage...
Cluster - fixed number of contiguous sectors. Smallest unit of space that can be allocated to a file (what does this imply?) Extents - portions of a file that are contiguous. Blocks - data groups of user defined (O/S defined) length.
13
Francisco J. Estrada
CSC228H - File Structures and Data Management
The cost of disk access... Seek time, usually the largest part of the total cost. Rotational delay, usually about half a revolution, but doesn't have a very large impact. Transfer time
Sequential reading is more efficient than moving over the disk surface.
14
Francisco J. Estrada
CSC228H - File Structures and Data Management
Organization of magnetic tape... Magnetic supports tape only serial access, data stored in tracks that run across the length of the tape. Large storage capacity in removable media. Transfer rates similar to HD
15
Francisco J. Estrada
5
CSC228H - File Structures and Data Management
CD-Rom... Storage organization is radically different to the organization of a Hard Disk. Seeking performance is poor (500 ms. - 1 sec.)! Information organized according to the following: - 1 `second' of data (music) = 75 sectors, 1 sector = 2Kb - To address data, the minute, second, and sector of play must be specified. Transfer rates 1x - 48x (1x = 150 Kb/sec.)
16
Francisco J. Estrada
CSC228H - File Structures and Data Management
Buffers... Enhance performance by reducing disk access In general any file will be associated with more than one buffer How can we increase performance by using multiple buffers? Buffer management is generally done by the O/S, but your program can create and maintain its own buffers.
17
Francisco J. Estrada
CSC228H - File Structures and Data Management
Unix I/O
Layered structure 4 tables are used to locate information on a file requested by a process: - A file descriptor table (maintained by the process) - An open file table (maintained by the kernel) - A table of index nodes (maintained by the kernel) - A file allocation table (part of the index node)
18
Francisco J. Estrada
6
CSC228H - File Structures and Data Management
Inodes...
Contain information about the files, such as access permission, owner, size, and pointers to the blocks that make up the file. Stored on disk as part of the file. A directory contains pointers to the inode corresponding to all files listed within it. UNIX views the disk as a randomly addressable array of fixed size blocks.
19
Francisco J. Estrada
CSC228H - File Structures and Data Management
Inodes (continued)... Different types of pointers to file blocks are contained within the inode: - Direct pointers - Single indirect pointers - Double indirect pointers - Triple indirect pointers The maximum number of pointers determines the maximum allowed number of blocks, and hence, the maximum allowed size for any file.
20
Francisco J. Estrada
CSC228H - File Structures and Data Management
Example: The inode contains 10 direct pointers, 1 single indirect pointer, 1 double indirect pointer, and 1 triple indirect pointer. Assume each block can hold 256 pointers. - The maximum number of addressable blocks is: (10) From the direct pointers + (256 *1) From the single indirect pointers + (256*256*1) From the double indirect pointers + (256*256*256*1) From the triple indirect pointers = 16,843,023 blocks. - If each block holds 1Kb, then the max. file size is 16Gb!
21
Francisco J. Estrada
7
CSC228H - File Structures and Data Management
Field and record organizat...