Transcript slides

File System Implementation
04/05/2004
CSCI 315 Operating Systems Design
1
Acyclic-Graph Directories
Have shared subdirectories and files.
links: soft (symbolic)
hard
04/05/2004
Unix: ln (read man page);
need to keep a reference count on
each file or directory.
CSCI 315 Operating Systems Design
2
Acyclic-Graph Directories
(Cont.)
• Different names (aliasing) for the same file
or directory.
• If dict deletes list  dangling pointer.
Solutions:
– Backpointers, so we can delete all pointers.
Variable size records a problem.
– Backpointers using a daisy chain
organization.
– Entry-hold-count solution.
04/05/2004
CSCI 315 Operating Systems Design
3
General Graph Directory
04/05/2004
CSCI 315 Operating Systems Design
4
General Graph Directory (Cont.)
• How do we guarantee no cycles?
– Allow only links to file not subdirectories.
– Garbage collection.
– Every time a new link is added use a cycle
detection algorithm to determine whether it is
OK.
04/05/2004
CSCI 315 Operating Systems Design
5
File System Mounting
• A file system (partition) must be mounted before it can be
accessed. Mounting allows one to attach the file system on one
device to the file system on another device.
• A unmounted file system needs to be attached to a mount point
before it can be accessed.
existing
04/05/2004
unmounted
CSCI 315 Operating Systems Design
6
File Sharing
• Sharing of files on multi-user systems is desirable.
• Sharing may be done through a protection scheme.
• On distributed systems, files may be shared across a
network.
• Network File System (NFS) is a common distributed filesharing method.
04/05/2004
CSCI 315 Operating Systems Design
7
Protection
• File owner/creator should be able to control:
– what can be done,
– by whom.
Discretionary Access Control (DAC)
• Types of access:
–
–
–
–
–
–
04/05/2004
Read,
Write,
Execute,
Append,
Delete,
List.
CSCI 315 Operating Systems Design
8
Protection
• Mandatory Access Control (MAC):
– System policy: files tied to access levels = (public, restricted,
confidential, classified, top-secret).
– Process also has access level: can read from and write to all
files at same level, can only read from files below, can only write
to files above.
• Role-Based Access Control (RBAC):
– System policy: defines “roles” (generalization of the Unix idea
of groups).
– Roles are associated with access rules to sets of files and
devices.
– A process can change roles (in a pre-defined set of possibilities)
during execution.
04/05/2004
CSCI 315 Operating Systems Design
9
Access Lists and Groups
•
•
Mode of access: read, write, execute
Three classes of users
RWX
a) owner access
7111
RWX
b) group access
6 110
RWX
c) public access
1 001
•
Ask manager to create a group (unique name), say G, and add some
users to the group.
For a particular file (say game) or subdirectory, define an appropriate
access.
•
owner
group
chmod
761
public
game
Associate a group with a file: chgrp G game
04/05/2004
CSCI 315 Operating Systems Design
10
File-System Structure
• File structure:
– Logical storage unit,
– Collection of related information.
• File system resides on secondary storage
(disks).
• File system is organized into layers.
• File control block – storage structure
consisting of information about a file.
04/05/2004
CSCI 315 Operating Systems Design
11
Layered File System
04/05/2004
CSCI 315 Operating Systems Design
12
A Typical File Control Block
04/05/2004
CSCI 315 Operating Systems Design
13
In-Memory File System Structures
file open
file read
04/05/2004
CSCI 315 Operating Systems Design
14
Virtual File Systems
• Virtual File Systems (VFS) provide an objectoriented way of implementing file systems.
• VFS allows the same system call interface (the
API) to be used for different types of file
systems.
• The API is to the VFS interface, rather than any
specific type of file system.
04/05/2004
CSCI 315 Operating Systems Design
15
Schematic View of Virtual File
System
same API for
all file system
types
ext3
04/05/2004
FAT 32
CSCI 315 Operating Systems Design
NFS
16
Directory Implementation
The directory is a symbol table that maps file names to pointers that lead to
the blocks comprising a file.
• Linear list of file names with pointer to the data
blocks:
– simple to program, but…
– time-consuming to execute.
• Hash Table:
– decreases directory search time,
– collisions – situations where two file names hash to
the same location,
– fixed size.
04/05/2004
CSCI 315 Operating Systems Design
17
Allocation Methods
An allocation method refers to how disk
blocks are allocated for files:
• Contiguous allocation,
• Linked allocation,
• Indexed allocation.
04/05/2004
CSCI 315 Operating Systems Design
18
Contiguous Allocation
• Each file occupies a set of contiguous blocks on
the disk.
• Simple – only starting location (block #) and length
(number of blocks) are required.
• Random access.
• Wasteful of space (dynamic storage-allocation
problem).
• Files cannot grow.
04/05/2004
CSCI 315 Operating Systems Design
19
Contiguous Allocation
• Mapping from logical to physical.
Q
LA/512
R
– Block to be accessed = ! + starting
address
– Displacement into block = R
04/05/2004
CSCI 315 Operating Systems Design
20
Contiguous Allocation of Disk Space
04/05/2004
CSCI 315 Operating Systems Design
21
Extent-Based Systems
• Many newer file systems (i.e. Veritas File
System) use a modified contiguous allocation
scheme.
• Extent-based file systems allocate disk blocks in
extents.
• An extent is a contiguous set of block. Extents
are allocated for each file. A file consists of one
or more extents.
04/05/2004
CSCI 315 Operating Systems Design
22
Linked Allocation
Each file is a linked list of disk blocks: blocks
may be scattered anywhere on the disk.
block
04/05/2004
=
pointer
CSCI 315 Operating Systems Design
23
Linked Allocation (Cont.)
• Simple – need only starting address
• Free-space management system – no
waste of space
• No random access
Q
• Mapping
LA/511
R
Block to be accessed is the Qth block in the linked chain
of blocks representing the file.
Displacement into block = R + 1
File-allocation table (FAT) – disk-space allocation used by
MS-DOS and OS/2.
04/05/2004
CSCI 315 Operating Systems Design
24
Linked Allocation
04/05/2004
CSCI 315 Operating Systems Design
25
File-Allocation Table
04/05/2004
CSCI 315 Operating Systems Design
26
Indexed Allocation
• Brings all pointers together into the index
block.
• Logical view.
index table
04/05/2004
CSCI 315 Operating Systems Design
27
Example of Indexed Allocation
04/05/2004
CSCI 315 Operating Systems Design
28
Indexed Allocation (Cont.)
• Need index table
• Random access
• Dynamic access without external fragmentation,
but have overhead of index block.
• Mapping from logical to physical in a file of
maximum size of 256K words and block size of
512 words. We need only 1 block for index table.
Q
LA/512
R
Q = displacement into index table
R = displacement into block
04/05/2004
CSCI 315 Operating Systems Design
29
Indexed Allocation – Mapping
(Cont.)
• Mapping from logical to physical in a file of unbounded
length (block size of 512 words).
• Linked scheme – Link blocks of index table (no limit on size).
Q1
LA / (512 x 511)
R1
Q1 = block of index table
R1 is used as follows:
Q2
R1 / 512
R2
Q2 = displacement into block of index table
R2 displacement into block of file:
04/05/2004
CSCI 315 Operating Systems Design
30
Indexed Allocation – Mapping
(Cont.)
• Two-level index (maximum file size is
Q
5123)
LA / (512 x 512)
1
R1
Q1 = displacement into outer-index
R1 is used as follows:
Q2
R1 / 512
R2
Q2 = displacement into block of index table
R2 displacement into block of file:
04/05/2004
CSCI 315 Operating Systems Design
31
Indexed Allocation – Mapping
(Cont.)

outer-index
index table
04/05/2004
CSCI 315 Operating Systems Design
file
32
Combined Scheme: UNIX
(4K bytes per block)
04/05/2004
CSCI 315 Operating Systems Design
33
Free-Space Management
• Bit vector (n blocks)
0 1
2
n-1
bit[i] =

…
0  block[i] free
1  block[i] occupied
Block number calculation
(number of bits per word) *
(number of 0-value words) +
offset of first 1 bit
04/05/2004
CSCI 315 Operating Systems Design
34
Free-Space Management
(Cont.)
• Bit map requires extra space. Example:
block size = 212 bytes
disk size = 230 bytes (1 gigabyte)
n = 230/212 = 218 bits (or 32K bytes)
• Easy to get contiguous files
• Linked list (free list)
– Cannot get contiguous space easily
– No waste of space
• Grouping
• Counting
04/05/2004
CSCI 315 Operating Systems Design
35
Free-Space Management
(Cont.)
• Need to protect:
– Pointer to free list
– Bit map
• Must be kept on disk
• Copy in memory and disk may differ.
• Cannot allow for block[i] to have a situation where bit[i]
= 1 in memory and bit[i] = 0 on disk.
– Solution:
• Set bit[i] = 1 in disk.
• Allocate block[i]
• Set bit[i] = 1 in memory
04/05/2004
CSCI 315 Operating Systems Design
36
Linked Free Space List on Disk
04/05/2004
CSCI 315 Operating Systems Design
37
Efficiency and Performance
• Efficiency dependent on:
– disk allocation and directory algorithms
– types of data kept in file’s directory entry
• Performance
– disk cache – separate section of main memory for
frequently used blocks
– free-behind and read-ahead – techniques to optimize
sequential access
– improve PC performance by dedicating section of
memory as virtual disk, or RAM disk.
04/05/2004
CSCI 315 Operating Systems Design
38
Various Disk-Caching Locations
04/05/2004
CSCI 315 Operating Systems Design
39
Page Cache
• A page cache caches pages rather than disk
blocks using virtual memory techniques.
• Memory-mapped I/O uses a page cache.
• Routine I/O through the file system uses the
buffer (disk) cache.
• This leads to the following figure.
04/05/2004
CSCI 315 Operating Systems Design
40
I/O Without a Unified Buffer
Cache
04/05/2004
CSCI 315 Operating Systems Design
41
Unified Buffer Cache
• A unified buffer cache uses the same page
cache to cache both memory-mapped
pages and ordinary file system I/O.
04/05/2004
CSCI 315 Operating Systems Design
42
I/O Using a Unified Buffer
Cache
04/05/2004
CSCI 315 Operating Systems Design
43
Recovery
• Consistency checking – compares data in
directory structure with data blocks on disk, and
tries to fix inconsistencies.
• Use system programs to back up data from disk
to another storage device (floppy disk, magnetic
tape).
• Recover lost file or disk by restoring data from
backup.
04/05/2004
CSCI 315 Operating Systems Design
44
Log Structured File Systems
• Log structured (or journaling) file systems record each update to
the file system as a transaction.
• All transactions are written to a log. A transaction is considered
committed once it is written to the log. However, the file system
may not yet be updated.
• The transactions in the log are asynchronously written to the file
system. When the file system is modified, the transaction is removed
from the log.
• If the file system crashes, all remaining transactions in the log must
still be performed.
04/05/2004
CSCI 315 Operating Systems Design
45