Transcript File Systems - monismith.info

File and IO Systems
Operating Systems
CS550
Disk drive
• Various organization types
• Boot block contains code to do initial system
boot
• Disks may have one or more partitions
– Multiple Operating Systems may be installed
[1, 2]
3
File System Types
• Partitions may be controlled by different file
systems
• FAT/FAT32
• NTFS
• Ext2, Ext3, Ext4
• Lustre
• PVFS
• ReFS
• Many others
[1, 2, 3, 4, 5]
FAT/FAT32
•
•
•
•
•
•
•
•
•
•
File allocation table
MS-DOS/Windows
Allocation table contains pointers to files
FAT entries point to other FAT entries to create linked clusters
Cluster size – 1K to 32K
Max partition size – 2TB
Max file size – 4GB
No file permissions, no transaction log
Maintains free clusters in similar linked list fashion
3 pointers need to be changed when creating/adding cluster to a
file
• Can move whole fat chain to free list when deleting a file
• Lots of issues with scan disk because there is no transaction log
[1, 2]
NTFS
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
New technology file system
Effort by MSFT to create more reliable file system
Cluster size - 512B - 64KB
Max partition size - 256TB
Max file size - 16 TB
File names – unicode
Permissions – yes
Transaction log - yes
Uses master file table
Each directory entry points to an entry in the MFT
MFT entry contains permissions info and list of pointers of allocated clusters
MFT entries are of fixed size
Once one is full another is allocated for a file and linked to the previous MFT entry
Bit map is used to keep track of free clusters
Capable of encrypting all data on disk
[1, 2, 4]
Ext4
•
•
Ext4 (fourth extended filesystem)
Journaling file system for linux - keeps track of changes to be made in a journal
– Circular log in the file system
– Faster to bring back online
– Typically atomic
•
•
•
•
•
Older versions - ext, ext2, and ext3
Max volume size - 1 Exabyte (2^60 bytes)
Max file size 16 TB using 48-bit block addressing (64-bit is in progress for larger
files)
64000 subdirectories allowed per directory
Uses extents - contiguous physical blocks
– Improves file performance and reduces fragmentation
•
•
•
•
Single extents can map unto 128 MB of contiguous space with 4KB block size
Can preallocate space on disk
Uses delayed allocation to allocate multiple blocks at once allowing for more
contiguous files
Uses nanosecond timestamps for files
[5]
Lustre
• Lustre (linux cluster) - parallel distributed file system used on 15 of
top 30 supercomputers
• Three parts
– Metadata server has single metadata target per Lustre filesystem with
filenames, directories, access permissions and file layout.
• Stored on a single local disk file system (only used in pathname and
permissions checks)
– One or more object storage servers (OSSes) that store file data on one
or more object storage targets (OSTs)
• One OSS serves 2 - 8 OSTs or object storage targets
• Each OST manages a single local disk file system
– Clients that access and use the data which are presented with a
unified namespace for all files and data in file system
•
•
Size only limited by sum of capacities of OSTs
Most use enhanced ext4 called Idiskfs
[3]
Other File Systems
• Many other file systems exist
– CD-ROM
– DVD
– Networked file systems
• Virtual file systems used to allow access to
multiple file systems by an application
[1, 2]
Mounting and Unmounting
• Removable media and fixed disks must be
mounted
• File management system must determine type
of media and start appropriate file system
manager
• Inform virtual file system of its existence
• Referred to as mounting
• Undoing these steps is called unmounting
[1, 2]
IO Systems
• Want to take application I/O request and send
it to physical device and take response and
send it back to application
• Optimize performance of I/O request
• Recall I/O hardware
– Have device interface to control devices for CPU
– Device interface controls device while CPU is
doing work
[1, 2]
DMA
• Direct Memory Access
• Interrupts for slow devices and faster devices cause lots of CPU
overhead
• DMA works with OS (e.g. for reading from a file)
–
–
–
–
Application performs system call to read data from a file
Device converts request to a list of commands
Commands sent to device interface hardware
Device interface initiates and controls disk operation (CPU does other
work)
– Device interface uses DMA hardware to transfer data directly to and
from the memory buffer
– Once a block of data has been transferred, the device interface sends
an interrupt to the CPU to indicate completion of the request
[1, 2]
Disk Drives
• Multiple parts
– Arm
– Read/write head (on arm)
– Platters
– Tracks (on platters)
– Spindle (to make platters spin)
[1, 2]
Disk IO Scheduling
• First come first served (FCFS)
– Service IO request in order they arrived
• Shortest seek time first (SSTF)
– Handle request that requires moving arm the least amount first
– Can starve far away requests
• Elevator (SCAN)
– Once arm starts moving in one direction it keeps moving until
the last track is reached
– Only then does it move in the opposite direction
• Circular SCAN
– Like SCAN, but arm does not stop to handle requests while
moving in the reverse direction
[1, 2]
References
• [1] J. M. Garrido and R. Schlesinger, Principles of Modern Operating
Systems, Jones and Bartlett Publishers, Inc., Ontario, Canada, 2008.
• [2] H. M. Dietel, P. J. Dietel, and D. R. Choffnes, Operating Systems,
3rd Ed., Pearson Education, Inc., New Jersey, 2004.
• [3]Lustre 2.x File System Operations Manual,
http://build.whamcloud.com/job/lustremanual/lastSuccessfulBuild/artifact/lustre_manual.html, Intel
Corp., 2012, Oracle, 2011.
• [4] NTFS, http://en.wikipedia.org/wiki/NTFS, modified Nov. 29,
2012.
• [5] Ext4, https://ext4.wiki.kernel.org/index.php/Main_Page,
modified Oct 17, 2012.
[1, 2]