Transcript File System - Computing Sciences

Week Twelve Agenda
• Announcements
The final exam will be True/False, Multiple
Choice, and Essay.
Make presentation reservation now!
Presentation dates are
Tuesday December 9 and December 16 for
SectionV1WW
• Link of the week
• Review week eleven lab assignment
• Week eleven expected outcomes
• Next lab assignment
• Break-out problems
• Upcoming deadlines
• Lab assistance
Link of the week
• Domain name registration Web site
https://www.pairnic.com/services.html?gclid=COXiuLv6mI0
CFSCTWAod81M57A
• Determine if a domain is currently in use.
http://www.pair.com/
Online success starts with a great domain.
• What is a domain name?
A domain name (for instance, "example.com") is an
identification string that defines a realm of administrative
autonomy, authority, or control on the Internet. Domain names
are formed by the rules and procedures of the Domain Name
System (DNS). Any name registered in the DNS is a domain
name.
Link of the week
• What is the function of the Domain Name System?
The DNS translates Internet domain and host names to IP
addresses. DNS automatically converts the names we type
in our Web browser address bar to the IP addresses of
Web servers hosting those sites.
• BIND (Berkley Internet Name Domain)
BIND is the most widely used DNS software on the Internet.
On Unix-like operating systems it is the de facto standard.
Originally written by four graduate students at the Computer
Systems Research Group at the University of California,
Berkeley (UCB), the name originates as an acronym from
Berkeley Internet Name Domain,[5] reflecting the
application's use within UCB.
Link of the week
• What is meant by top level domain?
A top-level domain (TLD) is one of the domains at the
highest level in the hierarchical Domain Name System of the
Internet. The top-level domain names are installed in the root
zone of the name space. For all domains in lower levels, it is
the last part of the domain name, that is, the last label of a fully
qualified domain name.
Link of the week
Top Level Domain (TLD)
gov – Government
edu – Educational
org – Other organizations ( often nonprofit)
mil – Military
com – Commercial business
net – Network organizations
ca – Canada
th – Thailand
Link of the week
Link of the Week
Today, the Internet Corporation for Assigned
Names and Numbers (ICANN) manages the
top-level development and architecture of
the Internet domain name space. It
authorizes domain name registrars, through
which domain names may be registered and
reassigned.
Link of the Week
The right-most label conveys the top-level
domain; for example, the domain name
www.example.com belongs to the top-level domain
com.
The hierarchy of domains descends from the right to the
left label in the name; each label to the left specifies a
subdivision, or subdomain of the domain to the right.
For example: the label example specifies a node
example.com as a subdomain of the com domain, and
www is a label to create www.example.com, a
subdomain of example.com.
Link of the week
http://www.iana.org/cctld/cctld.htm
Iana - Internet Assigned Numbers Authority
The top-level domains (TLDs) such as com, net
and org are the highest level of domain names of the
Internet. Top-level domains form the DNS root zone
of the hierarchical Domain Name System. Every
domain name ends with a top-level domain label.
Each country has an ISO (International
Organization for Standardization) country code
designated as its domain name.
Link of the week
A hostname is a domain name that has at
least one associated IP address. For example,
the domain names www.example.com and
example.com are also hostnames, whereas the
com domain is not. However, other top-level
domains, particularly country code top-level
domains, may indeed have an IP address, and if
so, they are also hostnames.
Link of the week
Hostnames impose restrictions on the
characters allowed in the corresponding domain
name. A valid hostname is also a valid domain
name, but a valid domain name may not
necessarily be valid as a hostname.
Link of the week
UNIX-like file system is hierarchical. It is
interpreted from left to right. The top-level
domain name appears at the left end.
/var/mail/dandrear
The URL (Uniform Resource Locator) is also
hierarchical. Its top-level domain name
appears at the end, but is read from right to
left.
alpha.einstein.edu
beta.ca.mail2web.com
Link of the week
Review week eleven lab assignment
Kernel Initialization
Kernel initialization generates output on the console. You can
examine the output from the initialization by viewing
/var/log/dmesg, which logs all kernel messages during this
procedure.
User Initialization
The /etc/inittab format has not changed, but its level of
involvement in the boot up procedure has been reduced with
Red Hat Enterprise 6.0.
id:runlevels:action:process
The init process accesses the /etc/inittab file for one system
configuration setting (Red Hat Release 6.0).
# initdefault run level is 3
id:3:initdefault:
Review week eleven lab assignment
Red Hat 6.0 Release:
Runlevels are a concept from UNIX System V used by the init
daemon or other system initialization systems to define modes of
system operation.
Eight runlevels are permitted, the first seven are numbered 0-6 and
the eighth is named S or s (both are permitted).
System initialization is started by /etc/init/rcS.conf
The rcS.conf file contains no information about previous run levels.
The stop run level is set to “S”. In the event that the system stops, the
default is single user. The /etc/rc.d/rc.sysinit script is executed.
# Individual runlevels are started by /etc/init/rc.conf
#
# Ctrl-Alt-Delete is handled by /etc/init/control-alt-delete.conf
#
# Terminal gettys are handled by /etc/init/tty.conf and
/etc/init/serial.conf,
# with configuration in /etc/sysconfig/init.
Review week eleven lab assignment
User Initialization
/etc/profile file is the basic setup for all users.
The /etc/profile contains the environmental
variables and commands used by most shell
users.
User preferred environmental set up
$HOME/.profile
$HOME/.bash_profile
If the /etc/profile and .profile files are not present
during the initialization, a prompt is still
displayed.
Use the ls –a command to display the
.profile or .bash_profile file under your
$HOME directory.
Review week eleven lab assignment
Metadata
Metadata is data about data. It may describe a
single piece of data or multiple content items
and hierarchical levels.
Example: Legacy card catalogs in a library.
Metadata can be the name of a US corporation,
or a registered trademark.
Review week eleven lab assignment
Metadata
Metadata is structured information that describes
and allows us to find, manage, control and
understand other information. Structured
metadata represents the specifications of this
information. In this case, a better definition of
this design concept would be "data about the
containers of data".
Review week eleven lab assignment
Types of Metadata
There are many different types of metadata.
Business Intelligence metadata
General IT metadata
IT metadata management products
File system metadata
Image metadata
Review week eleven lab assignment
Linux runlevels:
0 - halt (Do NOT set initdefault to this)
1 - Single user mode
2 - Multiuser, without NFS (The same as 3, if you do not have
networking)
3 - Full multiuser mode
4 - unused
5 - X11
6 - reboot (Do NOT set initdefault to this)
The initialization process for a terminal uses getty to set it up for
login by a user. These commands are not used within user
shell scripts. Their scripting counterpart is stty.
The mingetty is the minimal getty for console(s).
Review week eleven lab assignment
/etc/inittab actions
respawn – The process will be restarted whenever it
terminates.
wait – The process will be started once when the
specific runlevel is entered.
once – The process will be executed once when the
specific runlevel is entered.
boot – The process will be executed during system
initialization.
initdefault – This entry specifies the runlevel which
should be entered after system boot.
sysinit – The process will be executed during system
boot. It will be executed before any boot or
bootwait entries.
Review week eleven lab assignment
/etc/inittab Actions
powerwait – The process will be executed when the
power goes down.
powerokwait – This process will be executed as soon as
init process is informed that the power has been
restored.
powerfailnow – This process will be executed when the
init process is informed that the battery of the external
UPS power is failing.
ctraltdel – The process is executed when the init process
receives the SIGINT signal. This means that someone
on the system console has pressed the Ctrl-Alt-Del
key combination.
Review week eleven lab assignment
UNIX-like commands:
ps –s
ps –e –o pid,ppid,stat
ps –aux
ps –x
ls –a
ps -aux | grep crond
nice (UNIX)
(view zombie processes)
(display all processes / listed fields)
(view status of all processes)
(view only active processes)
(view “.” processes)
(view crond executing)
(modified scheduling priority)
(-20 to 19 range)
If a user wanted to compress a large file, but not slow down other
processes, they might run the following:
nice -n 19 tar cvzf archive.tgz largefile
Review week eleven lab assignment
UNIX-like commands:
umask
(displays umask numeric values)
umask –S
(displays umask symbolic values)
fg (Continues a stopped job by running it in the foreground,
some shells are not able to run this command ).
bg (Continues a stopped job in the background)
cat /etc/shells
echo $SHELL
chsh –s /bin/ksh
printenv
df
(Display available shells on system).
(Display current shell being utilized).
(Change to a different shell) .
(command to display environment
variables).
(report file system disk space)
Review week eleven lab assignment
Unix/Linux Processes
Linux defines a unit of work as a task or process.
UNIXdefines a unit of work as a process.
Each process is identified by a unique PID
(Process ID).
Each user is identified by a unique UID (User
ID).
Each group of users is uniquely identified by a
GID (Group ID).
Each process is started from a previous process
using the fork and exec command.
The init process is created by the kernel using a
non-traditional procedure called hand-crafted
or spontaneous.
Review week eleven lab assignment
Non-Inherited Parent Process Attributes
File locks (semaphores)
Child resource utilization is set to zero
Pending signals (kill –l signals)
Daemon processes offer services like web pages
serving, email transferring, time synchronization, and
similar. They usually consume little CPU and
memory, and run quietly without user interaction.
They mainly communicate with other computer
programs, or with other computers via network.
Review week eleven lab assignment
Crontab Utilization:
cron is the periodic event scheduler of your system.
cron enables users to schedule jobs (commands or
shell scripts) to run periodically at certain times or
dates. It is commonly used to automate system
maintenance or administration, though its generalpurpose nature means that it can be used for other
purposes, such as connecting to the Internet and
downloading email. The following examples give you
an idea of its usefulness and necessity.
• regular daily backups
• periodic mail checking
• polling a device(s) for input
• sending regular reports
• sending user email every time a cron function is
performed
Review week eleven lab assignment
Crontab Utilization:
cron file locations
/etc/crontab
/etc/cron.deny
/etc/cron.allow
Typical directory configuration
/etc/cron.d/hourly
/etc/cron.d/daily
/etc/cron.d/weekly
/etc/cron.d/monthly
Review week eleven lab assignment
How frequent does crond execute on Unix-like systems?
Disable email
To prevent email messages from being sent to your mail
account, create a record that contains the following syntax:
* * * * * echo [email protected] > trash_bin
To cancel the email redirection, enter the following:
rm trash_bin
If the output is not needed, the following syntax can be
included:
* * * * * echo “Hello Administrator” > /dev/null 2>&1
The /dev/null device is a special file that disregards all data written
to it.
Review week eleven lab assignment
crontab format:
* * * * * command to be executed
crontab values:
Field
Value
minute
00-59
hour
00-23 (military time)
day
1-31
month
1-12
weekday
0-6 (0=Sunday)
Example records:
There are a total of 6 fields in the crontab file
30 5 11 12 * echo “Welcome to Franklin University”
30 * * * * echo “Unix Administration”
Review week eleven lab assignment
Demonstrate: Display crontab entries and /var/mail/dandrear file
and introduce the tail –f u_monitor.csv command.
Who can use the crontab commands (create, list, edit or remove)?
What does the “*” character represent in the first five fields of a
crontab record? Match all values
A single integer will match that exact time value.
A comma separated list of integers can invoke multiple matches
like 1,2,3.
Example: 5,10,45 * * * * echo “Unix Administration”
Review week eleven lab assignment
crontab restrictions
If your name appears in the cron.allow file. If that files doesn’t
exist, you can use crontab.
If your name does not appear in the cron.deny file. If cron.deny
exists and is empty, all users can use crontab. If neither file
exists, only the root user can use crontab.
crontab command without options
Read standard input
Exit using “Control C” so that the existing crontab is
unmodified.
Exit using “Control D” will cause the current users’
crontab to be replaced with no information.
Review week eleven lab assignment
crontab commands
crontab -e
Edit crontab file
crontab –l
Display your crontab file
crontab –r
Remove your crontab file
crontab –v
Display last time crontab was
edited.
Review week eleven lab assignment
Samba
The /etc/samba/smb.conf file controls
most aspects of how Samba
works and is divided into two
sections. Each section begins with
a line that starts with an open
bracket ([), includes some text, and ends
with a closed bracket (]).
Review week eleven lab assignment
[globals] Defines global parameters
[printers] Defines printers
[home]
Defines shares in the homes directory
Comments in the smb.conf can start with either a pound
(#) or (;).
# A private directory, usable only by Bob.
; [bobsdir]
;
comment = Bob’s Service
;
path = /usr/somewhere/in/space
;
writeable = yes
;
printable = no
;
public = no
Review week eleven lab assignment
What is the functionality of Network File System
(NFS)?
It is a remote file system designed by Sun
Microsystems, available on computers from most
UNIX system vendors.
It allows the server to share selected local
directory hierarchies with client systems on a
heterogeneous network. Files on the remote
computer (fileserver) appear as if they are present
on the local system. The physical location of a
file is irrelevant to an NFS user.
Review week eleven lab assignment
Network Services
A variety of services available across a network
NFS: Developed by Sun Micro Systems
Runs on UNIX, DOS, Windows, VMS, and Linux
Files on the remote computer appear as if they are present on the local
system.
Server configuration - /etc/exports file is the main NFS
configuration file and it consists of two columns:
Column #1 - lists the directories the network or DNS domains the
can get access to the directories.
Column #2 – lists NFS options in brackets.
#/etc/exports/data
/files
*(ro,sync)
/home
192.168.1.0/24 (rw,sync)
/data/test
*.my-site.com (rw,sync)
/data/database
192.168.1.203/32 (rw,sync)
Review week eleven lab assignment
Network Services
A variety of services available across a network
NFS:
Use the chkconfig command to configure the required
NFS daemons to start at boot.
Use exportfs –a command when no directories have been
exported to NFS.
Use exports –r command when adding a shared directory
to export only the new entries.
Use the showmount –a command to list all currently
exported directories.
In the /etc/fstab file, add a similar entry:
#/etc/fstab
192.168.1.100:/data/files /mnt/nfs nfs soft,nfssvers=2 o 2
Review week eleven lab assignment
Network Services
A variety of services available across a network
Remote login:
rlogin (remote login) is a UNIX command that allows an
authorized user to login to other UNIX machines (hosts)
on a network and to interact as if the user were physically
at the host computer. Once logged in to the host, the user
can do anything that the host has given permission for,
such as read, edit, or delete files.
SSH is a protocol that allows computers to communicate
with each other over encrypted connections. An SSH
client is used to log in to a remote machine (running an
SSH server) and allows the execution of commands on
that machine.
Review week eleven lab assignment
Is there an orphan process?
root
1 0 0 2011 ?
03:20:28 /sbin/init
dandrear 13087 13085 0 22:26 pts/3 00:00:00 grep init
root 11856 9595 0 21:13 ?
00:00:00 sshd: dandrear [priv]
root 11406 9595 0 20:58 ?
00:00:00 sshd: dandrear [priv]
root 10542 9595 0 20:18 ?
00:00:00 sshd: dandrear [priv]
dandrear 13090 13085 0 22:26 pts/3 00:00:00 sort -r
dandrear 13089 13085 0 22:26 pts/3 00:00:00 grep dandrear
dandrear 13088 13085 0 22:26 pts/3 00:00:00 ps -ef
dandrear 13085 11427 0 22:26 pts/3 00:00:00 /bin/ksh ./pid_ppid.sh dandrear
dandrear 13074 1 0 22:26 pts/4 00:00:00 client_ser
dandrear 11859 11858 0 21:13 pts/4 00:00:00 -ksh
dandrear 11858 11856 0 21:13 ?
00:00:00 sshd: dandrear@pts/4
dandrear 11427 11426 0 20:59 pts/3 00:00:00 -ksh
dandrear 11426 11406 0 20:59 ?
00:00:00 sshd: dandrear@pts/3
dandrear 10548 10547 0 20:18 pts/2 00:00:00 -ksh
dandrear 10547 10542 0 20:18 ?
00:00:00 sshd: dandrear@pts/2
Week ten, eleven, and twelve expected
outcomes
Upon successful completion of this module, the student will be
able to:
Manipulate user accounts.
Describe how cron is used to invoke repetitive processes.
Manipulate process structure including:
a. fork and exec,
b. Initialization process,
c. Background/foreground,
d. PS tool.
Explain basic UNIX security issues.
Describe disk and file system structure.
Use backup and restore archival operations on a system.
Establish network services.
Investigate the structure of the LDAP directory using LDAP
commands.
Next lab assignment
X-11 is the name of the window manager used
for most UNIX and Linux operating systems. A
window manager is software that interfaces the video,
keyboard and mouse drivers together to allow the
user to communicate with the computer via a GUI.
X-11 is unique in the sense that it runs over a
network connection, specifically over IP. This
enables a remote user to use his or her local XWindows environment to interact directly with the
other computer’s window manager as if they were
sitting on that machine.
Next lab assignment
X-11 uses a client-server model, where the
window manager is the server, and the user is using
various clients. Applications that can be run in the
graphical user interface are often written to use the X11 software libraries to enable them to be used
remotely. This is one of the strengths and power of
the Unix system. X-11 enables a remote system to
treat your local computer, it screen, keyboard and
mouse as just another set of terminal interface devices
that use an X-11 driver to communicate with the XWindows manager.
Next lab assignment
Next lab assignment
Disk and File System Structure
• Copy-on-Write
It is an optimization strategy for multiple users
can give the same pointer for a resource. The
primary advantage is that if a caller never makes
any modifications, no private copy is needed.
This strategy is used in virtual memory operating
systems pages in memory that might be modified
by either the process or its copy are marked copyon-write.
Next lab assignment
Network Services
A variety of services available across a network
Utilize a Web browser:
All network services rely on the ability to convert a host or
domain name to an IP address.
DNS are complex. The number of host names and IP
addresses in the Internet is very large.
DNS Local contains actual translations for the machines
in its local network
DNS Global contains more information about translations.
A single translation could involve several DNS before
resolving the IP address.
Next lab assignment
The Domain Name System (DNS) is a hierarchical
distributed naming system for computers, services, or
any resource connected to the Internet or a private
network. It associates various information with
domain names assigned to each of the participating
entities. Most prominently, it translates easily
memorized domain names to the numerical IP
addresses needed for the purpose of locating
computer services and devices worldwide. By
providing a worldwide, distributed keyword-based
redirection service, the Domain Name System is an
essential component of the functionality of the
Internet.
Next Lab Assignment
• UNIX/Linux file system is contained under the
root directory denoted by a slash “/”.
• Users don’t have to worry about the physical
locations of files.
• The system administrator must be familiar
with mounting and un-mounting storage space
(/mnt).
• NIS+ (Network Information Service) is a
directory service. Replaced almost entirely by
LDAP.
Next Lab Assignment
File System
1. It is an organization of data and metadata on a storage device.
2. It is the methods and data structure that an operating system
uses to keep track of files on a disk or partition; that is the way
the files are organized on the disk.
3. Think of a file system as a protocol. File systems give
meaning to the data on a particular storage device.
What is a block device?
It is a device in which the data that moves to and from occurs in
blocks (like a disk sector) and supports attributes such as
buffering and random access behavior. Block devices include
hard drives, CD-ROMs, and RAM disks.
What is a character device?
It is a device that does not have a physical addressable media.
Character devices include serial ports and tape devices, where
data is streamed character by character.
File System
Next Lab Assignment
Next Lab Assignment
Super Block
Contains information about each mounted file system.
The actual data structure in Linux is called struct vfsmount.
Information held is mount flags, mount time, device name,
pointer to super block and device block size. The Linux 2.0
kernel keeps a static array of such structures to store up to 64
mounted file systems.
Data Block
Data blocks are used to store the data for a file. There is
space only for a few data block numbers in the inode,
however if more are needed, more space for pointers to the
data blocks is allocated dynamically. These dynamically
allocated blocks are indirect blocks. The name indicates that in
order to find the data block, one has to find its number in the
indirect block first.
Next lab assignment
Directory tree
/
jones
dandrear
date
dev
tmp
bin
usr
wc
ksh
tty03
.profile
bin
foo
null
Next Lab Assignment
ext3 File System
Super Block
File Manager
Linux Virtual File System
Kernel
Next lab assignment
UNIX base root file system tree structure
/bin (commands)
/dev (devices)
/etc (system configuration/executables)
/sbin (boot commands)
/usr/sbin (administrative commands)
/home (users home directories)
/lib (shared libraries)
/mnt (temporary mount directory)
/opt (optional software)
/proc (processes)
/stand (boot-related files)
/var (spooling)
Next lab assignment
lost+found directory
The fsck utility is generally able to repair most
system damage and the frequency with which you
will find anything at all in your lost+found directories
has slowed significantly
If you have a system crash and the Linux file system
checker (fsck) recovers corrupt files they are placed
here.
Next lab assignment
Unix-like System Security
• Security on UNIX-like systems has focused on
the following areas:
Files and directories
umask
PATH variable
password authentication
UNIX to UNIX commands
One-way function
Next lab assignment
Disks versus File Systems
Introduction:
Application(s) interface with file systems and
not disks.
Why?
Because disks have shortcomings and are not
responsible for data recovery.
File systems must return the stored data to a
consistent state after an unplanned system
crash.
Next lab assignment
File System
Data recovery
Data loss
Fsync system call
The fsync() function requests that all data for the
open file descriptor named by fildes is to be
transferred to the storage device associated with the
file described by fildes. The nature of the transfer is
implementation-defined. The fsync() function shall
not return until the system has completed that action
or until an error is detected.
Next lab assignment
File System
Track caching
Shortcoming of track caching – the file system
expects the data to be stable store. A system crash
with data in the cache will not deliver the user
application integrity that was promised.
Ways to circumvent this shortcoming – use nonvolital memory and provide microcode restart
after power failures.
Next lab assignment
The problem with using non-volatile memory and
microcode restart is the cost. This solution is the
disk controller, but is rarely an option.
Newer Disks
Provide tag queuing – each request passed to the disk
driver is assigned a unique number. Most disk
controllers can accept at least 16 pending requests.
In combination, the fsync and tag queuing are
combined.
Tag queuing was first implemented in SCSI disks.
Enabling both reliability and speed. ATA disks
lacked tag queuing. Later ATA adopted a tag queuing
called Tag Command Queuing (TCQ).
Next lab assignment
Serial ATA has a new definition called Native Command
Queuing (NCQ) that has a bit set in the write command
that tells the drive if it should report completion when
media has been written or when cache has been hit. If
the driver correctly sets this bit, then the disk will display
the correct behavior.
Disk Sector Size
From 1950s to 2010 the disk sector size was 512. In
2010, the sector size was increased to 4,096.
Problems associated with this retrofit
Error rate per bit has risen, requiring the use of ever
longer correction codes. The error code must have
enough redundancy for each sector to handle a high
correction rate even though most sectors will not require
it.
Next lab assignment
Is the increase density worth the extra overhead?
Sectors are 8 times larger, which eliminates
88% of the sector start and stop headers. You
want to reduce the number of non-data bits on
the disk. In effect, using 4,096 byte sectors
doubles the amount of user data that can be
stored on a given disk.
Legacy File Systems
The 4,096-byte sectors have to emulate the old
512-byte sector disks. This mode causes the
disk to be at least 50% slower.
Next lab assignment
How to avoid the slowdown!
Files smaller than or equal to 512-bytes are
accumulated in a 4,096-byte block.
Conclusion
Your file system must know the type of disk
technology to which they are running to ensure
reliable data retrival and performance.
Use flash-memory technology!
Break-out problems
1. How often do the following two commands execute?
# 1.1. This is a comment.
* * * * * c:\tools\CheckForNewFiles.exe
# 1.2. This is a comment.
* * 3 * * c:\tests\backup.bat
2. Define: File System Super block
3. Define: Domain Name Server
4. Define: Domain name
4. Define: RFC
6. What application is BIND associated
7. What is the “lost+found” area considered on a Unix-like
system?
8. Linux Virtual File System
9. One-Way Encryption or One-Way Function
10. Compare a block device to a character device
Upcoming deadlines
Section number V1WW:
Lab Assignment 11-1, Knoppix File System Exercise
is due December 1, 2013.
Lab Assignments 12-1 and 12-2 are due December 8,
2013
Lab Assignment 14-1, CGI Script is due December
15, 2013.
Module 6, contains study information for weeks 10,
11, and 12.
Lab assistance
• Questions
• Comments
• Concerns
• I will be available after this Franklin Live
session to discuss any problems and/or
concerns regarding lab assignments.