Transcript Part I: Introduction

Sharing Unix File Systems
 NFS (Network File System ) allows you to share
file systems
 Reasons for sharing file system

Transparent to user
• User can keep use their familiar commands
• Access the same file from multiple nodes.





To provide disk space to diskless clients
To prevent duplication
To provide centrally supported programs and data
To share data among users
Simplifies central support tasks
• Such as backup.
• Space usage monitor
NFS
1-1
NFS
 Candidates could be:
 Home dirs,
 Web page,
 /usr/local and other common utilities
 Documentations
 Temporary huge space request
 Source code repository
 ……
NFS
1-2
NFS protocol versions
 NFS was introduced by Sun Microsystems in 1985
 The original public release of NFS was version 2
 In 1990s, version 3
 Increases performance
• Makes writes safely asynchronous

Better support for large files
 version 4
 No ancillary protocols – integrated locking and mount
 Compound operations – bundle multiple RPC together in a
single exchange
 Strong security – uses RPCSSEC_GSS API
 Require use of transport protocols that offer congestion
control – hence NFS v4 will not support UDP transport.
NFS
1-3
More about NFS
 NFS run on top of Sun’s RPC (Remote Procedure
Call) protocol

RPC provides a system-independent way for processes to
communicate in a client-server fashion over a network.
 Some NFS versions run on top of UDP and TCP
 Why?
 WebNFS
 Promote the use of NFS over the Internet
 A extention of NFS protocol that allow easier access to
servers and clients through Internet firewalls
 A WebNFS-enhanced web browser can use an NFS URL
to access data directory from the server
– nfs:/www.YourCompany.com/
NFS
1-4
Security and NFS
 NFS protocol was originally designed with
essentially no concerns for security
 Tight control over shared filesystems
Don’t export any filesystems if anyone that you
don’t trust has root access on a client host
 If you have a firewall, block port 2049, block
SunRPC portmap deamon port 111

 Kerberos to authenticate to ensure that
remote users really are who they say they
are.
 Ipsec to encrypt the data
NFS
1-5
Network File System
 NFS consists of a number of components
Mounting protocol
 Mount server
 Daemons that coordinate basic file service
 Several diagnostic utilities
 Basically two parts:


Client
• Mount a directory

Server
• Export a directory
NFS
1-6
Server-side NFS
 Mount a filesystem
 Daemon: mountd
 Access files
 Daemon: nfsd
 Both daemon started at boottime
 Mountd and nfsd use the same database ( (xtab on
most systems, sharetab on Solaris)
 There are commands (exportfs, share) to
add/remove the entries in the database

Exportfs reads /etc/exports file
• Run exportfs –a
• To remove entries, run exportfs –u
NFS
1-7
Server-side NFS

On Solaris, /etc/dfs/dfstab is a shell script
• Run shareall command, which equivalent to sh
/etc/dfs/dfstab


To remove entries, run unshare
Solaris also provides command “exports”, which
translates export options to share/unshare commands
 Any directory can be exported
 Client can mount a subdirectories of an exported
directory
• Example: On HP-UX 11i
You can do above, but you may not to mount another sub dir from
the same exported directory.
 Each device to be exported separately
 For example
• /user is a separate partition, / can be exported without
exporting /users
NFS
1-8
NFS Daemons on Solaris

The following daemons are running on server side
• nfsd [nservers]
• mountd
• nfslogd

The following daemons are running on both server side
and client side
• rquotad
• lockd
• statd

How to start the daemons:
• Daemons are under /usr/lib/nfs
• Daemons are started at boot time by /etc/init.d/nfs.client
and nfs.server
• nfs.client starts statd and lockd
• nfs.server starts nfsd (16 copies default), mountd, nfslogd
• inetd start rquotad.
NFS
1-9
Configure NFS on Solaris
 The share command to export a dir
share –F nfs [-o options] pathname
 Options are:
 rw
• Export read-write to the entire world

rw=accesslist
• Export read-only with access only by listed hosts



ro
ro=accesslist
root=list
• Lists hosts permitted to access this filesystem as root
• Otherwise, root access from a client is equivalent to access
by “nobody” (UID –2)
NFS
1-10
Configure NFS on Solaris

anon=uid
•

defines the uid used for users who do not provide a valid user ID.
nosub
• Forbids clients to mount subdirectories of the exported
directory

nosuid
• Prevents setuid and setgid files from being created via NFS
 Access list format
 Separated by :
 Identify computers by
•
•
•
•
individual hostnames/IP
Domain, .domain
NIS netgroup, groupname
Network, @network[/prefix]
NFS
1-11
Configure NFS on Solaris
 Examples: on Colossus /etc/dfs/dfstab
share -F nfs -o
root=aa.aaa.aa
rw=keeper.csl.mtu.edu:csl:scifi:language:pizza:icu:cs:fishlab:cec
/home/csdept
share -F nfs -o
root=xx.xxx.xx:yy.yy.yyy
rw=csl:language:scifi:pizza:icu:cs:fishlab:physpecial:cec:cs.mtu.edu
/export/major
NFS
1-12
Configuring NFS on Linux
 /etc/exports
 The NFS server configuration file
 Controls which files/dirs are exported
 Access control
 The general format of entries:
directory [host(option)]
 Host can be
•
•
•
•
•

empty, means every host
individual host, name or IP address
Domains, *wrotethebook.com
Networks, IP address/mask(prefix length)
Netgroups, @group1
Option can be
• ro
• rw
NFS
1-13
LINUX NFS Permission Control
 NFS server trusts local authentication
 USE GID/UID to do regular Unix permission
control.


Mismatch uid/gid will be a unexpected problem
What about root?
• Do you want root users on NFS client to act like root users
on NFS servers? Probably Not.
• NFS prevents by default setting root_squash
– Mapping root to nobody UID/GID
• Set no_root_squash to allow it.

Option all_squash
• Map all uids and gids to the anonymous user. Useful for
NFS-exportd public FTP directories, news spool directories,
etc.
NFS
1-14
LINUX NFS
 Assign anonymous a UID/GID using option
anonuid=XXX, anongid=XXX
This option is primarily useful for PC/NFS clients, where
you might want all requests appear to be from one user.
 UID/GID mapping file using option
map_static=filename
 Command exportfs,
 build /var/lib/nfs/xtab
 export all in /etc/exports file using “exportfs –a “
 Sync /etc/exports and /var/bin/nfs/xtab “exportfs –r”
 Temporarily export a filesystem
• exportfs fox:/usr/local –o rw


remove from export list “exportfs –u fox:/usr/local”
remove all export list “exportfs –ua “
NFS
1-15
Client-side NFS
 Mounting remote filesystems
Use mount command to establish temporary
network mounts
 List in /etc/fstab or /etc/vfstabb
 Automatic mounting services such as automount

 what are exported or where
 Ask administrator
 Find out yourself:
• what have been exported from a host:
showmount –e hostname
NFS
1-16
Mounting Remote Filesystems
 Using mount command
Basic format:
mount hostname:remote-dir local_dir
•
•
hostname is the NFS server
local_dir must exist already
 Using umount command
umount local_dir[remote_dir]
NFS
1-17
Mounting NFS
 On Solaris, entries in /etc/vfstab are mounted by
mountall during system startup
 On Linux, entries in /etc/fstab are mounted via
mount –a in startup files.
 NFS mount flags ( page 502)

Options:
•
•
•
•
rw
ro
bg
hard
– cause the operations that try to access it to block until the server
comes backup
• Soft
– What about a job that has been running 18 hours and will be
done in an hour and aborted?
NFS
1-18
Mounting NFS

More mounting options
• intr
– Allow user to interrupt blocked operations
• nointr
• retrans=n
– Specifies the number of times to repeat a request before
returning an error on a soft-mountd filesystem
• timeo=n
– Set the timeout period for requests
• rsize
– 8K for the same network
• wsize
• vers
• tcp
NFS
1-19
Common mount options on Linux
Option
Function
async
auto
When –a is used
dev
Allow character and block special files on this file system
exec
noauto
Don’t mount even with -a
noexec
nosuid
Don’t allow setuid, setgid
nouser
Only root can mount
remount
ro
rw
suid
Allow programs to run setuid or setgid
sync
user
Ordinary users to mount the file system
atime
Update inode
noatime
defaults
Rw, suid, dev, exec, auto, nouser, and async
NFS
1-20
Dedicated NFS file Servers
 Fast, reliable file service is one of the
most important elements
 Dedicated NFS servers
Optimized for file service
 Storage scale smoothly
 Reliable, simplified software
 Redundant hardware
 Support different file formats for different
platforms, such as NFS, CIFS
 Easier to administer than UNIX file servers
 Good backup and checkpoint facilities

NFS
1-21
NFS Automounter
 Automatically mounts NFS when needed
 Why automount?
• Maintaining /etc/fstab can be tedious
• Minimizes the number of active mount points to
reduce/Avoid chaos when servers crashes

How?
• Mount a virtual filesystem driver on the directories for
automatic mount to occur
– Kernel-resident filesystem driver called autfs is used
• When the user references a directory withing the virtual
filesystem, the automounter intercepts the reference and
mount the actual filesystem the user is trying to reach.

NFS
1-22
NFS Automounter
 Daemons

automountd and automount are started by
/etc/init.d/autofs
• Automount reads the configuration file, setup autofs
mounts
• References to automounted filesystems are handled
by a separate daemon automountd

On solaris 10, you can start automounter
• #svcadm enable system/filesystem/autofs
• The actual script is under /lib/svc/method/svcautofs
NFS
1-23
automount
 automount
 Basic configuration files
• Master map
• Direct maps
• Indirect maps
Direct and indirect maps provide information
about filesystems that are to be automounted
 A master map lists direct and indirect maps
that automount should pay attention to.

• Only one master map that can be active
• The default one is /etc/auto_master
NFS
1-24
automount

Indirect maps
•
•
•
•
Automount several filesystems underneath a common
directory
The path is specified in the master maps, not in the
indirect map itself
The name indirect map will be used as subdirectory
to install the mount
– Example, indirect map
Info –ro chimchimL/chimchim/info
Using ls a directory full of indirect mounts can be
confusing
» automount does not show the subdirectories until
their contents have been accessed
» What does ls do?
NFS
1-25
automount

Direct maps
• List the maps that do not share a common prefix
– Such as:
/cs/tools –ro anchor:/cs/tools
/usr/src chimchim:/usr/src
• Each implemented with a separate autofs mount
• Require slightly overhead
• But mount point and directory structure are always
accessible by commands such as ls
NFS
1-26
automount

master map
• format: mount-point map-name option
– List the direct and indirect map
– For indirect map, it specify root directory used by the
mounts defined in the map.
– /- is for direct maps
– Option set the defaults for all mounts within the map
[ruihong@cslserver nis]$ more /etc/auto_master
# cs and csl don't use auto_home and auto yet.
#/net -hosts -nosuid,nobrowse
#/home auto_home
-nobrowse
#/local auto_local
-nobrowse
/fisher auto_fisher
/auto_direct
NFS
1-27
automatic automount
 Have automount to figure it out itself
 Query mountd running on a remote server to find out
what filesystems the server exports
 Use –host as a map name, automount will map remote
hosts’exports into the specified automount directory
• Example
The serer chimchim exported /usr/share/man
The client’s master map has a line
/net -hosts –nosuid,soft
The mount point will be
/net/chimchim/usr/share/man
• It does enumerate all possible hosts
• It waits for individual subdirectory names to be
references,then runs off and mounts the exported
filesystem
NFS
1-28
automatic automount

Key substitutions
• Impress a degree of regularity on the automounter
maps
• The ampersand (&) expands to the matched key value
in a map
• The asterisk(*) is a default case.
• Example: a indirect map:
usr1-rw
thud:/export/home/usr1
usr2-rw
thud:/export/home/usr2
usr3-rw
thud:/export/home/usr3
usr4-rw
thud:/export/home/usr4
Can be rewritten as only one line
* -rw thud:/export/home/&
NFS
1-29
NFS Automounter
 Automount can use maps from NIS server.
 Other type maps, like auto_home mount
# Master map for automounter
#
#+auto_master
#/xfn -xfn
/net -hosts -nosuid
/home auto_home
/- auto_direct
# Home directory map for automounter
#
#+auto_home
craig
almond:/export/home/craig
*
pecan:/export/home/&
NFS
1-30
Practice
In Lab4, we created a dir called /research. As a
group, let’s try the following tasks
 Export /research, so




some hosts in your group as rw
some hosts in your group as ro
one extra host have root mapping
Access the exported from /net/hostname/home
 Mount /research from some hosts in your group
 Automount the exported /research to /research
from some hosts in your group
NFS
1-31
Network Information Service
 NIS
Was originally called the Sun Yellow Pages
 Is an administrative database

• What information does database store?
 Provides
central control and automatic
dissemination of important files.
NFS
1-32
Network Information Service
 The unit of sharing in NIS is the record, not the
file.

A record is usually corresponds to one line
 Mast server maintains the authoritative copies of
system files


Are kept in their original locations and formats
Edited with a text editor before
 A server process makes the contents of the file
available over the network.
 A server and its clients constitute an NIS
“domain”


associated with A set of maps
Different from DNS domain
NFS
1-33
Network Information Service
 On the master server

Data file are preprocessed into database files
(called maps) by a hashing library
• One key associated with each entry
• A system file may have to be translated into several
NIS maps
– Example:
» /etc/passwd is translated into two different NIS Maps
passwd.byname
passwd.byuid
• The record order is not preserved.
NFS
1-34
Network Information Service
 On the slave server
 Relieve load on the master and keep clients functioning
when master is down.
 Replicate the network maps from master server
 On the client
 Does not distinguish between the master server and
slaves
 Traditional
• Use IP broadcasting to locate servers
• Place at least one NIS server on every physical network

Some implementation allows to specify the server name
instead of broadcasting
NFS
1-35
Network Information Service
 Databases are called NIS maps
/etc/passwd
 /etc/group
 /etc/netgroup
 /etc/hosts
 /etc/networks
 /etc/protocols
 /etc/services
 /etc/aliases
 /etc/auto_master
… …

NFS
1-36
Network Information Service
 Netgroups
 Sets of users, machines, and nets for easy reference in
other system files
 The format of netgroup entry is
Groupname list-of-members

Members are separated by whitespace. A member is
• Either a netgroup name
• Or a tripnet of the form
(hostname, username, nisdomainname)
a empty value in a field matching any value in that field

Netgroup can be used in several system files that define
permissions
• Configuring NFS export ( /etc/exports file)
• Restrict remote logins and shell access ( /etc/hosts.equiv
file)
• Local login access ( /etc/passwd file)
NFS
1-37
Network Information Service
 Exmaples
 /etc/netgroup
grouplab \
(goofy.csl.mtu.edu,,) (mickey.csl.mtu.edu,,) \
(minnie.csl.mtu.edu,,) (donald.csl.mtu.edu,,) \
(simba.csl.mtu.edu,,) (nala.csl.mtu.edu,,) \
(tramp.csl.mtu.edu,,)
noncs1121d \
(,msbenson,) (,behart,) (,adrogers,) (,agwalthe,) (,adpeters,) (,ammurrel,) \
(,aeblechi,) (,jaeul,) (,srkelley,) (,daperry,) (,anlevend,) (,frtuntla,) \
(,cwminnic,) (,rrmilkov,) (,iatrifon,) (,ksrivast,) (,talange,) (,mtnielse,) \
(,csdummy,)

/etc/dfs/dfstab
share -F nfs -o
root=aa.aaa.aa
rw=keeper.csl.mtu.edu:csl:scifi:language:pizza:icu:cs:fishlab:cec
/home/csdept

/etc/passwd
+@nonenrolled::::::/usr/local/shells/gone
+@cs1121nons::::::/usr/local/shells/limitaccess
NFS
1-38
Network Information Service
 How NIS works
 NIS’s data files are stored in one directory
• Usually /var/yp

Each NIS map is stored in a hashed format in a
subdirectory named for the NIS domain
• Exact Map files names depends on the hashing library being
used.
• For example:
– On cslserver, under /var/yp/csl.mtu.edu, there are ndbm files:
password.byname.dir
password.byname.pag
auto.direct.dir
auto.direct.pag
NFS
1-39
Network Information Service

There is a makefile under /var/yp
• Which calls makedbm to generate NIS maps from flat files.
• After you modify a system file, cd to NIS dir which is
/var/yp, and run make.
– Make checks the modification date and rebuild some maps.


Slave server run ypxfr command regularly as cron to pull
the map copies
Server runs yppush to instruct each slave server to
execute ypxfr.
• Yppush is used in makefile after rebuilding a updated map.
• a special map called ypservers
– no correspond flat file
– constructed automatically
NFS
1-40
Network Information Service
 After the initial configuration

Deamon ypserv running only on the servers
• Accept queries from clients and answers them by
looking up information in the hashed map files

Deamon ypbind running on every machine,
include servers
• C library contacts the local ypbind to answer a query
– provided /etc/nsswitch.conf says to check with nis
• Ypbind locates a ypserv and returns its identify to
the C library
• C library contacts the server directly
NFS
1-41
NIS commands and daemons (page 527)

















ypserv
ypbind
domainname
ypxfr
ypxfrd
yppush
makedbm
ypmake
ypinit
ypset
ypwhich
ypcat
ypmatch
yppasswd
ypchfn
ypchsh
yppassdd
NIS server daemon
NIS client daemon
Sets the NIS domain a machine is in
Downloads current version of a map from master
Serves request from ypxfr (runs on master server)
Makes slave servers update their versions of a map
builds a hashed map from a flat file
rebuild map
Configure the host as a master or slave server
Makes ypbind connect to a particular server
find out which server the current host is using
Prints the values contained in an NIS map
prints map entries for a specified key
Changes a password on the NIS master server
Changes GECOS info on the NIS master server
Changes a login shell on NIS master server
Server for yppasswd, ypchsh, and ypchfn
NFS
1-42
Set UP NIS
 Procedure
 Prepare the source file
 Initialize master server
• Run ypinit -m
• Starting NIS service
– Run ypserv

Initialize slave server
• Set domainname
• Run ypinit command with –c and master serve ip.

Setup NIS clients
•
•
•
•
Setup default domainname
/etc/nsswitch.conf
Run ypinit –c
Start ypbind daemon
– Run ypbind
NFS
1-43
NIS
 Case study:

Let’s take a look at where the startup files for
starting NIS server and NIS client.
• /etc/init.d/rpc
• /usr/lib/netsvc/yp/ypstart (ypstop)

Maps
• /var/yp/csl.mtu.edu

Let’s take a look at the make file
NFS
1-44
NIS
 Work with DNS
 Hostname can be converted to IP address by DNS, NIS,
and the host file.
 /etc/nsswitch.conf
 Define the order in which the sources are searched.
Choices are:
• Dns
• Nis
• Files
For example:
hosts: dns nis files
networks: nis [NOTFOUND=return] files
services: nis files
protocols: nis files
NFS
1-45
NIS+
 NIS+ is a completely new software product and
structured entirely differently from NIS



Sun’s new administrative database released in early 90s
Provides all the functionality of NIS
Improved security
•
•
•
•


NIS does not authenticate servers or clients
NIS+ use secure DES
NIS provide same access to everyone in the domain
NIS+ provide access levels to different users
NIS+ is a hierarchical decentralized architecture.
NIS+ has enhanced data structures.
• Multiple columns table and relation query between tables

NIS+ is complex and not as popular as NIS
NFS
1-46
NIS practice
Refer to the Sun document

“System Administration Guide: Naming and
Directory Service “
• section “ NIS Setup and Administration”
 As a group
 Set up a NIS server and a slave server
 On the NIS server
• create a user account named visitor1 with home dir
/research/visitor1
 Set
up a NIS client
• Modify /etc/nsswitch.conf
– To have passwd line to include nis
NFS
1-47
Copying files around
 Not elegant solution, but it works

Example:
• A change to a startup script
• A new version of software
 File copying systems can use “push” model
or “pull” model



rdist: push files
rsync: push files more securely
expect: pull files
NFS
1-48
Copying files around
 Rdist

The easiest way to distribute files from a
central server.
• Sort of like make
– A text specification of the files to be distributed
– Rdist copies file when they are out of date.
• Preserves owner, group, mode and modification time
of files.
• When updating an existing file, it first deletes the
old version before installing the new
– Makes rdist suitable for transferring executables that might
be in use during the update.
NFS
1-49
Copying files around

It runs on rsh and relies on rsh-style
authentication to gain access to remote system
• The protocol can also be changed from command line
rdist -P /usr/local/bin/ssh -f myDistfile

Control file has the statements of the form
Label: pathnames -> destination commands
Where:
» label: associate a name with the statement
» pathname: lists of files to be copied
» destination: list of hosts to copy files to
» commands: modify the behavior ( default is to copy
files to the equivalent paths)
NFS
1-50
Copying files around
EXAMPLE:
SYS_FILES = (/etc/passwd /etc/group /etc/mail/aliases)
GET_ALL = (chimchim lollopop barkadon)
GET_SOME = (whammo spiff)
all: ${SYS_FILES} -> ${GET_ALL}
notify barb;
special /etc/mail/aliases “/usr/bin/newaliases”
some: ${SYS_FILES} -> $ {GET_SOME}
except /etc/mail/aliases;
notify eddie@spiff;
NFS
1-51
Copying files around
 Rsync: push files
 Flexiblie and faster replacement of rcp
• Only transmit the differences
• Support for copying links, devices,owners, groups and
permissions
• Can use any transparent remote shell rsh, ssh
• Can use rsync daemon mode



No controlfile
Must run repeatedly to transfer a set of files to multiple
hosts.
Example:
• #rsync –gopt –password-file=/etc/rsync.pwd /etc/passwd
lollipop::sysfiles
• #rsync –gopt –e ssh /etc/passwd /etc/shadow lollipop:/etc
NFS
1-52
Copying files around
 Expect: pull files
 Expect is a TCL extension scripting language
• Allows to you write control scripts for interactive
programs
– The output produced by each operation is examined to
determine what input should be send next

Use expect to retrieve and install them
• Fundamental expect commands are:
– spawn : start up a subprocess to control
– send: feed input to a subprocess
– expect: take action depending on a subprocess’s output

Example: Make system files available via FTP
from central server
NFS
1-53
Copying files around
Example: ftps the /etc/passwd file from the machine netserver
spawn /usr/bin/ftp netserver
while 1 { expect {
“Name *:” {send “netclient\r”}
“Password:” {send “netclientpassword\r”}
“ftp> “ {break}
“failed” {send_user “Can’t log in .\r”; exit 1}
timeout {send_user ‘Timeout problem. \r”; exit 2}
}}
send “lcd /etc\r”
expect “ftp>” {send “cd pub/sysfiles\r”}
expect “ftp>” {send “get passwd\r”}
expect “ftp>” {send “quit\r”, send_user “\r”}
exit 0
NFS
1-54
Summary
 NFS
 NIS
 Copying file around
NFS
1-55