Transcript Introduction

CIT 470: Advanced Network and
System Administration
Distributing Files
CIT 470: Advanced Network and System Administration
Slide #1
Topics
1.
2.
3.
4.
5.
Sharing Files
Copying Files: push vs pull
rsync
Network Filesystems
Administering NFS
CIT 470: Advanced Network and System Administration
Slide #2
Sharing Files
System files
– Centralize administration: shared logins, naming.
– Solution: copy files between machines.
– Alt Solution: Directory services (LDAP.)
User files
– User wants access to files on every machine.
– Solution: copy files between machines.
– Alt Solution: Network filesystems.
CIT 470: Advanced Network and System Administration
Slide #3
Copying Files
Advantages
– No network services to set up.
– Works everywhere.
Decisions
– Push vs Pull
Solutions
–
–
–
–
ftp
wget
ssh
rsync
CIT 470: Advanced Network and System Administration
Slide #4
Automating ftp
#!/usr/bin/expect
spawn ftp mysvr.nku.edu
expect “username:”
send “ftp\r”
expect “password:”
send “[email protected]\r”
expect “ftp>”
send “bin\r”
expect “ftp>”
send “prompt\r”
expect “ftp>”
send “mget *\r”
expect “ftp>”
send “bye\r”
expect eof
CIT 470: Advanced Network and System Administration
Slide #5
wget
Non-interactive file retrieval
– Protocols: ftp, http, https.
– Useful for automating file xfer in scripts.
– Ex: wget http://svr.nku.edu/files/etc/hosts
Options
–
–
–
–
–
Authentication and proxying.
Quiet
Recursive: follows links in HTTP documents.
Resume
Retries
CIT 470: Advanced Network and System Administration
Slide #6
ssh-based copying
• Securely copy files to/from another host.
• Limitations
– scp copies list of files on command line (-r for
recursive) to single destination.
– Copies all files, not just updated files.
– Must share keys to authenticate securely.
– sftp most suited for manual fs exploration.
CIT 470: Advanced Network and System Administration
Slide #7
rsync
• Synchronizes file trees between machines.
• Advantages
–
–
–
–
Makes remote tree identical to local one.
Only copies files that have been changed.
Only copies file parts that have been changed.
Useful for local mirroring, staging dirs, &c too.
• Transport Mechanisms
– rcp: insecure, avoid.
– scp: secure, commonly used.
– rsync: rsync protocol, best for anonymous use.
CIT 470: Advanced Network and System Administration
Slide #8
rsync over ssh
Push
rsync -av -e ssh local root@svr:test
Pull
rsync -av -e ssh root@svr:test local
Test
rsync -avn -e ssh root@svr:test local
CIT 470: Advanced Network and System Administration
Slide #9
Fine tuning rsync
Deleting removed files (be careful)
rsync -av -e ssh --delete local
root@svr:test
Excluding unwanted files.
On the command line
rsync -av -e ssh --exclude=“*.bak” -exclude=".?*.sw?” local
root@svr:test
Through a file
rsync -av -e ssh --excludefrom=~/exclude-list local
root@svr:test
CIT 470: Advanced Network and System Administration
Slide #10
rsync server
Setting up an rsync server
– Create an rsyncd.conf file.
– Server: rsync --daemon
– Client: rsync svr::public/new.tgz .
Simple, but be careful about security.
– Often secure by DNS name or IP address.
– Can secure by user with rsync secrets file.
– No encryption (need to use ssh tunnel.)
CIT 470: Advanced Network and System Administration
Slide #11
rsyncd.conf
# "global-only" options
syslog facility = local5
# global options which may also be defined in modules
use chroot = yes
uid = nobody
gid = nobody
max connections = 20
timeout = 600
read only = yes
# module: [public]
path = /home/rsync
comment = Tarball archive
hosts allow = *.nku.edu, 10.18.3.0/24, 10.30.4.4
ignore nonreadable = yes
refuse options = checksum
dont compress = *
CIT 470: Advanced Network and System Administration
Slide #12
Other File Distribution Systems
rdist
– Older tool like rsync but slower, fewer features.
unison
– Unlike rsync, handles updates on both sides.
– Conflict resolution like CVS to handle case
when file is modified on both sides.
cfengine
– Maintains state of system according to policy.
– Copies files as needed to meet policy.
CIT 470: Advanced Network and System Administration
Slide #13
Automating File Copying
Write a cron job.
– Script can verify data before/after copy too.
How to deal with many machines?
– Add a random delay using a simple script:
#!/usr/bin/perl
# sleep 0-15 minutes (0-900s)
sleep rand() * 900;
CIT 470: Advanced Network and System Administration
Slide #14
Network Filesystems
Idea: Use filesystem to transparently share files
between computers.
Solution:
– Client mounts network fs as normal.
– Client filesystem code sends packets to server(s).
– Server responds with data stored on a regular ondisk filesystem.
CIT 470: Advanced Network and System Administration
Slide #15
NFS
Network File System
– Transparent, behaves like a regular UNIX filesystem.
– Uses UNIX UIDs,GIDs,perms but can work on Win.
– Since NFS is stateless, file locking and recovery are
handled by rpc.lockd and rpc.statd daemons.
Security
–
–
–
–
Server only lets certain IP addresses mount filesystems.
Client UIDs have same permissions on server as client.
Client root UID is mapped to nobody, but
Root can su to any client UID to access any file.
CIT 470: Advanced Network and System Administration
Slide #16
CIFS
Microsoft Network Filesystem




Derived from 1980s IBM SMB net filesystem.
Originally ran over NetBIOS, not TCP/IP.
\\svr\share\path Universal Naming Convention
Auth: NTLM (insecure), NTLMv2, Kerberos
Implementation


MS Windows-centric (filenames, ACLs, EOLs)
Samba: UNIX client and server software.
CIT 470: Advanced Network and System Administration
Slide #17
AFS
Distributed filesystem
– Global namespace: /afs/abc.com/vol_home1
– Servers provide one or more volumes.
– Volume replication with RO copies on other svrs.
Cells are administrative domains within AFS.
– Cells contain multiple servers.
– Each server provides multiple volumes.
Security
– Kerberos authentication
– ACLs with user-administered groups
CIT 470: Advanced Network and System Administration
Slide #18
NFSv4
New model of NFS
–
–
–
–
–
Only one protocol (no separate mount,lock,etc.)
Global namespace.
Security (ACLs, Kerberos, encryption)
Cross platform + internationalized.
Better caching via delegation of files to clients.
CIT 470: Advanced Network and System Administration
Slide #19
Adminstering NFS
1.
2.
3.
4.
5.
6.
7.
NFS Versions
Using NFS
NFS Services
Server and Client Configuration
Automounter
Security
Performance
CIT 470: Advanced Network and System Administration
Slide #20
NFS Verions
v2 (1984) UDP 32-bit
v3 (1992) TCP 64-bit.
v4 (2000) Distributed, x-platform, security.
CIT 470: Advanced Network and System Administration
Slide #21
Using NFS
Client
1.
2.
3.
4.
5.
Start portmap
…
…
…
Mount filesystems.
Server
1.
2.
3.
4.
Start portmap
Start NFS services.
Configure exports.
Export filesystems.
CIT 470: Advanced Network and System Administration
Slide #22
NFS Services
portmap — RPC service for Linux
portmap
nfs — NFS file server processes.
rpc.mountd
rpc.rquotad
nfsd
nfslock — Optional file locking service.
rpc.statd
CIT 470: Advanced Network and System Administration
Slide #23
NFSv2/3 Processes
rpc.mountd — Handles client mount requests.
rpc.nfsd — NFS server processes.
rpc.lockd — Process for optional nfslock service.
rpc.statd — Handles server crashes for nfslock.
rpc.rquotad — Quotas for remote users.
CIT 470: Advanced Network and System Administration
Slide #24
rpcinfo
> rpcinfo -p
program vers
100000 2 tcp
100000 2 udp
100021 1 udp
100021 1 tcp
100011 1 udp
100011 2 udp
100011 1 tcp
100011 2 tcp
100003 2 udp
100003 3 udp
100003 2 tcp
100003 3 tcp
100005 2 udp
100005 2 tcp
100005 3 udp
100005 3 tcp
proto port
111 portmapper
111 portmapper
32774 nlockmgr
34437 nlockmgr
819 rquotad
819 rquotad
822 rquotad
822 rquotad
2049 nfs
2049 nfs
2049 nfs
2049 nfs
836 mountd
839 mountd
836 mountd
839 mountd
CIT 470: Advanced Network and System Administration
Slide #25
NFSv4 Processes
nfsd — NFSv4 server processes. Handles mounts.
rpc.idmapd — Maps NFSv4 names
(user@domain) and local UIDs and GIDs. Uses
/etc/idmapd.conf.
rpc.svcgssd — Server transport Kerberos auth.
rpc.gssd — Client transport Kerberos auth.
CIT 470: Advanced Network and System Administration
Slide #26
Server Configuration
1. Configure /etc/exports
List filesystems to be exported.
Specify export options (ro, rw, etc.)
Specify hosts/networks to export to.
2. Export filesystems.
exportfs
3. Start NFS server (if not already started)
service portmap start
service nfs start
CIT 470: Advanced Network and System Administration
Slide #27
/etc/exports
Format: directory
Options
ro, rw
async
sync
all_squash
root_squash
no_root_squash
anon{uid,gid}
hosts(options)
Read-only, read-write.
Server replies before write.
Save before reply (default)
Map all users to anon UID/GID.
Map root to anon UID (default)
Don’t map root (insecure.)
Set anonymous UID, GID.
Examples:
/home
*.example.com(rw,sync)
/backups
192.168.1.0/24(ro,all_squash)
/ex/limited
foo.example.com
CIT 470: Advanced Network and System Administration
Slide #28
Client Configuration
Manual mounting
mount -t <nfs-type> -o <options>
server:/remote/export /local/directory
Mounting via /etc/fstab
server:/remote/export /local/directory <nfs-type>
<options> 0 0
NFS Type is either nfs or nfs4.
CIT 470: Advanced Network and System Administration
Slide #29
Mount Options
hard or soft — Error handling
hard: NFS requests will uninterruptible wait until server back.
soft: NFS requests will timeout and report failure.
intr — NFS requests can be interrupted if server unreachable.
nfsvers=2,3— NFS protocol version (not 4)
noexec — Prevents execution of binaries.
nosuid — Disables setuid for security.
rsize,wsize=# — NFS data block size (default 8192)
sec=mode — NFS security type.
sys uses local UIDs and GIDs.
krb5 uses Kerberos5 authentication.
krb5i uses Kerberos5 authentication + integrity checking
krb5p uses Kerberos5 auth + integrity checking + encryption.
tcp, udp — Specifies protocol to use for mount.
CIT 470: Advanced Network and System Administration
Slide #30
Automounter
Manages NFS mounts
Automounter maps vs /etc/fstab.
Mounts filesystems only when needed:
Makes administering many filesystems easier.
Improves startup speed.
Provides uniform namespaces.
Ex: mounts /home/home7 as /home on login.
/etc/auto.master points to maps
/home /etc/auto.home
Maps describe mounts
* -fstype=nfs4,soft,intr,nosuid server:/home
CIT 470: Advanced Network and System Administration
Slide #31
Security
Limit which hosts have access to filesystems.
– Specify hosts in /etc/exports.
– Use iptables to limit which hosts can use NFS.
Limit mount options
– Default to ro unless writes are necessary.
– Disable suid and execution unless needed.
– Map root to nobody.
Block NFS at network firewalls.
– Block all protocols, not just port 2049.
Use NFSv4 with Kerberos auth + encryption.
CIT 470: Advanced Network and System Administration
Slide #32
Performance
Measuring performance
nfsstat
/proc/net/rpc/nfsd
Optimizations
–
–
–
–
–
Increase the block size. Problem: fragments?
Set the async option on mounts.
Faster network card.
Faster disk array.
NVRAM cache on array to save NFS writes.
CIT 470: Advanced Network and System Administration
Slide #33
References
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
Michael D. Bauer, Linux Server Security, 2nd edition, O’Reilly, 2005.
cfengine, http://www.cfengine.org/
Mike Eisler, Ricardo Labiaga, Hal Stern, Managing NFS and NIS, 2nd edition,
O’Reilly, 2001.
expect, http://expect.nist.gov/
Aeleen Frisch, Essential System Administration, 3rd edition, O’Reilly, 2002.
Evi Nemeth et al, UNIX System Administration Handbook, 3rd edition,
Prentice Hall, 2001.
NFS HOWTO, http://nfs.sourceforge.net/nfs-howto
RedHat, Red Hat Enterprise Linux 4 System Administration Guide,
http://www.redhat.com/docs/manuals/enterprise/RHEL-4-Manual/sysadminguide/, 2005.
RedHat, Red Hat Enterprise Linux 4 Reference Guide,
http://www.redhat.com/docs/manuals/enterprise/RHEL-4-Manual/refguide/ch-nfs.html, 2005.
rsync, http://www.samba.org/rsync/
Unison, http://www.cis.upenn.edu/~bcpierce/unison/
CIT 470: Advanced Network and System Administration
Slide #34