Transcript 2 Days Unix & Shell Scripting

UNIX 1.1 UCF
- Course Content
© 2008 Wipro Ltd - Confidential
UNIX Operating System
© 2008 Wipro Ltd - Confidential
Operating System (OS)
• OS is a system software
• OS provides an environment for execution of programs
• OS acts as an interface between the user and the hardware
of the computer system.
• Kernel is the Core of the OS
• Unix kernel program is a combination of 2 programs
genunix (generic unix) and unix.
© 2008 Wipro Ltd - Confidential
Features of UNIX
•
•
•
•
•
Multi-user, multitasking, timesharing
Portability
Modularity
File structure
Security
© 2008 Wipro Ltd - Confidential
Layered Architecture
KERNEL
HARDWARE
SHELL
© 2008 Wipro Ltd - Confidential
Unix File System - Directories
Directory
Description
/
Primary hierarchy root and root directory of the entire file system hierarchy.
/bin/
Essential command binaries that need to be available in single user mode ; for all
users, e.g., cat, ls, cp.
/etc/
Host-specific system-wide configuration files
/home/
Users‘ home directories , containing saved files, personal settings, etc.; normally a
separate partition.
/opt/
Optional application software packages .
/sbin/
Essential system binaries, e.g., init, ip, mount.
/tmp/
Temporary files
/usr/
Secondary hierarchy for read-only user data; contains the majority of (multi-)user utilities
and applications.
© 2008 Wipro Ltd - Confidential
UNIX Commands
© 2008 Wipro Ltd - Confidential
Login and authentication
login
access computer; start interactive session
logout
disconnect terminal session
passwd
change local login password; you must set a strong password that
is not easily guessed
sudo
allows users to run programs with the security privileges of
another user (normally the superuser, or root)
© 2008 Wipro Ltd - Confidential
Information
date
show date and time
history
list of previously executed commands
man
show online documentation by program name
who
who is on the system and what they are doing
whoami
who is logged onto this terminal
top
show system stats and top CPU using processes
uptime
show one line summary of system status
© 2008 Wipro Ltd - Confidential
File management
cat
combine files
cp
copy files
ls
list files in a directory and their attributes
mv
change file name or directory location
rm
remove files
ln
create another link (name) to a file
chmod
set file permissions
chown
Change ownership of file/directory
gzip, gunzip
compress/decompress a file
find
find files that match specific criteria
© 2008 Wipro Ltd - Confidential
Display contents of files
cat
copy files to display device
more
show text file on display terminal with paging control
head
show first few lines of a file(s)
tail
show last few lines of a file; or reverse line order
vi
full-featured screen editor for modifying text files
grep
display lines that match a pattern
diff
compare two files and show differences
file
examine file(s) and guess type: text, data, program, etc.
wc
count characters, words, and lines in a file
© 2008 Wipro Ltd - Confidential
Directories
cd
change to new directory
mkdir
create new directory
rmdir
remove empty directory (you must remove files first)
mv
change name of directory
pwd
show current directory
© 2008 Wipro Ltd - Confidential
Disks
df
summarize free space on disk filesystems
du
show disk space used by files or directories
© 2008 Wipro Ltd - Confidential
Special character handling
*
match any characters in a file name
~user
shorthand for home directory of user
$name
substitute value of variable name
\
turn off special meaning of character that follows
'
in pairs, quote string with special chars, except !
"
in pairs, quote string with special chars, except !, $
`
in pairs, substitute output from enclosed command
© 2008 Wipro Ltd - Confidential
Controlling program execution
&
run job in background
^c
kill job in foreground
^z
suspend job in foreground
fg
restart suspended job in foreground
bg
run suspended job in background
;
delimit commands on same line
ps
show process information
kill
kill background job or previous process
© 2008 Wipro Ltd - Confidential
Controlling program input/output
tee
copy input to both file and next program in pipe
script
make file record of all terminal activity
Printing (BSD based)
lpr
send file to print queue
lpq
examine status of files in print queue
lprm
remove a file from print queue
© 2008 Wipro Ltd - Confidential
Networking/communications
ssh
remote login/command execution; encrypted
scp
remote non-interactive file copy; encrypted
sftp
remote interactive file copy; encrypted
telnet
remote network login; plain text password - not
recommended
ftp
network file transfer program; plain text password - not
recommended
© 2008 Wipro Ltd - Confidential
VI Editor
• Two modes of the vi editor
– Command mode: Can perform commands in the vi editor(default
mode)
– Editing (Insert) mode: Can type information in the vi editor
• Why learn the vi editor?
– The vi editor is the most frequently used editor in all flavors of Linux
and UNIX
– It is also useful in case the flavor of Linux/UNIX does not have a
graphical interface
© 2008 Wipro Ltd - Confidential
Some Simple VI Commands
j
move cursor down one line
k
move cursor up one line
h
move cursor left one character
l
move cursor right one character
u
UNDO WHATEVER YOU JUST DID; a simple toggle
i
insert text before cursor, until <Esc> hit
a
append text after cursor, until <Esc> hit
o
open and put text in a new line below current line, until <Esc> hit
O
open and put text in a new line above current line, until <Esc> h
:q
Quit a file ( use : q! for forceful quit without saving)
:w
Save a file
© 2008 Wipro Ltd - Confidential
File Access Permissions
• Refers to the permissions associated with a file with respect to the
following
• Permission Levels
•
User (owner) (u)
•
Group (wheel, staff, daemon, etc.) (g)
•
World (guest, anonymous and all other users) (o)
• Permission Settings
•
Read (r)
•
Write (w)
•
Execute (x)
© 2008 Wipro Ltd - Confidential
Changing Permissions - chmod
• chmod u+x file_name
Syntax:
chmod <category> <operation> <permission> <filename(s)>
or
chmod <octal number> filename
Octal Number
4 - for read
2 - for write
1 - for execution
$ chmod 744 xyz
this sets read, write and execute permissions for owner,
read permission for group and others
© 2008 Wipro Ltd - Confidential
Directory Creation/Removal
Command Syntax
mkdir [OPTION] DIRECTORY
rmdir [OPTION] DIRECTORY
$ mkdir(rmdir) <path>/<directory>
$ mkdir(rmdir) –p <directory1>/<directory2>/<directory3>
Example:
$ mkdir project1
This creates a directory project1 under current directory
$ mkdir –p project2/project1
This creates 2 level of directories
$rmdir –p dir1/dir2/dir3
Remove the directory recursively
© 2008 Wipro Ltd - Confidential
Command - cd
cd command is used to change the directory
• cd
- take to the home directory
• cd ..
- takes to the parent directory
• cd /
- takes to the root directory
• cd ~user - takes to the user home directory.
© 2008 Wipro Ltd - Confidential
File-Related Commands
File Operation
Command
Copying a file
cp
Moving a file
mv
Removing a file
rm
Displaying a file and concatenating files
cat
Command - cp
Used to copy files across directories
Syntax
cp <source file> <new file name>
Example
cp file1 file2
© 2008 Wipro Ltd - Confidential
Command - cp
Options to cp
• -p
• Copies the file and preserves the following attributes
•
•
•
•
owner id
group id
permissions
last modification time
-r
• recursive copy; copy subdirectories under the
directory if any
-i
• interactive; prompts for confirmation before
overwriting the target file, if it already exists
© 2008 Wipro Ltd - Confidential
Command - mv
Used to move a file, or rename a file
Preserves the following details
• owner id
• group id
• permissions
• Last modification time
-f
suppresses all prompting (forces overwriting of
target)
© 2008 Wipro Ltd - Confidential
Command - rm
Used to remove a file
• Syntax : rm file(s)
-f
suppresses all prompting
-i
prompts before deleting destination file
-r will recursively remove the file from a directory (can be used to
delete a directory along with the content )
Caution: Use “i” option along with “r” to get notified on deletion
© 2008 Wipro Ltd - Confidential
Command – chown & chgrp
$ ls –l
-rwxr-xr-x
-rwxr-xr-x
1 user1 training
3 user1 faculty
12373 Dec 15 14:45 a.out
4096 Dec 24 11:56 awkpro
$chown user2 a.out
$ls –l
-rwxr-xr-x
-rwxr-xr-x
1 user2 training
3 user1 faculty
12373 Dec 15 14:45 a.out
4096 Dec 24 11:56 awkpro
$ chgrp training awkpro
$ls –l
-rwxr-xr-x
-rwxr-xr-x
1 user2 training
3 user1 training
12373 Dec 15 14:45 a.out
4096 Dec 24 11:56 awkpro
© 2008 Wipro Ltd - Confidential
Command - ln
Linking files
• Hard Link (in the same filesystem)
• $ ln /usr/bin/clear /usr/bin/cls
• Hard link uses the same inode number
• Soft Link (in different filesystems also used to link
directories)
• $ ln –s /usr/bin/clear /home/user1/cls
© 2008 Wipro Ltd - Confidential
Standard Files
• Standard Input file
• Keyboard, file descriptor is 0
• Standard Output file
• Monitor, file descriptor is 1
• Standard Error file
• Monitor, file descriptor is 2
© 2008 Wipro Ltd - Confidential
I/O Redirection
< file
redirect standard input from file
> file
redirect standard output to file
2> file
redirect standard error to file
2>&1
merge standard error with standard output
$ cat > abc
$ ls –l > outfile
$ cat xyz abc > outfile 2> errfile
$ cat xyz abc > outfile 2>&1
© 2008 Wipro Ltd - Confidential
Filters
• Filters are programs that takes its input from the
standard input file, process it, and sends it to the
standard output file.
• Commonly used filter commands
• sort
• grep
• cut
• head
• tail
• paste
© 2008 Wipro Ltd - Confidential
sort
Sorts the contents of the given file based on the first char of each line.
-n
numeric sort (comparison made
according to strings numeric value)
-r
reverse sort
Example:
$ du /home | sort –nr
list files from /home in decreasing order of their file size
grep (global regular expression print)
• grep -Global Regular Expression Printer is used for searching regular
expressions
•
Syntax
• grep <options> <pattern> <filename(s)>
© 2008 Wipro Ltd - Confidential
grep options
-c
-n
-v
-i
displays count of line where the pattern occurs
displays line numbers along with the lines
displays all lines except lines matching pattern
Ignores case for matching
Patterns:
*
matches 0 or more characters
^pqr
Matches pqr at the beginning of the line
pqr$
Matches pqr at the end of the line
“.”
Matches any one character
© 2008 Wipro Ltd - Confidential
Filter Command – head / tail
head:
Displays the first n lines of the file
$ head -3 file1
tail:
Displays the last n lines of a file
$ tail -3 file1
Can also specify the line number from which the data has to be displayed till
the end of file
$ tail +5 file1
© 2008 Wipro Ltd - Confidential
Command Piping
• Allows the output (only the standard output) of a
command to be sent as input to another command.
• Multiple pipes may appear in one command line.
Example:
$ cat file1 | head | wc -l
© 2008 Wipro Ltd - Confidential
ftp - commands
• File Transfer Protocol (FTP), a standard Internet protocol, is the
simplest way to exchange files between computers
• FTP service runs on Port 21
• Ftp program supports the following commands
 get
receive file from host
 mget
receive multiple files from host
 put
send file to host
 mput
send multiple files from host
© 2008 Wipro Ltd - Confidential
ftp - commands
Ftp program supports the following commands as well
 ls
list directory of host
 cd
change directory on the host
 lcd
change directory on the local machine
To set transfer format
 asc / ascii
 bin/ binary
set to ascii mode
set to binary mode
Progress indication of transfer (to see the data flow)
 hash command
Quitting ftp session
 bye command
© 2008 Wipro Ltd - Confidential
Tape Archive - tar
• Tar is an archiving utility to store and retrieve files from an
archive, known as tar file.
• Archives and extracts files to and from a single file called a
tar file.
• Though archives are created on a tape, it is common to have
them as disk files as well.
• SYNTAX
tar c|t|x [vf destination] source...
c – create a tar file
t – list the contents of the tar file
x – extract the contents of the tar file.
© 2008 Wipro Ltd - Confidential
Tape Archive - tar
Examples:
• $ tar –cvf my.tar *.dat
Create a new tar file containing all .dat files (assuming a.dat, b.dat and c.dat exist)
• $ tar –xvf my.tar
Extract the contents of the tar file my.tar
• $tar -c file1 file2
write the file1 and file2 files to a new archive on the default tape drive
• $tar -xf my.tar test.file
Extract test.file from the tar file my.tar
• $tar –tvf my.tar
view the contents of the tar file called my.tar
© 2008 Wipro Ltd - Confidential
find
• Lets user to search set of files and directories based on various criteria
Syntax: find [path...] [expression]
[path] where to search
[expression]
• What type of file to search (specified with –type option)
• What action to be applied (–exec, –print, etc.)
• Name of the files (specified as part of –name option, enclosed in “ “)
Examples
• $ find . –name “*.c” -print
lists all files with .c extension from the current dir & its subdirectories
• $ find /etc -name hosts
Will find the file named “hosts” under /etc and print --- /etc/hosts
© 2008 Wipro Ltd - Confidential
find
• $ find /home
lists all files and directories under /home
• $ find . –atime 2
lists files accessed exactly 2 days ago
• $ find . –atime +2
lists files accessed more than 2 days ago
• $ find / –mtime –2
lists files modified less than 2 days ago
© 2008 Wipro Ltd - Confidential
Sed
•
Sed is a stream editor. A stream editor is used to perform basic text
transformations on an input stream (a file or input from a pipeline).
•
But it is sed's ability to filter text in a pipeline which particularly
distinguishes it from other types of editors.
 Example 1: command to change a to A and b to B in a file called ‘test’ is #sed -e 's/a/A/' -e 's/b/B/' test
 Example 2: find and replace the word “star" to “planet" globally in the file
called ‘universe’ is - #sed '/s/star/planet/g' universe
© 2008 Wipro Ltd - Confidential
IOSTAT / NETSTAT / VMSTAT
• The iostat utility displays kernel I/O statistics on terminal, device and cpu
operations.
Example : Display tty statistics every 10 seconds – $ iostat –t 10
• netstat - Display network connections, routing tables, interface statistics,
masquerade connections, and multicast memberships
Example : command to display the Interface statistics - $ netstat –i and
command to display routing table - $ netstat –nr
• vmstat reports information about processes, memory, paging, block IO,
traps, and cpu activity.
Example : To display five summaries at 2-second intervals - $ vmstat 2 5
© 2008 Wipro Ltd - Confidential
Job Scheduling
‘at’ : In Unix the at command is used to schedule other to be executed once, at a particular
time.
More precisely, it reads a series of commands from standard input and collects them into one
"at-job" which is carried out at a later date.
 Example : $ echo "cc -o test test.c" | at 1145
$ at 1145 jan 31
at> cc -o test test.c
at> ^D
•
‘Cron’ : is a time-based job scheduer in Unix. Cron enables users to schedule jobs
(commands or shell scripts) to run periodically at certain times or dates. It is commonly used
to automate system maintenance or administration.
 Example : command t list all cronjobs - $ crontab –l, command to execute a cron job - $
crontab –e <job name> and command to remove a cron job - $ crontab –r <job name >
© 2008 Wipro Ltd - Confidential
Unix Process and PS
•
•
•
•
•
•
Process - a program in execution
When program is executed, a new process is created
The process is alive till the execution of the program is complete
Each process is identified by a number called pid
The ps command is used to display the characteristics of a process
It fetches the pid, tty, time, and the command which has started the process.
 -f
lists the pid of the parent process also.
 -u
lists the processes of a given user
 -a
lists the processes of all the users
 -e
lists all the processes including the system
processes
•
Back Ground process :


Enables the user to do more than one task at a time.
If the command terminates with an ampersand (&), UNIX executes the command in
the background
 nohup
– Lets processes to continue to run even after logout
– The output of the command is sent to nohup.out if not redirected
© 2008 Wipro Ltd - Confidential
Killing a Process, TOP and NICE
•
•
•
•
kill: Kills or terminates a process
kill command send a signal to the process
 The default signal is 15 ( SIGTERM)
kill -9 (SIGKILL)
 Terminates the process abruptly
pkill : command used to kill a process by name
 pkill <command name>
 Pkill -9 <ommand name>
•
‘top’ Comamnd : Provides an ongoing look at processor activity in real time. It displays a
listing of the most CPU-intensive tasks on the system, and can provide an interactive
interface for manipulating processes.
•
‘nice’ Command : The nice command lets you run a command at a priority lower than the
command's normal priority. If you do not specify an Increment value the nice command
defaults to an increment of 10. The priority of a process is often called its nice value.
© 2008 Wipro Ltd - Confidential
© 2008 Wipro Ltd - Confidential