Transcript UNIX Operating System - Computing Sciences

ITEC400 Week One
Robert D’Andrea
Agenda
Introduction
Administrative Announcements
Link of the Week
Expected Outcomes
This Week’s Topics
Next Week’s Lab Assignment
Break Out Problems
Upcoming Deadlines
Lab assistance, questions, and chat time
Introduction
Instructor
Professor Bob D’Andrea
Software Engineer
Instructor at Franklin University since
2004
Home phone No. 614.898.0457
Cell No. 616.519.5853
[email protected]
Program Chair of Information Security
Mr. Todd Whitaker
Phone # 614.918.8321
[email protected]
Administrative Announcements
Instructor commitment
Respond daily to student emails.
Post student midterm and final exam status on the
course web page folder after each exam has been
received from the Student Learning Center (SLC).
Office hours
Monday through Friday: 9:00 AM – 6:00 PM
Saturday and Sunday: 12:00 PM – 6:00 PM
Personalized Adobe Connect session can be
scheduled on demand by a student needing
individual assistance.
Administrative Announcements
Course Web Site
Read the Class Communication section?
Attendance
Email and/or call me if you are unable to attend class.
Otherwise, you will be marked “Not Present” and you will
receive an email notice from Franklin University.
cs.franklin.edu server
Everyone will need a login to access the cs.franklin.edu
machine. Putty is the prescribed open source interface for
this class. You will use putty to access the cs.franklin.edu
machine. All student lab assignments will be tested under
your /export/home/$USER/itec400/homework directory.
Administrative Announcements
Recorded Adobe Connect Sessions
Each week, a Power Point presentation is recorded under
the following directory:
http://cs.franklin.edu/~dandrear/itec400/Summer_2016_Unix_Presentations
Franklin Live file name format
The naming convention used for the weekly Power Point
files is listed below.
-rwx---r-x 1 dandrear faculty 1274880 Sep 7 11:08 Week_One_1_UNIX_ppt.ppt
-rwx---r-x 1 dandrear faculty 395026 Sep 7 11:07 Week_One_1_UNIX.pptx
The Power Point presentations are available on Monday
mornings for printing and/or review prior to the online
session. The recorded presentation could contain slight
variations from the actual class version.
Administrative Announcements
Email format for completed lab assignments
Email a notification when an assignment is completed
and ready to be graded. The “Subject” line of your email
notification should contain the following information and
format:
<User name> <Section Number> <Lab Assig 1-3>
Example: dandrear V1WW Lab Assign. 2-1
Email format for questions
<User name> <Section Number> <Question(s)>
Example: dandrear V1WW Question(s)
Scripts and text file suffix
All scripts and text files shall end with the proper suffix
(e.g. ,sh, .pl, .txt and .cgi).
Homework assistance
Do not solicit help from the Internet for lab assignments.
If caught soliciting assistance on the Internet, Franklin
University will take action against you.
Administrative Announcements
Drop Box
All lab assignments must be submitted to
the Drop Box. The Drop Box is located under your
myFranklin course web page “Course Communication”
category.
Student drop box documentation link is listed below
http://online.franklin.edu/forms/StudentDropBoxManu
al.doc
/export/home/<user name>/itec400/homework
The homework directory is where your scripts
and text files will reside for grading.
Administrative Announcements
Turnitin.com Assignments
The Light Directory Access Protocol (LDAP)
and Public Domain/Open Source lab assignments must
be submitted to Turnitin.com. Your report will be
verified by Turnitin.com for author originality. If
Turnitin.com indicates that your percentage of
originality is 45 percent or greater, I will not grade the
paper. I will recommend that you seek assistance from
the Student Learning Center (SLC).
See the Announcement folder for the turnitin.com
login information.
Administrative Announcements
VMware software
The Knoppix software can also be downloaded
from the course web page or from the following link:
http://cs.franklin.edu/~blackbue/Knoppix-VM-setup.zip
Creating scripts using Knoppix software
ftp from your Knoppix software to cs.franklin.edu
sftp://[email protected]/home/dandrear
Link of the week
Open Source Software
◦ http://en.wikipedia.org/wiki/Open_source
◦ http://freshmeat.net
◦ http://sourceforge.net
Link of the week
Open Source Software
Definition of open source software.
 Distributed in source code format
 Developed in a public, collaborative manner
 Software is free of charge
 Allows anyone to apply modifications to the
software.
 Who benefits from open source products?
Course expected outcome
Upon successful completion of this course, students
will be able to:
Create non-trivial shell scripts.
Perform appropriate UNIX System Administration
tasks.
Compose non-trivial scripts using Perl programming
language.
Distinguish the roles of Linux and Open Source
software.
Incorporate the make utility appropriately within
programs.
Create an open source presentation of your choice.
Create a CGI script
UNIX Operating System
UNIX comes in a variety of constantly changing
flavors (SUNOS, HP-UX, BSD and Solaris, just to
name a few). Each of these UNIX types will have small
variations from all of the others. This may seem a bit
discouraging at first, but in reality each version of
UNIX has more in common with all of the other
software versions than differences. For instance, the
“ls” command will list the current directory in any
UNIX environment. The changes or semantics local to
any particular brand of UNIX should be explained in
the manual pages that come with that particular version
of software.
UNIX Operating System
The purpose of this explanation is not to explore
the differences between UNIX flavors, but rather to
assure you that their base lines are all equivalent.
UNIX Operating System
The UNIX operating system comes with many
commands that allows the user to interact with their
computer. UNIX commands are simply programs that
are executed when called on. The usual place for the
storage of these commands is the /usr/bin directory. The
commands available on a particular machine may vary.
There is a set number of standard commands that come
with a UNIX system, but there is no limit to the number
of commands that can be made available on a system.
UNIX Operating System
UNIX, Linux, and HP-UX are all operating
systems. The design of an operating system is to
coordinate and apply the various parts of the computer
-- the processor, the on-board memory, the disk drives,
keyboard, video monitors, and mouse to perform useful
tasks. The operating system is the central most software
program in the machine. It is the mechanism that
connects all the internal and external components with
administers, programmers, and system users.
UNIX Operating System
The UNIX operating system gives a computer
certain recognizable characteristics. However, it would
be difficult to distinguish between two different
computers, if they were loaded with the same operating
system. If you took two identical computers and
installed different operating systems on each of them,
their differences would be apparent to the observer.
UNIX Operating System
UNIX was created in the late 1960s, the initial
effort was to provide a multiuser, multitasking system
for programmers to use. The philosophy behind this
design was to keep it simple, yet powerful with utilities
and write portable code which accessed a network.
The UNIX operating system is made up of three
components: The kernel, standard utility programs, and
system configuration files.
UNIX Operating System
Kernel
The kernel is a core computer program
component of the UNIX operating system. Basically,
the kernel is a software program that is loaded into
memory when the machine is turned on. The kernel is a
memory resident control program. It controls the
allocation of hardware resources from the time the
machine is turned on until the system is shutdown. The
kernel knows what hardware resources are available
(like the processor(s), the on-board memory, the disk
drives, network interfaces, etc.), and it has the
necessary programs to talk to all the devices connected
to it. In summary, the kernel is a fundamental part of a
modern computer's operating system.
UNIX Operating System
Kernel
When a computer program makes requests of the
kernel, the request is called a system call. Various
kernel designs differ in how they manage system calls
and resources. For example, a monolithic
kernel executes all the operating system instructions in
the same address space in order to improve the
performance of the system. A microkernel runs most of
the operating system's requests in the background
in user space, to make the operating system
more modular and, therefore, easier to maintain.
UNIX Operating System
General kernel functions performed are:
• Managing the machine's memory and allocating
it to each process.
• Scheduling the work done by the CPU so that the
work of each user is carried out as efficiently as
is possible.
• Accomplishing the transfer of data from one part
of the machine to another
• Interpreting and executing instructions from the
shell
• Enforcing file access permissions
UNIX Operating System
UNIX Operating System
Standard utility programs
The standard utility programs include utilities
like diff, which compares the content of two files, and
other more complex utilities that communicate with the
operating system.
UNIX Operating System
System configuration files
The system configuration files are read by the
kernel, and many of the standard utilities to configure the
initial settings for some computer programs. Configuration
files are used for user applications, server processes, and
operating system settings. The files are normally written in
ASCII or UTF-8, depending on the operating system. They
are often referred to as simple databases.
UNIX provides several hundred utility programs,
often referred to as commands.
•Editing
•File maintenance
•Printing
•Sorting
•Programming support
•Online information
UNIX Operating System
User access to a system
Logging into a UNIX system requires two pieces
of information: a user name, and a password. After a
user name and password have been authenticated and
accepted, a UNIX system shell is created for them to
use. The shell’s prompt is usually displayed at the
cursor’s position on your screen. In order to get work
accomplished, you enter commands at this prompt. A
shell program is a command interpreter; it takes each
command or combination of commands and passes
them to the kernel to act on. It then displays the output
from this operation on the screen. There are usually
many shells available on a UNIX system, each with its
own unique set of capabilities.
UNIX Operating System
The shells found on the cs.franklin.edu (Einstein)
machine are:
/bin/sh
/bin/bash
/bin/ksh
/bin/tcsh
/bin/csh
The shell is probably the user’s most important
program on a UNIX system. The shell is the interface
between you and the UNIX operating system, and the
man-in-the-middle, the kernel.
UNIX Operating System
Shell scripts are plain-text files, so you must
create them using a text editor. A shell script begins
with a line at the top of the file that identifies the
preferred shell to be executed, as follows:
#!/bin/sh
#!/bin/bash
#!/bin/ksh
The first two characters are a special syntax that
tells the UNIX-like kernel that this is a script and to use
the rest of the line as a pathname to the program that’s
to interpret the script. This line is sometimes referred to
as the shebang, hash bang, or pound bang line.
UNIX Operating System
On most systems, /bin.sh is a symbolic link that
points /bin/sh.
UNIX Operating System
What is a shell?
Shell programs are designed to fit a specific
concept. They accept a command, interpret the
command, execute the command, and then possibly
wait for another command. The shell displays the
“prompt” to notify the user that it is ready to accept
another command.
UNIX Operating System
Putty Interface
login as: dandrear
Access denied
[email protected]'s password:
Last login: Sun Sep 6 20:31:02 2015 from 10.0.6.166
Red Hat Enterprise Linux Server release 6.0 (Santiago)
************
This server is subject to Franklin University Use Policy. You may review
the Use Policy with the command "UsePolicy".
************
To be informed of position postings or internship opportunities, subscribe
to the discuss listserv associated with your major (CS, ITEC, MIS, WEBD,
or MSCS). You can do that from http://listserv.franklin.edu.
If you run into difficulty, contact Jane Sieberth, [email protected].
***************
/export/home/dandrear>
UNIX Operating System
When you complete the coding of your script,
you will want to change the permissions to make it
executable. The chmod command should be used as
follows:
chmod 705 printnum.sh
After making the script executable, execute your
script by typing ./ to tell UNIX-like operating system to
run the script in the current directory, rather than search
the PATH variable. If you fail to find the script, it can
always be executed by running the shell program
followed by the script name, as bash printnum.sh
UNIX Operating System
The shell program recognizes a limited set of
commands, and must be given commands in a way that
it understands. Each shell command consists of a
command name, options, and arguments that are
separated by white space. Similarly to writing a
sentence where it must contain a subject, a verb, and
white space to be a valid. The basic shell command line
is:
command name [-options] [arguments]
UNIX Operating System
The command name is the name of the program
you want the shell to execute. The command option
usually is prefixed with a hyphen, which allows you to
alter the behavior of the command. The argument
position is the name of directories, files, or programs
that the command needs to access.
The shell program is executed by the UNIX
kernel for you. A UNIX program is referred to as a
process while the kernel is executing it. The kernel can
execute the same shell program or any other program
simultaneously for many users on a UNIX system, and
each executing copy of the program is an individual
process.
UNIX Operating System
Many basic shell commands are small
subroutines built into the shell program. The commands
that are not built into the shell require the kernel to start
another process to execute them.
When a non-built-in shell command is executed,
the shell requests the kernel to create a new sub-process
called a process, to perform the command. The child
process lives long enough to execute the command. The
shell waits until for the child process completes before
it accepts the next command.
UNIX Operating System
A shell is an entity that takes input from the user
and deals with the computer rather than have the user
deal directly with the computer. If the user had to deal
directly with the computer they would not get much
done as the computer only understands strings of 1's
and 0's. While this is a bit of a misrepresentation of
what the shell actually does, it provides a rough idea
that should cause the reader to be grateful that there is
such a thing as a shell to interface with.
UNIX Operating System
A good way to visualize a shell program
When a person drives a car, they don’t have
control of every detail that goes into making the engine
run or the electronic system or the braking of the
vehicle. All the user (or driver in this example) needs to
know is that “D” means drive and that pressing or
releasing the accelerator pedal will make the car go
faster or slower. The dashboard would also be
considered part of the shell, since pertinent information
relating to the user's involvement in operating the car is
displayed there. Pretty simple, huh.
UNIX Operating System
In fact, any part of the car that the user has
control of while operating the car would be considered
part of the shell. The idea of what a shell is should be
clearer now. It is a program that allows the user to use
the computer without him or her dealing directly with
it. It is in a sense a protective layer that prevents the
user and computer from coming into contact with one
another.
UNIX Operating System
What is parsing?
To parse a command line means to look at each
part of the command line and be able to extract
information into an intelligent format, so that it can be
executed by the computer. Since there are variations in
how different shells parse a command line, it can be
assumed that the shell in question is generic. When a
user enters a command line at the prompt, the shell
begins by analyzing the command line. The shell will
break the command line down into small indivisible
pieces called tokens (sometimes they are referred to as
atomic).
UNIX Operating System
What is parsing?
Each token is then analyzed in terms of its relationship
with the other tokens. This is similar to the way humans exam a
sentence. If a noun is present, but no verb, the sentence is
deemed incomplete. The shell behaves in much the same
manner. It doesn't only check for missing bits, it also makes sure
that what is there is in correct order. The shell may have to
examine a command line more than once to collect all of the
tokens. Each examination is called a pass. The reason for
multiple passes is that command lines can be quite complicated,
there can be all kinds of substitutions and considerations made
before a solid conclusion can be made about the command line
data.
UNIX Operating System
What is parsing?
On each pass the shell will make a required
substitution and then collect the available tokens. Since
the substitutions can be nested (substitutions containing
substitutions), the shell may require several passes to
collect all of the tokens. As stated above, if at this point
in the process the shell determines that the grammar of
the command line is incorrect, an error is displayed to
the user, or the command is executed. While the actual
order in which the tokens are gathered is interesting, it
is beyond the scope of this course. Where required
(such as aliases) the order of some of the parsing
procedure will be presented.
UNIX Operating System
Shell Variables
One way a shell lets you customize your working
environment is by using shell variables. A shell variable
is an item, known by a name, that represents a value of
some type. As the term “variable” implies, the value of
a shell variable can be changed.
There are two types of shell variables. First,
there are variables that act as off/on switches. Second,
there are variables that store a particular value as a
string of characters.
UNIX Operating System
Shell Variables
USER or USERNAME contains your current user
name.
SHELL holds the path to the current command shell.
PWD is the present working directory. This
environment variable is maintained by the system.
HOSTNAME is the current TCP/IP hostname of the
computer.
UNIX Operating System
Shell Variables
HOME variable points to your home directory.
MAIL holds the location of the user’s mail spool.
LD-LIBRARY_PATH is a variable to indicate
directories in which library files may be found.
TERM holds the name of the current terminal type.
EDITOR holds the text editor name you use.
UNIX Operating System
Shell Variables
PATH sets the path for a session, which is a
colon-delimited list of directories in which UNIX-like
systems search for executable programs when you type
a program name.
PATH=$HOME/bin:/opt/ant/bin:/opt/Lang/jdk1.3.0/j2s
dk1_3_0/bin:/usr/local/bin:/bin:/usr/bin:/opt/mysql/bin:
/opt/WWW:/opt/Mail:/opt/Lang/Perl/lib:/usr/local/lib:/u
sr/local/ssl/lib:/usr/local/pvm3/include:/usr/ucb:/opt/sf
w/bin:$HOME:.
UNIX Operating System
Why is AWK so important?
It is an excellent filter and report writer. Many
UNIX utilities generate rows and columns of
information. AWK is an excellent tool for processing
these rows and columns, and is easier to use AWK than
most conventional programming languages. It can be
considered to be a pseudo-C interpreter, as it
understands the same arithmetic operators as C. AWK
also has string manipulation functions, so it can search
for particular string(s) and modify the output. AWK
also has associative arrays, which are incredibly useful,
and is a feature most computing languages lack.
Associative arrays can make a complex problem a
trivial exercise.
UNIX Operating System
AWK - the original from AT&T
NAWK - A newer, improved version from AT&T
GAWK - The Free Software foundation's version
I suggest you either use NAWK, or GAWK, or
convert your AWK script into PERL using the "a2p"
conversion program which comes with PERL. PERL is
a marvelous language, and I use it all the time, but I do
not plan to cover PERL in this class session.
UNIX Operating System
Shell Set Command
Debugging part of a script:
set -x # activate debugging
# your commands go here...
set +x # stop debugging
UNIX Operating System
set -A is Korn Shell (ksh) specific (not available in bash
or POSIX SH) and it initializes an array with the
specified value(s).
$ set -A COLORS "red" "green" "blue"
$ print ${COLORS[0]}
red
$ print ${COLORS[1]}
green
$ print ${COLORS[2]}
blue
UNIX Operating System
Shell syntax
$# - Number of positional parameters (count)
$! - Background PID
$? - Return value
$$ - Process PID
$ - Provides the content of a variable ($NUMBER)
$0, $1, $2, $3 …$9 - This syntax represents the
positional parameters on the command line.
./printnum.sh 4
$0 = ./printnum.sh
$1 = 4
$# = 1
UNIX Operating System
Shell syntax
./printnum.sh 3 7 10
$0 = ./printnum.sh
$1 = 3
$2 = 7
$3 = 10
$# = 3
./maxlines.sh /bin
$0 = ./maxlines.sh
$1 = /bin
$# = 1
UNIX Operating System
Shell syntax
A variable is a place holder to the script known
as a parameter. They are represented by a dollar sign
($) followed by a number from 0-9. The $0 variable
represents the name of the script, $1 is the first
parameter to the script, $2 is the second parameter, and
so on.
exit 0 – The return values is a number from 0 to 255. A
value of zero (0) indicates a normal exit.
exit 1 - Indicates a failure occurred.
UNIX Operating System
“ “ - Double quotes. Removes special meaning of all
enclosed characters, except $, `, “, and \.
Example: print “The price is $Price.\n”;
(interpolation)
‘
’ - Literal quotes. Removes the special meaning
of all enclosed characters. A single quote cannot appear
within single quotes because a single quote denotes the
end of the string.
` ` - Single Back-Tic quotes. Used for command
substitution.
Example: echo The date is `date` (interpolation)
LINES=`wc -l $ENTRY | cut -c 1-7`
LISTING=`ls -l | cut -f 9`
UNIX Operating System
Example: ./maxlines.sh /bin
$0 = ./maxlines.sh
$1 = /bin
if [ -d "$1" ]
then
action statement
fi
#Print the new line counts
wc –l - Word count with –l (line option).
~ - Tilde (~ means /home/dandrear)
UNIX Operating System
while [ condition]
do
action statement(s)
done
if [ condition ]
then
action statement(s)
fi
UNIX Operating System
for name in *
do
action statements
done
Example:
while [ "$1" != "" ]
do
# Store the value zero (0) in the variable size.
$size = 0
done
UNIX Operating System
Example: ./printnum.sh 2 8
$0= ./printnum.sh
if [ $# –ne 1 ]
then
echo “Please enter a command line
argument”
fi
UNIX Operating System
logfile="/var/adm/messages"
foreach mon in Sun Mon Tue Wed Thu Fri Sat
do
grep $mon $logfile > $logfile.$mon
done
Hands-on-information
Review Lab Assignment 2-1 Simple Shell scripting.
Lab assignments should be recorded on cs.franklin.edu
(cs.franklin.edu) machine in your
“/export/home/$USER/itec400/homework” directory.
Demonstrate how to create a file using the “vi”
editor.
Execute printnum.sh and maxlines.sh scripts on the
cs.franklin.edu machine.
Lab Assignment 2-1 will be complemented with
script logic. Script logic will be utilized to jump start the
lab assignment. It contains script logic, 70% percent of the
needed coding, and helpful hints to assist your
programming skills. In addition to script logic, you will
receive a Shell and Perl Commands Quick Reference
document in email.
Hands-on-information
Set up course directories (case sensitive)
/export/home/dandrear
mkdir itec400
chmod 705 itec400
cd itec400
mkdir homework
chmod 705 homework
cd homework
Break Out Problems
Program statement (#!/bin/ksh)
less
ps –ef | wc –l
who | awk ‘{print $1}’ | sort –u | wc –l
ps –ef | awk ‘{print $1}’ |sort –u | wc –l
find / ex
ps –ef | awk ‘{print $9, $1}’
Shell language syntax: $#
Shell language command: for name in *
Shell language command:
NUMBER=$(($NUMBER - 1))
Shell variable: PATH
Upcoming Deadlines
Lab Assignment 1-1, Obtain a Proctor for Exams, due
May 8, 2016.
Lab Assignment 1-2, Install VMware Workstation and
Knoppix Virtual Machine, due May 1, 2016.
Lab Assignment 1-3, Introduction to Linux, due May 1,
2016.
Lab Assignment 2-1, Simple Shell Scripting, due May
15, 2016
Lab Assignment 3-1, Advanced Scripting, due May 22,
2016
.
Read Chapters 1 and 2 in your text book, Essential
System Administration.
Read Module One listed under the course Web site
UNIX Operating System
Questions?
Comments?
Concerns?
After each Adobe Connect session, I will remain online
to provide assistance or answer questions.
Lab assistance, questions, and chat time