Transcript Lecture 2

CS 3034: Widely Used Programming
Languages
Lecture 2: *Nix Basics
John Hurley
Cal State LA
Basic *NIX
• UNIX was a milestone in the development of
operating systems and of programming in general.
– In most programming contexts, time is measured in
relation to midnight, January 1, 1970, the year UNIX
was released. UNIX is so important in the history of
programming that we treat its inception as the
beginning of time.
– The original reason for creating C was to use it to
develop UNIX.
• Some forms of UNIX are still widely used today
– Mac OSX
– Oracle Solaris
2
Basic *NIX
Linux is fundamentally different from UNIX internally,
but it provides a "UNIX-like" interface for developers
and users.
– This was designed to save early users the trouble
of learning new OS commands and system calls.
– We usually call commands like ls and pwd "Unix
commands" even when we are actually using Linux
– It is usually possible to use the same commands, and
often even the same shell programs, in Linux and Mac
OSX with no changes
3
Linux
• The core of an OS is the "kernel," which does the essential work (more on
this in a later lecture). Linux uses a large, full-featured kernel where other
OSs use small kernels and load other modules as needed.
• Linux is open source, developed by a group of thousands of volunteers.
• Linux has very high market share for web and database servers. It has
small market share for desktop/laptops, mostly among technical people.
•
The kernel and some of the other aspects of Linux are used by many companies
and other groups of developers to create hundreds of different "distributions" of
Linux. The core of the OS is the same for all distros, but included applications and
user interface components differ. Each distro is aimed at users with particular
priorities.
•
Many distros are free and open-source. Some are commercial, and some use
hybrid models in which they provide the distro for free but charge for technical
support or other services.
•
For important software packages, like JDKs and DBMSs, each distro release has an
officially-supported version established at release time. Distro sponsors develop
installation packages tuned for their release.
4
Linux
• C and C++ are not fully platform-independent. Thus, code that works with
a particular compiler in a particular OS might not work with a different
compiler or different OS.
• The purpose of using the Digital Ocean VM in this part of the course is to
make sure you turn in code that compiled correctly with the same
compiler in the same OS release I will use for grading.
• Linux continues the historically close relationship between *nix operating
systems and C and C++
5
More *nix Basics
• You will access your VM using SSH and, optionally, SFTP.
• *nix command line terminals are called shells. The one we get automatically
when we ssh to the VM is an application called bash. There are other shells,
including tcsh, the "improved C shell." Bash and tcsh are broadly similar, but
there are sone syntactic differences. You can easily get and use tcsh for your vm,
but use bash for you work in this course unless you already know some other
shell well.
• The shell can be considered a programming language, and it is possible to use
native shell syntax for programming applications, usually ones useful in
administering and using *nix systems.
*nix commands
You can easily find lists of common Unix shell commands on line, for example at
http://mally.stanford.edu/~sr/computing/basic-unix.html
I assume you are already familiar with the following basic commands from CS
120 and/or cs122. If not, learn them as soon as you get your VM set up:
list files in current directory: ls and its arguments -l (more details) and -a (include hidden files)
navigation: cd and relative paths
delete a file: rm,
move/rename and copy files: mv and cp
create a directory inside the current one or using a relative path: mkdir
find out where you are (present working directory): pwd
find out where an application is installed: which
change access permissions: chmod and its arguments, either in the style chmod 755 or chmod a+r
upload using sftp: put
stop a foreground process: <ctrl> c
Create User Linux
Root has power to access, change, add, and delete data from the system,
change system parameters, install software that runs with full privileges, etc.
You should almost never log in as root because of the risk that a small error can
cause a disaster. You should certainly never let anyone *else* log in as root.
For most of your work, use a regular user account, which you can create when
you log in initially to change the root password. We will soon learn how to
temporarily get root privileges when needed.
adduser <username>
then passwd <username>
for example,
adduser mothra
password mothra
then supply
the password at the prompt
8
Sudo
sudo (superuser do or substitute user do) gives a user temporarily upgraded
privileges.
• can be used to give some users temporary permission to perform
operations otherwise reserved for root; can also be configured in other
ways
• In order to use sudo, a user must be listed in the sudoers file. While logged
in as root or as an existing sudoer, type usermod username -a -G wheel, eg
usermod mothra -a -G wheel
-a -G wheel means "append this username to the list of members of the
group called wheel." You can find various speculations online about why
the group is called wheel.
Sudo
The addition of the user to the group does not take effect until the next time
the user logs in. If the new sudoer is currently logged in, quit from that
account and then log back in.
When the user uses sudo, eg sudo dnf install someapp, s/he will be asked for
the user's (not root's) password.
Without further configuration, sudo gives the user most of the power of root.
Don’t give it to users unless both a) they need it and b) you trust them.
Most hacking attempts are designed to get root access. A malicious actor who
wanted to break into the system would have to guess or find out the user's
username as well as password, and would accomplish comparatively little if
the user turned out not to be a sudoer. However, like root, sudoers should
have passwords that are hard to guess.
Disable Root SSH Logins
• see a list of the last 20 unsuccessful login attempts:
– Sudo lastb | head -n 20
• Your VM will receive hundreds of failed ssh login attempts per day. Some of
these are caused by configuration mistakes (ie, incorrect ip addresses) in
applications on other systems. However, they will usually use the login
name root, which an application trying to access a DB or other server would
not use. This is evidence that they are mostly attempts to guess the
password and get control of the system.
• Here is how to disable root ssh logins:
– Make sure you have a sudoer set up first to do admin work. Test sudo from that
login before you do this!
– sudo vi /etc/ssh/sshd_config
• change PermitRootLogin from yes to no
• sudo systemctl restart sshd
• There are at least two open-source utilities, fail2ban and denyhosts, that
automatically blacklist particular addresses after several failed login
attempts. You can try them out on your own if you'd like.
Package Managers
*nix operating systems typically use package managers to automate software installation. These
tools use repositories containing software packages tuned for the OS (and, in the case of Linux, for
the distro) and usually get and install all dependencies automatically.
Different distros use different package managers. Most of them are broadly similar. Fedora uses
one called dnf.
The syntax for installing packages using dnf is dnf install <package name>, but use sudo. For
example
dnf install java-1.8.0-openjdk
To find the right package name for an application you want to install, use
dnf search substring
For example,
dnf search putty
Fedora intentionally makes it difficult to install non-libre (“free as in free speech”) software. If you
need something proprietary (for example, Chrome or certain codecs), do a web search for
instructions. It is possible to configure dnf to use proprietary repositories. It is also possible to
download and install packages directly
Sudo
If anything the lectures describe as basic *nix functions does not work, try the
package manager (in the case of Fedora, dnf.) If you are logged in as a non-root
user, you will need sudo for this. For example, if you want to try the utility less
but you get an error message telling you it is not available, try
sudo dnf install less
*nix command line editors
*nix command-line text editors include emacs, vi, vim, and others
• Some people have opinions about the relative merits of these.
• Since you won’t have a GUI for your Digital Ocean VM, you will need to use
these at least sometimes.
• I only use them for very simple editing; for anything complex, I use a GUI
editor like Geany, Notepad++, etc. on my desktop and, if the *nix machine is
remote, either sftp the edited file or copy and paste code into vim on the
remote machine. For this kind of use, all the command line editors are
about the same.
• vim claims to be "vi improved." You can install it with
sudo dnf install vim
• Use of these editors is not intuitive
–
–
–
–
–
–
vim myfile creates myfile if necessary, and opens it for editing
i allows you to insert text
<esc> gets you out of the text to a command mode
from the command mode, w saves the file
q quits
to quit without saving your changes, use <esc> q!
More *nix Basics
print a text file all at once in *nix: cat, eg
cat hello.s
use space bar to move forward one screen, q to quit
run an executable file from the current directory in *nix: ./filename, eg
./hello.out
if the file is somewhere other than the current directory, use a relative path
you must have execute permission!
read a text file one screenful at a time: more (or its alternative, less), eg
more hello.s
programs usually end with exit codes of 0 (program exited normally) or 1 (something
went wrong)
More *nix Basics
send output from one command as input for another one: |
get a list of past successful logins: last
get a list of past unsuccessful login attempts: lastb
see the beginning of a text file: head -n numlines
see the end of a text file: tail -n numlines
see the number of words in a file: wc
see the number of lines in a document: wc -l
search a file for a string and show all lines in which the string occurred: grep string
eg man -k last | grep login
Blacklist an ip address so that your VM does not accept logins from it:
sudo iptables -A INPUT -s [ip_address] -j DROP
this helps, but it is very easy for attackers to change ip addresses
grep
• grep searches text for a substring and returns
every line in which the substring occurs
[godzilla@fedora-1gb-sfo1-01 ~]$ cat mary
Mary had
a little lamb
little lamb
little lamb
and also some mint jelly
[godzilla@fedora-1gb-sfo1-01 ~]$ cat mary | grep lamb
a little lamb
little lamb
little lamb
*nix Shell History
• In addition to using the scroll keys to get back to past
commands, the history command is useful for this purpose. For
example, if I recall that the syntax for blacklisting an ip address
contains the word DROP, but I don’t remember the rest:
[godzilla@fedora-1gb-sfo1-01 ~]$ history | grep DROP
7 sudo iptables -I INPUT -s 183.2.202.255 -j DROP
11 sudo iptables -I INPUT -s 183.2.202.173 -j DROP
20 sudo iptables -I INPUT -s 159.226.63.186 -j DROP
622 history | grep DROP
1005 history | grep DROP
the second to last entry is from a previous time I ran history | grep DROP;
the last one is the current command
More *nix Basics
Put the above together with sudo (assuming you are logged in as a sudoer but not
as root):
Get a list of the last 50 unsuccessful login attempts:
sudo lastb | head -n 50
Get the first 50 unsuccessful login attempts
sudo lastb | tail -n 50
See how many unsuccessful login attempts have been made since the system was
installed:
sudo lastb | wc -l
See all of godzilla's successful logins:
sudo last | grep godzilla
See how many times godzilla has logged in:
sudo last | grep godzilla | wc -l
See how many unsuccessful attempts have been made to log in with usernames
that include "root":
sudo lastb | grep root | wc -l
Always do a step-by-step reality check with these commands to make sure you are
not getting garbage due to unanticipated data.
More *nix Basics
Direct output from a utility to a file: >
send info on the last 100 unsuccessful login attempts to a file called failures
sudo lastb | head -n 100 > failures
Use a file as input: <
send info on all the lines from the file failures that include the string 185 (proxy for
unsuccessful attempts from ip addresses that include 185) to a file called fail185
grep 185 < failures > fail185
Download a file using sftp (not ssh) client: get [remote filename] [local path local
filename]
get fail185 C:\Users\Godzilla\Downloads\fail185
More *nix Basics
& means run a program in the background (that
is, don't make the shell user wait while the
program is running). For example,
./myprogram &
This won’t work if there is user I/O!
ps: list running processes
kill [num]: kill a running process
control z: stop (not kill) a foreground process
fg bring a background process to the foreground
More *nix Basics
Consider this Java code:
public class LongLoop{
public static void main(String[] args){
long i;
for(i = 0; i < Long.MAX_VALUE; i+=1){}
System.out.println(i);
}
}
a)
If javac does not work, look up instructions for using dnf to install a JDK
(openJDK or the Oracle JDK).
b) Compile and run the program in the foreground. Hit control c when you
get bored
c) compile and run the program in the background, doing something else at
the command prompt for a while. Then use ps to find the job number (it
will be listed as just "java") and kill the process.
Tarball
tarball is a file compression format often used in
*nix systems. We may not use it in this course,
but it is a basic part of the *nix world.
Work with it using the tar utility. Use tar --help |
less for more info. The syntax you will use to
untar the sample code file in this week's lab will
be
tar -zxvf *code.intro.tgz
Put the tarball in a separate directory first, since
there are several files in the archive.
More *nix Basics
There are several ways to get help with *nix usage, but the output
is often cryptic. Searching the web is often the best way, but
here are some ways built into your OS:
Many *nix utilities offer usage information with the -h or -help flags, eg
last -h
man also contains help:
man utilityname: get the manual entry for the utility
man last
man -k utilityname: get an index of help topics containing the keyword
man -k last
help, eg help last
info, eg info last