Transcript CENG334 Introduction to Operating Systems

CENG334
Introduction to Operating Systems
Security
Topics:
Erol Sahin
Dept of Computer Eng.
Middle East Technical University
Ankara, TURKEY
URL: http://kovan.ceng.metu.edu.tr/ceng334
13/03/07
Goals of security in computer systems

Data confidentiality


Data integrity


Unauthorized user should not be able to modify and change your data
 Your bank account goes from 1,000,000 YTL -> 0 in a second
System availability


Secret data should remain secret
 Your bank account details
Nobody should be able to disturb the system to make it unusable
 DOS (Denial of Service) attacks
Exclusion of outsiders

Outsiders can take control of other people’s computers converting them into
zombies and use them to coordinate their attacks
 Sending spam or coordinating DOS attacks
Tanenbaum, Modern Operating Systems 3 e, (c) 2008 Prentice-Hall, Inc. All rights reserved. 0-13-6006639
2
Motivations of intruders and adversaries

Casual prying by nontechnical users


Snooping by insiders


Students, systems programmers and other technical personnel consider it a personal
challenge to break the security
Determined attempts to make money


Setting all files readable
Attempts to get info to access bank accounts
Commercial and military espionage

Serious and well-funded attempt to steal programs, trade secrets
3
Cryptography
Kerckhoff’s principle:
The algorithms should be public and the secrecy should reside
exclusively in the keys
4
Secret Key Cryptography

One of the earliest types of cryptography

Both encryption and decryption keys should remain secret


Both the sender and the receiver should have possession of the keys
 The keys should be sent through a secure channel!
Computation required for encryption and decryption are equal and
manageable.
5
Monoaphabetic substitution
Plaintext
A
B
C
D
E
F
G
H
I
J
K
L
M
N
O
P
Q
R
S
T
U
V
W
X
Y
Z
Q
W
E
R
T
Y
U
I
O
P
A
S
D
F
G
H
J
K
L
Z
X
C
V
B
N
M
Cipher text
Encryption key
6
Monoaphabetic substitution
Plaintext
Q
W
E
R
T
Y
U
I
O
P
A
S
D
F
G
H
J
K
L
Z
X
C
V
B
N
M
K
X
V
M
C
N
O
P
H
Q
R
S
Z
Y
I
J
A
D
L
E
G
W
B
U
F
T
Decryption key
7
Monoalphabetic substitution

One of the earliest type of cryptography

There are 26! = 4 X 10^26 possible keys

But statistical properties of natural languages can be used to crack.



In English, e is the most common letter
If in the ciphertext, v is the most common letter, then it is likely to encrypt e.
two-letter combinations can be used (th, in)

Suffers from the same vulnerability
8
Public Key cryptography

Distinct keys are used for encryption and decryption




Encryption key is made public
 Everyone can see it and use it to encrypt their messages to send
Decryption key is kept secret
 Only the receiver of the message can use this key to decrypt the message
For a well-chosen encryption key, it is virtually impossible to discover the
corresponding decryption key.
The source of the symmetry:


How much is: 213434545454545 X 213434545454545?
 Easy
What is the square root of 45554305193388235720661157025?
 difficult
9
One-way functions

y= f(x) such that for x,



For a given x, it is easy to compute f(x)
For a given f(x), it is difficult to compute x
F() can mingle the bits in complex ways
10
Digital signatures

Digital signatures used to sign e-mails or other documents


To detect that they are not modified by the received
A hashing function that computes a value for a given document

MD5
11
Watermarking


Left: three zebras and a tree
Right: three zebras and a tree and the complete text of five
Shakespere plays

Use the 8th LSB of the pixels to code the compressed text
 To the bare eye, it is invisible
12
Protection mechanisms

A computer system contains many “objects”, such as


A domain is a set of (object, rights) pairs


Files, directories, hardware, ..
 A file can be readable but not writable..
Right = read, write, execute …
In UNIX, the domain of a process is defined by

uid and guid (user and group id)
13
setuid

How does the passwd program work


When executed by the user, the process runs in the user’s domain
 Cannot modify the /etc/passwd file!
Solution: passwd program has its setuid bit set, that allows it to run with root access
 Modify /etc/passwd file
14
Authentication

Passwords




Physical objects


Keep a list of user:encrypted_password pairs
 Used to be visible
There is a function that encrypts passwords
 Not even the root can see your passwords
But open to cracking through guessing:
 Get the password file,
 Guess passwords using different combinations
 Find the matches
Smart cards
Biometrics



Hands
Eyes
fingerprints
15
Insider attacks

Logic bombs


Check whether you are still on the payroll
 If so, do nothing, if not delete the filesystem
Trap doors

Add code to the login program such that it bypasses normal check
16
Login spoofing

A user can write a program that looks exactly like the login screen

The victim comes, enters his password


The login sppofing program sends the password and quits
The victim thinks it mistyped, and enters again to the actual login program
17
Exploiting code bugs

Buffer overflow attacks

You know this well ;)
18
Exploiting code bugs

Format string attacks




stems from the use of unfiltered user input as the format string parameter in certain C
functions that perform formatting, such as printf().
A malicious user may use the %s and %x format tokens, among others, to print data
from the stack or possibly other locations in memory.
One may also write arbitrary data to arbitrary locations using the %n format token,
which commands printf() and similar functions to write the number of bytes formatted
to an address stored on the stack.
Format string bugs most commonly appear when a programmer
wishes to print a string containing user supplied data.

The programmer may mistakenly write printf(buffer) instead of printf("%s", buffer).
 The first version interprets buffer as a format string, and parses any formatting
instructions it may contain.
 The second version simply prints a string to the screen, as the programmer
intended.
Working example: http://julianor.tripod.com/bc/tn-usfs.pdf
19
Other code bugs

Return to libc attacks


Pretty much all C programs link with libc
 Libc contains insecure functions such as strcpy() which copies an arbitrary string
from any address to any other address
 Trick strcpy into copying the atackers program, called a shellcode, to the data
segment and have it executed there
 Example: http://www.infosecwriters.com/text_resources/pdf/return-to-libc.pdf
Integer overflow attacks

For instance give large numbers a image size to a program
 Multiplication causes integer overflow
 Results in smaller buffer than needed
 Provides the ground for buffer overflow attack
20
Other code bugs

Code injection attacks

Getting the target progra to execute code without realizing it is doing so.
> cp abc xyx
> cp abc xyz; rm –rf /
21
Other code bugs

Privilege escalation attacks


A daemon that allows user to schedule work to be done periodically.



The attacker tricks the system into giving it more access rights that it is entitled to
 Typically he tricks t into doing something that only root can do
run as root so that it can access files from the user’s directory
Has a directory in which it stores commands that are scheduled to run
 Users cannot write to this directory
The attacker’s program set its working directory to the daemon’s
directory.


It crashes and forces a core dump.
Core dumps are written into the working directory.
 Dumps can be written there since it is made by the system.
22
Malware
Software that does bad things

Trojan horses


Viruses



Hidden in a free program that is providing some useful function
 When called, it can do anything it wants
Is a program that can reproduce itself by attaching its code to another program
Can replicate through other programs
Worms


Replicates like viruses
Can self-replicate
23
Viruses

The lifetime:




Usually written in assembler
Inserted in a program or a file and distributed through filesharing, or downloads
 Free programs, pirated versions of commercial software
Once installe, the infected program is executed.
Companion viruses


Does not actually infect a program
When user types prog, MS-DOS looks for a program called prog.com or prog.exe
 Prog.com does not exist, but prog.exe is a useful executable provided on all the
systems
24
Viruses

Executable viruses


Simplest version: overwrite an existing exeutable.
Makes a readdir, and infects other executables
 Open the file and copy the virus code.
 Checks whether the file is already infected.
25
Viruses

Memory-resident viruses

Remains active in memory at the very top or the bottom of the address space.
 Can even change the memory bitmap to remain unnoticed.
 Typically captures one of the traps or interrupt vectors, such as the system call
trap, and get executed at every system call.
 Can also do lots of spying.
26
Viruses

Boot-sector viruses

Overwrite the MBR such that it executes at every boot, before the booting of the OS.
 No OS, no protection: vulnerable state for the system
 Can also disguise itself as bad sectors.
 At boot time the virus copies itself to RAM, and directs the interrupt vectors to
itself.
27
Viruses

Device driver viruses


Macro viruses


Infect a driver and get loaded automatically by the OS when it gets called.
 This is why device drivers should run as user processes.
Macros are programs that can be attached to data files such as Word or Excel.
Source code viruses

Change the c files such that
 #include <virus.h>
 Run_virus().
28
Spreading of viruses

Typically through shared files

through network



E-mails
Downloads
Browsing
 Sites that provide sexual content
 Sites that provide illegal download (warez’s)
 Sites that provide free programs
 Use checked software
29
Rootkits, and sony rootkit
30