Transcript netscape - Homeostasis

Operating System Stability
and Security through
Process Homeostasis
Anil Somayaji
Dept. of Computer Science
University of New Mexico
[email protected]
April 16, 2001
Summary
"
Most systems are insecure
"
Too much code to rewrite securely
"
Need post-hoc security:
"
–
detect novel attacks & prevent damage
–
want anomaly detection & automated response
pH (process Homeostasis):
–
delays anomalous system calls
–
stops buffer overflows, trojan code, kernel
security flaws
Overview
"
Introduction
"
pH: process Homeostasis
"
–
Rationale
–
Design
–
Results
–
Limitations
–
Portability
Towards a Homeostatic Operating System
Introduction
"
"
Before, only government and financial
institutions cared about computer security.
Security threats are now common:
–
"
1
email viruses, worms, distributed DoS
Existing approaches are insufficient:
–
Code audits are laborious & imperfect.
–
Virus scanners & network IDS need updates.
–
Manual monitoring does not scale.
We need new ways to secure systems.
The Success of Biology
"
"
"
"
Computers need to "survive" in diverse &
hostile environments.
Living systems are good at survival:
–
life is everywhere
–
billions of years of practice
Organisms cannot rely on system
administrators or signature updates!
One survival strategy: homeostasis
Homeostasis
"
Biological online monitoring and response
"
Maintenance of stable (normal) internal state
"
Examples:
–
temperature regulation
–
immune system
"
Detectors of system state (nerves, MHC)
"
Effectors to change state
–
sweat, shivers
–
antibodies, killer T-cells
Process Homeostasis (pH)
"
"
"
Inspired by biological homeostasis & immune
system
Intrusion detection & response system:
–
Linux 2.2 kernel extension
–
monitors system calls to detect anomalous
program behavior
–
responds by delaying system calls
–
minimal administration & no signature updates
Stops novel buffer overflows, trojan code,
kernel security flaws
System Calls
"
"
What is a system call?
–
UNIX interface between kernel and proceses
(running programs)
–
mediates access to disk, network, user I/O, other
processes
Why system calls?
–
conduit for damage to disk, network, processes
–
good for detection: security-relevant events
–
good for response: stop calls, prevent damage
Why Delay System Calls?
"
"
Effective & safe
–
Large delays stop attacks and abuse.
–
Small delays are unnoticeable.
–
Delays can be cancelled, allowing easy recovery
from false positives
Generic response
–
"
works in diverse situations
Understandable
Anomalies are Delayed Exponentially
Many Anomalies
One anomaly
Short Sequences of System Calls
window size 4
open, read, mmap, mmap, open, getrlimit, close
position 3
read
position 2
mmap
open
mmap
mmap
open, read
mmap
open
position 1
mmap
open
read, mmap
open
getrlimit
anomalous trace:
mmap, open, open, getrlimit
lookahead pair anomalies:
open, open
open, *, getrlimit
current
open
read
mmap
getrlimit
close
Training and Testing
sequence from netscape:
..., open, read, mmap
Training and Testing
sequence from netscape:
..., open, read, mmap
parse
lookahead pairs
read, mmap
open, *, mmap
...
Training and Testing
sequence from netscape:
..., open, read, mmap
parse
lookahead pairs
read, mmap
open, *, mmap
...
insert
training array
Training and Testing
sequence from netscape:
..., open, read, mmap
parse
lookahead pairs
read, mmap
open, *, mmap
...
insert
training array
testing array
copy
copy when:
" seen enough calls
" no changes for 1 week
" now "normal"
Normal Heuristics
new lookahead pairs
time
train_count = 25
last_mod_count = 19
normal_count = 6
train_count
normal_count
> 4
frozen for a week
frozen
normal
Training and Testing
sequence from netscape:
..., open, read, mmap
parse
present?
lookahead pairs
read, mmap
open, *, mmap
...
ok
test
abnormal
system call
missing?
testing array
Training and Testing
sequence from netscape:
..., open, read, mmap
parse
present?
lookahead pairs
read, mmap
open, *, mmap
...
insert
training array
ok
test
copy
abnormal
system call
missing?
testing array
copy when:
" seen enough calls
" no changes for 7 days
" now "normal"
Function of the Locality Frame
Many Anomalies
Locality Frame
One anomaly
Anomalous Behavior
"
pH detects
–
"
newly misconfigured programs (inetd)
pH stops
–
buffer overflows (fetchmail, ssh, named)
–
trojan code (su & ssh backdoors)
–
kernel security flaws
"
drop capabilities (privileges) flaw
"
ptrace/execve race
Demos
"
su trojan
"
inetd chargen
The Larger Problem
"
Software is not self-aware.
"
Many events are important for system health:
"
–
failing hardware
–
misconfigured programs, software bugs
–
security violations
–
unwise users
Need for more observant software:
–
online real-time monitoring
–
automated response
Approach
"
"
Want an automatic system
–
minimal administration burden
–
handle unforeseen events
Accept imperfect detection & response:
–
"
"
especially false positives
Anomaly detection & response:
–
true positives: interfere or stop attacks, warn
administrators, minimize damage
–
false positives: mild degradation
Homeostasis