Transcript intrusion detection

Log Correlation for Intrusion Detection:
A Proof of Concept
Cristina Abad, Jed Taylor, Cigdem Sengul,
William Yurcik, Yuanyuan Zhou and Ken Rowe
National Center for Supercomputing Applications (NCSA)
University of Illinois at Urbana-Champaign
National Center for Supercomputing Applications
On Intrusion Detection
• IDSs are important to protect networked
systems
• However, intrusion detection is difficult:
– poor performance (accuracy)
– limitations under current techniques (false
positives)
– many attacks, new attacks, changing all the time
– information overload to human users
– scalability (huge data volume, data management)
– many logs, each with different purposes and
National Center for Supercomputing Applications
formatting
Log Correlation
• Correlate data across heterogeneous logs to
increase IDS accuracy and/or reduce false
positives
• Logic:
– attacks leave traces in multiple different logs
– different attacks more easily identified in different logs
– different logging sensors makes gaming sensors
(evasion or squealing) by attackers more difficult
– some attacks are not evident if a single log is analyzed
National Center for Supercomputing Applications
Methodology
• Goal: show how log correlation can
help improve intrusion detection
• Approach:
– Top-down approach (attacks
logs)
• Take known attacks and analyze
their
behavior to infer which logs may
contain attack traces
– Bottom-up approach (logs
attacks)
• Gather relevant information from multiple
logs
to identify specific attack instances
National Center for Supercomputing Applications
An example mapping: Attack to log traces
1) DNS v. query probe to learn if BIND is vulnerable (syslog)
2) System is attacked with 'noop' attack (tcpdump)
3) Users are created to gain root access (telnet log)
4) Gains superuser access (syslog)
5) Tries to use the system for malicious purposes (Snort)
–
–
–
–
–
–
Downloads (ftp) attack toolkit from another system.
Installs backdoor
Covers moves by deleting affected logs
Days later comes back and logs in using the backdoor
Downloads and installs Trinoo client
Attempts to launch Trinoo DDoS
Source: Honeynet Project. “Know your enemy”
National Center for Supercomputing Applications
National Center for Supercomputing Applications
Bottom-up: A case of anomaly detection
• Detect virus Yaha (an e-mail virus)
• Correlate:
– host information (system call sequence)
• APISpy32 with small modifications
– network traffic information
• WinDump
• “Predict the next system call” method
• RIPPER, a data mining tool for rule learning
– Fed RIPPER with normal sequences only
National Center for Supercomputing Applications
Training Data
• 20 hours of users normal and abnormal activity
was logged
• Normal: sending and receiving e-mail
messages
• Abnormal: sending and receiving e-mail
messages including a message infected with
the Yaha virus
• 1.9 million data points were generated
• 20% of the captured data was saved for testing
(and not used for training)
National Center for Supercomputing Applications
Generating Rules
• RIPPER expects a list of system calls followed
by the value to predict
– A sliding window was used to scan the normal
traces and create a list of unique sequences of
system calls
• Heuristic: Number of connections per past 10
seconds was included as a data point
• RIPPER generated the rules based on the
training data, to predict
– Next system call
– Number of connections on the next 10 seconds
National Center for Supercomputing Applications
Applying Rules
• A sliding window was passed over the list of
predicted values and actual values
– If predicted value differs from actual value, penalty
value is set to confidence level of broken rule
– Penalty values from sliding window are added. If
greater than threshold, region is “abnormal”
• Number of abnormal regions is added up and
divided by total number of regions. This value
represents how abnormal a trace is
• Same process (learning and applying rules
was used for the number of connections)
National Center for Supercomputing Applications
Correlating system calls and traffic
• Penalty value of a region (when predicting
traffic information) was used to increase or
decrease “abnormality” threshold
• System calls were processed using adjusted
threshold for each region
National Center for Supercomputing Applications
Results
A normal behavior trace
An abnormal behavior trace
National Center for Supercomputing Applications
About the NCASSR
National Center for Advanced Secure Systems
Research
• Formed in August 2003
• Projects:
– Security Incident Fusion Tool (SIFT)
– Multicast Survivability and Security
– Cluster security
– PKI and CKM Scalability Study
– Mobile Sensor Authentification
– Mining Alarming Incidents in Data Streams (MAIDS)
– Malicious Code Reverse Engineering and Analysis
– Hardware Acceleration for Information Security
www.ncassr.org
National Center for Supercomputing Applications
Visualization for Intrusion Detection
• Developed two tools:
– NvisionIP
National Center for Supercomputing Applications
Visualization for Intrusion Detection
• Developed two tools:
– VisConnect
National Center for Supercomputing Applications
Future Work
National Center for Supercomputing Applications
Questions?
http://www.ncsa.uiuc.edu/~cabad
http://www.ncassr.org
National Center for Supercomputing Applications
RIPPER
• RIPPER: Repeated Incremental Pruning to
Produce Error Reduction
• William Cohen, 1995
• Obtains error rates lower than C4.5 rules,
scales nearly linearly with number of training
examples, and can efficiently process noisy
datasets containing hundreds of thousands of
examples
• Improvement of IREP (Incremental reduced
error pruning)
National Center for Supercomputing Applications
Example output of RIPPER
• 38:- p3 = 40, p4 = 4
– If p3 is lstat and p4 is write, then p5 is stat
• ...
• 5:- true
– If none of the above, then p5 is open
• Each of these rules has a “confidence” level:
obtained from the number of matched
examples and the number of unmatched
examples in the training data
• But, the training data should be nearly
“complete” with regard to all possible “normal”
behavior
National Center for Supercomputing Applications
National Center for Supercomputing Applications
National Center for Supercomputing Applications
Yaha
• E-mail virus
• Sends itself to all email addresses in MS Address
Book, Messenger, Yahoo Pager and ICQ list
• Appears to be a screen saver
• Uses its own SMTP implementation
• Selects “Subject” line randomly from a list
• Messages are sent signed with the infected computer's
user name
• Attachment name is randomly selected from a list
• Kills anti-virus and firewall processes running on
infected computer
• Configures itself to execute each time an “.exe” files is
executed
National Center for Supercomputing Applications
Trin00
• Trinoo master/slave programs implement a
distributed denial of service tool
• A trinoo network of 227 systems was used on
Aug 1999 to flood a single system at the
University of Minnessota, rendering it unusable
for over two days
National Center for Supercomputing Applications
Attacks
• Ftp-write: Remote FTP user creates .rhost file in world
writable anonymous FTP directory and obtains local
login
• Imap: Remote buffer overflow using imap port leads to
root shell
• Mailbomb: A DoS attack where the mailserver receives
many large messages for delivery in order to slow it
down
• Phf: Exploitable CGI script which allows a client to
execute arbitrary commands on a machine with a
misconfigured web server
• Smurf: DoS icmp echo reply flood
National Center for Supercomputing Applications
Smurf
• Attacker sends PING request to an Internet
broadcast address
• The return address of the request is spoofed to
be the address of the attacker's victim
National Center for Supercomputing Applications