Best Known Methods in Security Events Correlation
Download
Report
Transcript Best Known Methods in Security Events Correlation
Best Known Methods in
Security Events Correlation
Mohammed Fadzil Haron
GSEC GCIA
April 12, 2005
Agenda
Correlation overview
Knowledge requirements
Methodology
Data representation
Reaction
2
IT@Intel
Correlation defined
A relation existing between phenomena or
things or between mathematical or
statistical variables which tend to vary, be
associated, or occur together in a way not
expected on the basis of chance alone…[1]
[1]
3
IT@Intel
http://www.webster.com
Overview
Correlation is the next security big thing in
importance
An important tool in the security analyst’s toolbox
for monitoring security events
To be most effective, most – if not all – events
should be examined
Defense in depth means more data from different
technologies, vendors, and products
Huge amount of data to analyze; terabytes in size
and growing
Reduce false-positive and false-negative findings
compared to use of a single product/technology
Expensive manned 24x7 monitoring capabilities
4
IT@Intel
Ultimate goal
Et = Dt + Rt
Exposure time (Et): The time the resource,
information, or organization is susceptible to attack
or compromise.
Detection time (Dt): The time it takes for the
vulnerability or the threat to be detected.
Reaction time (Rt): The time it takes for the
individual, group, or organization to respond and
eliminate or mediate the vulnerability or risk.
“Time Based Security” by Winn Schwartau
5
IT@Intel
Security events flow
6
IT@Intel
Axiom on correlation
1.
2.
3.
4.
5.
6.
7.
8.
You only see the tip of the iceberg
Know the environment and perimeter of defense
well
Don’t trust the tool; trust your judgment
“Automate whenever possible” [1]
Use the simplest data representation possible
Balance between over-correlated and undercorrelated
Get the big picture
“The truth is in the packet” [1]
[1]
7
IT@Intel
Toby Kohlenberg, Intel Corp.
Knowledge requirements
Know your environment
Know your perimeter of defense
Automate tasks
Simplify data representation
8
IT@Intel
Know your environment
Knowing the ins and outs of your network is a
necessity
–
–
–
–
External network, DMZ and internal network architecture
Other networks, such as VPN and dial-up
Logistical and geographical locations of servers and users
Different operation systems, applications and functionality
of servers and client machines
– Network switches and routers in use
– Logistical and geographical locations of critical servers
(DNS, WINS, DHCP) as well as high-valued servers (web
servers, servers containing intellectual properties)
– You cannot know everything yourself, so know the
individual experts on each piece of the network puzzle
9
IT@Intel
Example of environment knowledge
usage
Can isolate IP addresses of Internet, DMZ and
internal network for different categorization
– Potential detection of external attack versus inside job
VPN and dial-up services introduce other threats
and need to be given separate consideration
Allows assignment of customized severity levels
for different services, such as DNS and servers
housing intellectual property, for upgraded security
needs
10
IT@Intel
Source of events
Host level – Syslog, HIDS/HIPS, eventlog,
log files, apps logs, anti-virus signature
level
Network level – NIDS/NIPS, NBAD, firewall,
network routers and switch logs, active
directory logs, VPN logs, third-party
authentication logs
Audit – Vulnerability scanning, OS and
patch level
Knowledgebase – Software vulnerabilities
and exploits
11
IT@Intel
Know your perimeter of defense
Firewall
IDS
IPS
Audit capabilities
Host level defenses
PENS
Vulnerability scanning data
And so on.
12
IT@Intel
Know your firewalls
Location – Outer-facing, inner-facing, DMZ,
internal, internal isolated network
Type – Packet filter, stateful, application
firewall/proxy
What’s allowed versus denied
Capabilities versus shortcomings
13
IT@Intel
Know your IDS/IPS
Which product deployed? NIDS, HIDS/HIPS,
NIPS
Where were they deployed? What kind of
traffic is being monitored?
What product/vendor deployed?
Capabilities versus shortcomings
14
IT@Intel
Know your audit capabilities
Where are logs being kept? Syslog server
or logs on host?
How long have logs being kept? Rotated?
Know your syslog servers
15
IT@Intel
Host level defenses
Anti-virus logs
Minimum security specification compliance
enforcement software logs
OS, service packs, patches-level
information
16
IT@Intel
Automate tasks as much as possible
Daunting tasks to detect intrusion due to:
– Amount of data involved reaching terabyte range
– Complexity of network environment architecture
with Internet presence, DMZ, WAN, MAN, PAN,
LAN, VOIP, VPN, Dial-up
– Complexity of perimeter of defense
– Large IP address ranges used internally, that is,
using Class A 10.x.x.x
– Multiple internally isolated networks with
different type of policies, and access controls
17
IT@Intel
What and where to automate
Data aggregation – at data source and event
manager
Manual, repetitive tasks – at event manager
and reaction
Data correlation – event manager
Simplify data representation – event
manager console
Incident notification – event manager
18
IT@Intel
Group your assets
Break down IP addresses into groups, such
as internal, DMZ and others for Internet
Determine and group all critical servers,
such as DNS, WINS, and DHCP
Determine and group all high valued
servers, such as file shares, web servers,
and FTP servers, and encrypted content
servers for intellectual properties
19
IT@Intel
Types of correlation
Sets
– String a group of events together to generate a
trigger
Sequences
– String a group of events together in sequence or
particular order to generate a trigger
Statistical
– Deviation of normal behavior, such as mean or
normal curve
20
IT@Intel
Methods of correlation
Rule
– Manually constructed, easy to create/update. Usually explicit in
nature and can be applied to set, sequence and threshold types.
Contains three elements: condition, time interval, and response.
Heuristic
– Similar to anti-virus signature. One signature can detect multiple
variations. More implicit than explicit in nature, thus potential for
higher false positives/negatives.
Fuzzy Logic / Artificial Intelligence
– Model approach to correlation that can dynamically adapt to
changing environment. Difficult to produce and still immature;
very cutting-edge.
Hybrid
– No one doing them all yet. Commonly used are heuristic and rule.
21
IT@Intel
Correlation constraint
Time
– Time should be considered when creating time
box correlation
– Correct time is critical in correlation
– Time synchronization is crucial
Context
– Order of events sequence is important
– Context can be necessary in correlation rules
22
IT@Intel
Sample of correlation flow
INTERNET
External
attacker’s
IP address
NO
Deny
Outer Firewall
NO
Deny
Inner
Firewall
NO
Accept
Deny
YES
DMZ IDS
detection
Deny
Accept
Inner IDS
detection
YES
Outer IDS
detection
YES
Outer Firewall
Accept/Deny
NO
Inner
Firewall
NO
Accept
Deny
Inner IDS
detection
23
YES
IT@Intel
Inner
Firewall
NO
Accept
DMZ IDS
detection
Accept
Inner IDS
detection
YES
Deny
YES
Inner
Firewall
Accept
NO Inner IDS YES
detection
Graphical representation
Seeing is believing
Pros
– Can represent huge data in simple and easy to
understand graphs
Cons
– Not many tools (commercial/open source) with
this capability
– If exist, limited capabilities
24
IT@Intel
Effective graphics should…
Show the data
Avoid distorting data
Present a large volume of data in small
space
Make large data sets coherent
Show several levels of detail
Provide clear purpose of data presentation
Represent the data and not the underlying
technology, methodology, and design
25
IT@Intel
Forms of data representation
Graphs
Link graph
Charts
Data maps
Time series
Narrative graphics (space and time)
Animation
Visualization
Virtual reality
26
IT@Intel
Scanning graph
(One source to many target relationship)
Mar 14 08:33:20 66.34.244.12:2827 -> xxx.yyy.1.1:18905 SYN ******S*
Mar 14 08:33:20 66.34.244.12:2830 -> xxx.yyy.1.2:18905 SYN ******S*
Mar 14 08:33:20 66.34.244.12:2833 -> xxx.yyy.1.3:18905 SYN ******S*
Mar 14 08:33:22 66.34.244.12:2836 -> xxx.yyy.1.4:18905 SYN ******S*
Mar 14 08:33:22 66.34.244.12:2839 -> xxx.yyy.1.5:18905 SYN ******S*
Mar 14 08:33:22 66.34.244.12:2842 -> xxx.yyy.1.6:18905 SYN ******S*
Mar 14 08:33:22 66.34.244.12:2845 -> xxx.yyy.1.7:18905 SYN ******S*
Mar 14 08:33:20 66.34.244.12:2848 -> xxx.yyy.1.8:18905 SYN ******S*
Harder to internalize
Scan activity easily recognized
27
IT@Intel
Link graph
Stage 1 of worm
propagation
28
IT@Intel
Link graph
Stage 2 of worm
propagation
29
IT@Intel
Link graph
Stage 3 of worm
propagation
30
IT@Intel
Moving average
(Simple network anomaly detection)
180
160
140
Example:
Monitoring port 445
120
100
Monitored Events
Moving Average
80
60
40
20
0
1
2
3
4
5
6
7
8
31
9
IT@Intel
Increase in moving
average, showing
an increase in
activities
Animation movie
Inbound connection attempts to San Diego
State University (SDSU) from external source
(unauthorized)
Representing 332 GB of raw data, 3.4 billion
raw syslog records, and 1 million events
Period of 1996-2002 (6 years)
Available at http://security.sdsc.edu/probesanimations/index.shtml
32
IT@Intel
Animation movie
33
IT@Intel
Reaction to correlated data
Enforcement for malware cleaning
Blocking to minimize malware propagation
and attack
Investigation for malicious non-worm
activities
Learning mode for improving data (reducing
false-positives and false-negatives)
34
IT@Intel
Conclusion
Correlation is a must tool for information
security professionals
Time saved in detection will allow faster
response time
Faster response time will minimize damages
to your assets
35
IT@Intel
Questions?
36
IT@Intel
References
Event correlation;
http://www.computerworld.com/networkingtopics/networking/
management/story/0,10801,83396,00.html
“Protecting the Enterprise with Scalable Security Event
Management, Part II - Intelligent Event Correlation”; Michael
Mychalczuk;
https://www.sans.org/webcasts/show.php?webcastid=90468
“Thinking about Security Monitoring and Event Correlation“;
http://www.securityfocus.com/infocus/1231
37
IT@Intel