Transcript Internet Traffic Analysis for Threat Detection

Internet Traffic Analysis
for Threat Detection
Joshua Thomas, CISSP
Thomas Conley, CISSP
Ohio University
Communication Network Services
Abstract
 Useful logs may already exist at your institution.
 Network transaction logging is a very useful, flexible,
and inexpensive tool for network security.
 Comprehensive network security relies on log
collection and analysis.
 Analysis of log files can be automated, and can
provide information that can be the basis for
prevention and response procedures.
Start with what you have
 The collection and analysis of network transaction
data is useful for a wide range of tasks




Security management
Network billing and accounting
Network operations management
Performance analysis
 As a result, some form of network transaction logs
may already exist within your institution, even if not
specifically implemented for network security reasons.
“Pointed stick”
 Low cost, high returns
 Simple to implement
 Nonspecific, flexible
 Non-restrictive
Fundamental need
 Network transaction logs are arguably the
most basic, necessary countermeasure in
network security.
 Logs should form the basis for decisions
regarding other security initiatives.
 Traffic analysis will be necessary to validate
the performance of other security
countermeasures.
Needs pyramid: Maslow’s Hierarchy
Self-actualization
Esteem needs
Belongingness and Love needs
Safety needs
Biological and Physiological needs
Needs pyramid: Network Security
IDS/IPS
Firewalls
Host Security
Security Staff
Network Transaction Logs
Transparent monitor
 Acts as a passive device, gathering traffic and
performance statistics at appropriate places in
networks (server or client locations)
 Is not necessarily a point of failure in your
network
 Cannot alter network traffic, as active devices
such as firewalls or IDS/IPS systems.
 However, monitoring can co-exist with other
network security devices, such as IPS/IDS
Transparent monitor: Simple setup
Upstream Provider
Hub
Network
Network Monitor
Scalable
 Mirroring traffic is relatively inexpensive.
 Institutions may choose to capture as much data as
possible and only perform limited analysis as needed.
 There are appropriate solutions for implementing
network transaction monitoring at just about every
level of a network.
 Small lab environment
 Single department
 University border
Transparent monitor: Large-scale
ISP 1
Network
Monitor
ISP 2
Selective memory
 In order to be able to store and analyze high
volumes of traffic, the memory demands must
be reduced in some way.
Selective memory: Depth
 IPS/IDS systems generally select certain
transactions (via signature matching, etc.) for
storage and analysis. In other words, only
communications that match a selection criteria
are recorded, and all other data is ignored.
!
!
Selective memory: Breadth
 Flow monitoring accounts for every transaction, but
does not retain the content of the transactions.
 Transactions contain both routing information and
content. Only routing information is retained.
 Applications that can capture this sort of transaction
data include Argus, tcpdump, Ethereal, cflowd, etc.
Flow metrics
 Metrics generally captured in network
transaction logs include:
 Source, destination IP addresses (for IP traffic)
 Beginning, end times
 Packet count
 Byte count
 TTL (for IP traffic)
 TCP flags (for TCP/IP traffic)
 TCP state progression (for TCP/IP traffic)
 Base sequence numbers (for TCP/IP traffic)
Inference
 Certain traffic characteristics are very useful in
making inferences about the nature of the
traffic.
 Examples:
 Amount of bandwidth consumed
 Number of connection attempts
 Connections to unused address ranges
Automation
 Identifying problems through inference can be
automated.
 Once the criteria has been clearly defined,
then the tasks that were once done by
humans can be performed by simple
programs.
 Once the identification of problems is
automated, then those results can be fed into
response procedures.
Examples
 Compare logs with blacklists, such as knownspyware or spam source IP lists
 Examine traffic destined for non-populated
subnets
 Noise-floor analysis
 TCP port usage
Endless possibilities
 We are constantly discovering new uses for
network transaction logs
About our institution
 4,820 employees (1,069 full-time faculty)
 20,143 students (18,497 full-time students)
 90+ Mbps Internet bandwidth (2 ISP’s)
 6,000,000,000+ packets per day
 3,000,000,000+ source packets
 3,000,000,000+ destination packets
 2,400+ GB per day (500+ DVD-ROMs)
 727 source GB per day
 1,675 destination GB per day
 ~12 GB Argus log files generated per day, on average
(0.6% of the total bytes represented)
References/Resources
 RFC 2724, “RTFM: New Attributes for Traffic
Flow Measurement.” (http://www.rfceditor.org/rfc/rfc2724.txt)
 Argus: http://www.qosient.com/argus