Transcript Memphis - Andrew.cmu.edu - Carnegie Mellon University

Waging War Against the
New Cyberwarrior
Tom Longstaff
[email protected]
CERT Coordination Center
Software Engineering Institute
Carnegie Mellon University
Pittsburgh, PA 15213
Sponsored by the U.S. Department of Defense
1
2001 Carnegie Mellon University
Incidents Reported to CERT/CC
2001
2002
2
2001 Carnegie Mellon University
52,658
82,094
Vulnerabilities Reported
3
2001
2,437
2002
4,129
2001 Carnegie Mellon University
Cyber Strategy
Cyber-war is not just simple hacking
Sociology of warriors vs. hackers
- Morale
- Organization
- Vigilance vs. assumed invulnerability
Motivation of warriors vs. hackers
- Accountability vs. anarchy
- Delayed vs. immediate gratification
- Internal vs. external gratification
Preparation of warriors vs. hackers
- Training
- Intelligence / strategy
4
2001 Carnegie Mellon University
Incident Trends
5
2001 Carnegie Mellon University
Intruder Technology
Intruders use currently available technology
to develop new technology
coordinate
propogate
coordinate
propogate
compromise
compromise
compromise
compromise
scan
scan
scan
scan
scan
1997
6
1998
2001 Carnegie Mellon University
1999
2000
Information Collection, Analysis
and Sharing for Situational
Awareness
7
2001 Carnegie Mellon University
Overview
Challenge statement
• Too much data – too little information – not shared
Operational Need
CERT Vision/Goals
Our Approach
Project Maturity
Wrap up
8
2001 Carnegie Mellon University
Data Challenge
System & Network Administrators overwhelmed
• Data overload
• Important data often not collected
• Local/parochial focus
Poor Network Situational Awareness
Network Security Information is not shared
• Unconnected “Islands of Information”
• Ineffective, non-standard security tools and processes
• Non-technical reasons (organizational and liability)
• Unwilling to yield autonomy to gain better information
Attackers share information more
efficiently
9
2001 Carnegie Mellon University
Our Vision
An operationally flexible system providing:
•Clear avenues for exchanging relevant data
•Improved local monitoring
•Improved cueing methods
•Cross organization analytical capabilities
•Improved indications and warning
•Cross organization situational awareness
10
2001 Carnegie Mellon University
Our Goal
Collect structured, sanitized, and
representative situational awareness data in a
standardized format to:
• Recognize and respond faster (prior to
damage)
• Permit collection of focused information on
activity and trends
• Alert operators for proactive response
• Provide tools for sites to manage incident
information
11
2001 Carnegie Mellon University
Bi-directional Solution
Top-down
•Collection, organization, and analysis
of data from wide, shallow sensors
Bottom-up
•Federation of data from narrow, deep
sensors
-Alerts from IDSs and Firewalls
-Raw data from sniffers & recorders
12
2001 Carnegie Mellon University
Top-Down Approach
Similar to the DEW line* – early indication that an attack may
be coming facilitated by sensing the entire network
Analysis for I&W
• Hacking involves reverse engineering: the attacker must
probe, examine and determine the “right” approach
• Frequently precursors to attacks are buried in the “noise”
• Improve our ability to detect attacker behavior in the preattack stages
Preventive Analysis
• Detect configuration errors
* DEW - Distant Early Warning
13
2001 Carnegie Mellon University
Top-Down
Edge Router
Netflow
Collector
T1
OC3
Internet
100Mb
Firewall/Router
Real time collection;
analysis and alert
tools
14
2001 Carnegie Mellon University
Intranet
Top-Down
Collect coarse data
• No payload data
• Headers Only – Source, Destination IP and ports;
protocol; times; traffic volumes (e.g. packets and bytes)
• Both inbound and outbound
Collect wide data
• >95% network coverage
• Multiple networks
Collect a lot of data
• Requires a data center with large computational and
storage capacity to facilitate historical analysis
• Scalable collection and analysis
• Outbound data indicates planted code or insiders
15
2001 Carnegie Mellon University
Top-Down - Wide Shallow Sensors
Netflow
• Originally defined by CISCO but increasingly
becoming standard
• See what the router sees
Records of “flows” created at the router
• Assist in routing and in reporting network traffic
statistics
Consists of flow records aggregated from packets
Sent to a collector and aggregated into different
information records for varied analysis.
16
2001 Carnegie Mellon University
Inbound Slammer Traffic
UDP Port 1434 Flows
40000000
35000000
30000000
Flows
25000000
20000000
15000000
10000000
5000000
Hour 1/24:00-1/25:18
17
2001 Carnegie Mellon University
18
16
14
12
10
8
6
4
2
0
22
20
18
16
14
12
10
8
6
4
2
0
0
Slammer: Precursor Detection
UDP Port 1434 - Precursor
160000
140000
120000
Flows
100000
80000
Series1
60000
40000
20000
0
0
1
2
3
4
5
6
7
8
9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 0
Hour 1/24:00 1/25:04
18
2001 Carnegie Mellon University
1
2
3
4
Slammer: Precursor Analysis
Focused on hours 6, 7, 8, 13, 14
Identified 3 primary sources, all from a known
adversary
All 3 used a fixed pattern
Identified responders: 2 out of 4
subsequently compromised.
19
2001 Carnegie Mellon University
Detecting Scans
Detect scans against client network hosts
•Higher intensity scans
•“Low and slow” scans
•Coordinated (distributed) scanning
20
2001 Carnegie Mellon University
Low-Packet Filtering
Sessions Vs. Time, December 12th-14th 2002
450
400
Low
Packet
Sessions
350
Records
300
All tcp
Sessions
250
200
150
100
50
0
Time
Time (30 second bins)
21
2001 Carnegie Mellon University
Stealth Tool Detection
We are studying extremely slow (“1 packet a day
scanner”) traffic on the Internet.
As an initial trial, we identified sources sending
between 1 and 3 packets of TCP (non-Web) traffic per
day into the client’s networks. We applied this to the
period September 1-11, finding that 0.00001% of the
traffic matched this pattern.
Further analysis yielded a fingerprint for one tool. The
tool’s profile appears to match Compaq Insight
Manager XE on the client network.
22
2001 Carnegie Mellon University
Bottom-Up Approach
Using data from Commercial Off the Shelf (COTS) security solutions
already deployed
• e.g., Intrusion Detection Systems, firewalls, system logs, Snort,
RealSecure, PIX, IPTables, syslog
Custom-developed technology (AirCERT), currently not present in
commercial products, to integrate, convert, analyze, and share the
data
Combination enables analysis of security event data from across
administrative domains
• Different entities
• Different scales:
- Subsidiary
- Corporation
- Sector
23
2001 Carnegie Mellon University
Bottom-Up
To other subnets….
Firewall/Router
AirCERT
Collector
IDS
System
24
2001 Carnegie Mellon University
Intranet
Sensor
(Packet Capture)
Web
Server
Mail
Server
Bottom-Up
Collect data from by security devices (firewalls and intrusion
detection devices)
• All or part of a packet
• Testimonials (e.g., IDS alerts), and associated contextual
data
Collect widely varied data
• Maximize network diversity (e.g., edge vs. transit; many
administrative domains)
• Maximize sensor diversity (e.g., IDS, firewall)
Configurable volume of data
• Determined by local site and collaborators
• Scalable collection and analysis
25
2001 Carnegie Mellon University
Bottom-Up
Implementation
• Flexible, open-source, standards-based reference
implementation of an Internet-scalable threat
assessment system
Capability consists of components for
• Data Collection
• Data Sharing
26
2001 Carnegie Mellon University
Implementation
Edge Router
Netflow
Collector
T1
OC3
Internet
100Mb
Firewall/Router
Collector
Intranet
IDS
System
27
2001 Carnegie Mellon University
Sensor
(Packet Capture)
Web
Server
Mail
Server
What Do You Do With This
Data?
Predictive numerical and statistical analysis
• Calculate long-term trends
• Profile traffic – map servers, create baselines
• Continual monitoring for attack precursors
Traffic Analysis
• Routing Anomalies and flaws
• Packet/Byte Characteristics
Weak general results can drive strong focused analysis
Analysis from Top-Down can drive Bottom-Up, and viceversa
28
2001 Carnegie Mellon University
What Else Do You Do With This
Data?
Manage and analyze event data at all points in reporting
hierarchy to detect and identify
• Compromise with cross-site data
• Coordinated, distributed attacks
• Slow and stealthy scans
• Network attack “fronts”
• Multi-site trends
- Distinguish between local and global activity
– Targeted scans
– Vulnerability probes
29
2001 Carnegie Mellon University
Integrating Top-Down & Bottom-Up
Analysis
Augment data collection and configuration at the “leaves”
Supplement or verify existing local security analyses and
processes
Employing cues gained from analysis at the “root”, focus
analysis on data previously deemed benign or ignored
Verify suggestive top-down and cross-site analysis by the
selective analysis of data collected at the “leaves”
30
2001 Carnegie Mellon University
ACID Architecture
Network Link
Snort or Firewall
ACID
Alert
Database
Web Server (PHP)
Browser
Browser
Browser
(Analyst #1) (Analyst #2) (Analyst #N)
ACID can only analyze what is in the Alert Database
31
2001 Carnegie Mellon University
Views of Data (grouping)
• ACID has no implicit analysis functionality -- only presents
the data by
-
Event (Signature)
Classification
IP Address
Port
Flow
Time
Sensor
- Charts grouped by time, IP, classification and ports
32
2001 Carnegie Mellon University
- User defined queries
Event (Signature) view
Unique Alert
• Identifies the different type of attacks
Reference
Signature
Number of
Number of
Src/Dst IP
Classification
Sensors
Total Number of Occurrences
from Main, click on number next to ‘Unique Alert’
33
2001 Carnegie Mellon University
First/Last
Occurrence
Classification view
• Identifies the different event classifications
Classification
Number of Number of Number of
Src/Dst IP
Events
Sensors
Total Number
of Occurrences
First/Last Occurrence
From Main, click on the number next to ‘categories’
34
2001 Carnegie Mellon University
Address view
• Identifies mostly frequently attacked machines
• Identifies network blocks of frequent attackers
IP Address
Fully Qualified
Domain Name
Number of
Sensors
Number of
Total Number Unique
of all Events
Events
From Main, click on number after ‘IP’
35
2001 Carnegie Mellon University
Number of
times seen
in opposite
direction
Port view
• Identifies most commonly targeted services
Port
Number of
Sensors
Number of
Unique Events
Total
Number
of all Events
First/Last Occurrence
Number of
Src/Dst IP
From Main, click on number after ‘Port’
36
2001 Carnegie Mellon University
Flow view
• Identifies suspicious events by flow activity
Protocol
FQDN and IP
of Source
FQDN and IP
of Destination
Unique
Destination
Ports
37
Number of
Unique Events
From Main, click on number after ‘Unique IP LInks’
2001 Carnegie Mellon University
Total
Number
of all Events
Sensor view
• Aggregate statistics on sensor
Sensor ID
Sensor Name
Total Number Number of
of all Events Unique
Events
First/Last Occurrence
Number of
Src/Dst IP
From Main, click on number next to ‘# of Sensors’
38
2001 Carnegie Mellon University
Temporal view Alert Listing
• Identifies event chronology
Event (Signature)
[ Query Seq. Number,
Returned
by any
Sensor
ID,
Event ID ]
39
2001 Carnegie Mellon University
Timestamp
Searches or Alert Listing
Layer-4 IP
Src/Dst
IP and Port encapsulated
Snapshots protocol
Temporal view (2)
Graph Alert Detection Time
• Graphs number of alerts aggregating on hour, day, or
month
• Visually represents peak attack periods
Time Interval
Number of Events
occurring in the
time interval
From Main, click on ‘Graph Alert Detection Time’
40
2001 Carnegie Mellon University
Drill-Down: Individual Alert
41
Click on the ID in any Alert Listing
2001 Carnegie Mellon University
Drill-Down: IP Address
• Provides statistics on an individual IP address
• Links to external registries and tools to gather information
about the address
Click on the IP address in any Alert Listing
42
2001 Carnegie Mellon University
User Interface: Main
43
2001 Carnegie Mellon University
User Interface: Navigation
Currently Selected Criteria
Browsing Buttons
Checkbox to
select alert
44
2001 Carnegie Mellon University
ACID Browser “Back”button
Alert Actions
Analysis Example:
Most Frequently Targeted TCP Services
45
2001 Carnegie Mellon University
Project Maturity
Top-Down
• Highly efficient data partitioning and packing format
- Does not rely on a relational database
– Packs 90+Gb per day into less than 30Gb
• Generic analysis tools written to perform ad-hoc analysis
- Processes a day’s worth of data in under 10 minutes
- Rapid analytical tool development API
• Operational deployment at sponsor site
Bottom-Up
• Prototype collection infrastructure developed and tested
• Active involvement in IETF security standards activity
• Pilot testing in progress
46
2001 Carnegie Mellon University
Project Maturity: Continuing Efforts
Involve more pilot sites
Improve analytical capabilities
Improve automated configuration
Continue standards development efforts
Increase collection diversity by supporting additional COTS
Persuade vendors to adopt standards
Planned Extensions to Netflow Analysis
• Enhanced with additional data based on payload but
packed into the existing form-factor
• Aggregation into session records
• Matching aggregated session records into transaction
records
47
2001 Carnegie Mellon University
Summary
Transformational approach to data collection, sharing,
analysis and response for Computer Network Defense
Provides timely, focused information to operators – providing
cues for immediate action
Provides tools for local, tailored analysis
Provides local, enterprise and Internet Situational Awareness
information
Levels the playing field
48
2001 Carnegie Mellon University
Modeling and Simulation
How do we drink from this fire hose?
Goal is to use the volume of information to
gain a predictive power over our adversaries
49
2001 Carnegie Mellon University
Emergent Algorithms
New Ideas
• Survivability is an emergent property of
a system
Attack
Recognize & Resist
• Emergent algorithms are distributed
computations that fulfill mission
requirements in the absence of central
control and global visibility
• Local actions + Near-neighbor
interactions => Complex global properties
Adapt
Recover
Impact
• A new methodology for the design
of highly survivable systems and
architectures
• Ability to produce desired global
effects through cooperative local
actions distributed throughout a
system (“self-stabilizing”)
50
2001 Carnegie Mellon University
Current Research
Design an emergent algorithm simulation
environment and language (“Easel”) to:
• Simulate and visualize the effects of
specific cyber-attacks, accidents and
failures
• Create a test-bed for mission-critical
systems
The nature of complex,
unbounded systems
Easel is a new computer language designed to
simulate complex, unbounded systems. Such
systems exhibit the following properties
•
•
•
•
•
•
Large numbers of autonomous components
Incomplete and imprecise information
Limited local knowledge
No central control
Bounded number of neighbors
Competing objectives
Such systems are more survivable because of
• adaptability
• graceful degradation
• no critical points of failure
• awareness of the local environment
51
2001 Carnegie Mellon University
Six explorations in survivability
cascade failure in organizations
failure propagation through an organizational network
network topology generation
survivability is a function of topology
simple network message routing
illustration of a very simple routing algorithm
network attackers and defenders
attackers compromise and defenders patch
epidemic dynamics
local contact leads to global infection
seismic collapse of a building
elastic response of linked beams to seismic shaking
52
2001 Carnegie Mellon University
Where can Easel help?
Provide independent verification that
complex system designs have no serious
survivability flaws
Analyze scenarios with respect to impact
of:
•
•
•
•
•
design assumptions
human error
incomplete or imprecise information
common mode failures
single point of failure leading to cascading
failure
• organized malicious attacks
53
2001 Carnegie Mellon University
Dealing with the Threat - Fusion
Analysis Efforts
Data Collection
• AirCERT
• Open source correlation
Individual Event Analysis
Statistical Analysis
Modeling and Simulation
54
2001 Carnegie Mellon University
What’s Next?
Our coordination of information must be
commensurate with the enemy’s ability to use
this information against us
We must create a new world of checks and
balances to match the appropriate use of
information in the pursuit of malfeasants
The key to this revolution is local administration
of information while maintaining global
coordination
55
2001 Carnegie Mellon University
Changes in Intrusion Profile
1988
• exploiting passwords
• exploiting known
vulnerabilities
The definition of “vulnerability”
on the Internet is approaching
that of the DoD in trusted
systems
56
2001 Carnegie Mellon University
Today
• exploiting passwords
• exploiting known
vulnerabilities
• exploiting protocol flaws
• examining source and
binary files for new
security flaws
• abusing anonymous FTP,
web servers, email
• installing sniffer programs
• IP source address
spoofing
• denial of service attacks
• widespread, automated
scanning of the Internet
• deep vuls in SNMP, SSL,
WEP, …
Scanning for Victims
Today:
Wide scale scanners collect information
on 100,000s of hosts around the Internet
Sniffers now use the same technology as
intrusion detection tools
Number and complexity of trust
relationships in real systems make victim
selection easier
57
2001 Carnegie Mellon University
Scanning for Victims
Tomorrow:
Use of data reduction tools and more queryoriented search capability will allow reuse of scan data
Inexpensive disk and computation time will
encourage the use of cryptography and persistent
storage of scan data
Scan data becomes a commodity like marketing
information
58
2001 Carnegie Mellon University
The Future of Probes
We’re very likely to see more:
• widespread brute-force scanning with little regard for
being detected
• stealthy probes like SYN and FIN that require packet
logging to detect
• attempts to hide the origin of the probes through
spoofing and decoys
• automated vulnerability exploits that probe and
compromise in a single step
59
2001 Carnegie Mellon University
Typical Intruder Attack
Internet
Yesterday
Intruder
60
scans remote sites to identify targets,
then
attacks vulnerable or misconfigured hosts
2001 Carnegie
Mellon University
Distributed Coordinated
Attack
Internet
Today Intruder
61
scans remote sites to identify targets,
then
attacks vulnerable or misconfigured hosts
2001 Carnegie
Mellon University
Distributed Coordinated
Attack
Uses 100s to 1000s of clients (10,000s)
Is triggered by a “victim” and “time” command
Command channels include IRC, SNMP,
ICMP
May include dynamic upgrade and be spread by
worms
Will simultaneously attack the victim from all clients
Today used in DoS attacks only
62
2001 Carnegie Mellon University
Issues for Responding to DoS Attacks
Filtering/detecting this attack is problematic!
The intruder’s intent is not always clear in denial of
service attacks. The intruder might be
• using the DoS attack to hide a real attack
• misusing resources to attack someone else
• attempting to frame someone else for the attack
• disabling a trusted host as part of an intrusion
Attacks also frequently involve
• IRC abuse
• intruders attacking each other
• retaliation for securing systems
63
2001 Carnegie Mellon University
The Future is Automation
Put these together and what do you get?
• tools to scan for multiple vulnerabilities
• architecture identification tools
• widely available exploits
• pre-packaged Trojan horse backdoor programs
• delivery and recon through active content
Bad news!
Together, these publicly available tools could be
modified to launch wide-spread scans and
compromise systems automatically.
64
2001 Carnegie Mellon University
Warning Signs of Today
We
•Tolerate unexpected program behavior
•Place little value on software quality
•Assemble parts with no clear idea what
each part does nor who created it
•Spread highly capable and functional
components through the hands of the
unenlightened
65
2001 Carnegie Mellon University
Tom Longstaff’s Predictions for
the Next Decade (well, at least the next 3
years)
Network crime on the rise
Many countries and NGOs preparing
information warfare weapons
Insiders and planted vulnerabilities control
the battlespace
Information warfare will be combined with
traditional tactics (e.g., Iraq)
66
2001 Carnegie Mellon University