Transcript What is Botnet?

An Introduction of
Botnet Detection – Part 1
Guofei Gu, Wenke Lee (Georiga Tech)
Reference

Guofei Gu, Wenke Lee, et al.

BotHunter: Detecting Malware Infection through IDS-driven
Dialog Correlation


BotSniffer: Detecting Botnet Command and Control Channels in
Network Traffic


ACM NDSS 2008
BotMiner: Clustering Analysis of Network Traffic for Protocol- and
Structure-independent Botnet Detection


USENIX Security 2007
USENIX Security 2008
Moheeb Abu Rajab, et al.

A Multifaceted Approach to Understanding the Botnet
Phenomenon

2009/5/15
ACM IMC 2006
Speaker: Li-Ming Chen
2
What is Botnet? (1/2)



Bots: compromised hosts, “Zombies”
Botnets: networks of bots that are under the control of a
human operator (botmaster)
(generally looks like) Worm + C&C channel


Command and Control Channel
Disseminate the botmasters’ commands to their bot armies
Communication (IRC, HTTP, … (can be encrypted))
Worm
2009/5/15
Attack
(DoS, spamming, phishing site, …)
Propagation
(vulnerabilities, file sharing, P2P, …)
Speaker: Li-Ming Chen
3
What is Botnet? (2/2)

C&C Channel – Comm. protocols:

Most popular: IRC (Internet Relay Chat)



Open-source protocol, flexible
Others: HTTP, P2P…
Uses of Botnets





2009/5/15
DDoS
Spam
ID/information theft
Phishing attacks
Distributing other malware
Speaker: Li-Ming Chen
4
Lifecycle of a Typical Botnet Infection
Major roles:
• Botnet (bots)
• Victim
• C&C Server
• Botmaster
authentication
(optional)
(borrow infection
strategies from
traditional
2009/5/15 malicious attacks)
6. Malicious activities (e.g., DDoS)
Speaker: Li-Ming Chen
5
Why Botnet is hard to detect?

Botnet infection involves multiple steps



However, predefined state transition models do
not work well in botnet infection monitoring
Due to:



Only looking at one specific aspect likely to fail
Rare to accurately detect all steps
Difficult to predict the order and time-window in
which these events are recorded
Botnet can have very flexible design of C&C
channels
2009/5/15
Speaker: Li-Ming Chen
6
Overview of the 3 Approaches
BotMiner
(Security’08)
2009/5/15
Speaker: Li-Ming Chen
7
Outline





What is Botnet?
BotHunter – Detecting Infection Lifecycle
BotSniffer – Detecting C&C Channel
BotMiner – Protocol- and Structure-independent
Botnet Detection
My Comments
2009/5/15
Speaker: Li-Ming Chen
8
BotHunter (USENIX Security’07)


Snort-based sensor suite for botnet infection
detection
Recognize the infection and coordination dialog
that occurs during a successful bot infection


2009/5/15
Observe the 2-way communication flows between
internal assets and external entities
Identify data exchanges that match a state-based
infection sequence model (by dialog correlation)
Speaker: Li-Ming Chen
9
BotHunter System Architecture
Recognize bi-directional
warning signs of local infection
Correlate this evidence against
the defined dialog infection model
dialog
transitions
Allows user to report
bot infection profiles
for global evaluation
2009/5/15
Speaker: Li-Ming Chen
10
SCADE & SLADE

SCADE (sCan Anomaly Detection)

Inbound & outbound scan detection (E1 & E5):



Based on protocol and Dst. port, monitor number of scans to
or from local hosts
Assign weights to different ports and compute anomaly score
for each local host
SLADE (payLoad Anomaly Detection)


2009/5/15
Based on n-gram byte distribution anomaly detection
More robust to polymorphic blending attack
Speaker: Li-Ming Chen
11
Bot Infection Dialog Model (1/2)


Design bot infection dialog model for assessing
bi-directional flows across the network boundary
Roles:


A – attacker, V – victim, C – C&C server
5 potential dialog transitions:





2009/5/15
E1: external to internal inbound scan
E2: external to internal inbound exploit
E3: internal to external binary acquisition
E4: internal to external C&C communication
E5: internal to external outbound infection scanning
Speaker: Li-Ming Chen
12
Bot Infection Dialog Model (2/2)
Not strict ordering of events,
but a typical infection dialog
(BotHunter) min. requirement
for bot declaration:
1. E2 AND E3-E5
2. At least two distinct signs of
E3-E5
 Assign weights
to different events
And then perform
correlation
2009/5/15
Speaker: Li-Ming Chen
13
Network Dialog Correlation Matrix
Summarize ongoing
dialog warnings for
a specific local host
Sensor alerts for each
dialog warning
• Each dialog might have 1 or 2 expiration intervals
(soft/hard prune timer)
• When timer expires, compute dialog threshold score and
detect bot based on 2 conditions
2009/5/15
Speaker: Li-Ming Chen
14
Output: Bot
Infection Profile


(Example of a BotHunter profile)
When a dialog sequence
is found and cross the
threshold for bot
declaration, BotHunter
produces a bot profile
Represents a full analysis
of roles of the bot dialog
2009/5/15
Speaker: Li-Ming Chen
15
Outline





What is Botnet?
BotHunter – Detecting Infection Lifecycle
BotSniffer – Detecting C&C Channel
BotMiner – Protocol- and Structure-independent
Botnet Detection
My Comments
2009/5/15
Speaker: Li-Ming Chen
16
BotSniffer (ACM NDSS’07)

Identify centralized botnet C&C channels in a
monitored network


Including C&C servers and bots
Why focus on C&C channel?

C&C is essential to a botnet


C&C detection is important


2009/5/15
Without C&C, bots are just discrete, unorganized infections
C&C channel is relatively stable and unlikely to change within
botnets
Botmaster control bots via C&C channel (weakest point)
Speaker: Li-Ming Chen
17
BotSniffer – the Approach

Observation:



Due to the pre-programmed activities related to C&C,
Bots within the same botnet will likely demonstrate
spatial-temporal correlation and similarity
BotSniffer:




2009/5/15
Focus on IRC & HTTP based C&C channels
Capture spatial-temporal correlation in network traffic
Utilize statistical algorithms to detect botnets
Has theoretical bounds on FP and FN rates
Speaker: Li-Ming Chen
18
Centralized C&C Channels
• Botmaster can control bots
via broadcast (real-time control)
• Bots respond to the commands
in pre-programmed fashions
• Relatively loose behaviors
(not real-time)
• Sets the command in a file
2009/5/15
Speaker: Li-Ming Chen
19
Spatial-Temporal Correlation and
Similarity


Regardless of the push and pull style
Invariants in botnet C&C channel:

1. bots need to connect to C&C servers


2. bots need to perform tasks and respond to the
received commands (and in a similar fashion)


2009/5/15
(Virtually) long-lived session of C&C channel
Message response (IRC-based reply)
Activity response (perform malicious tasks)
Speaker: Li-Ming Chen
20
Response Crowd of Botnet Members
Bots have much stronger (and more consistent) synchronization and
correlation in their responses than normal users
2009/5/15
Speaker: Li-Ming Chen
21
BotSniffer System Architecture
(data reduction)
Port-independent,
payload inspection
(focus on IRC)
2009/5/15
Speaker: Li-Ming Chen
22
Correlation Engine

Group clients according to their Dst. IP and ports

Perform group analysis of spatial-temporal
correlation and similarity based on two
properties

Response Crowd Density Check


Response Crowd Homogeneity Check

2009/5/15
(Quantity  everybody acts!)
(Quality  everybody acts in the same way!)
Speaker: Li-Ming Chen
23
Response Crowd Density Check

For each time window, check if there is a dense
response crowd in a group
E.g., > 50% group members have message/activity behavior


Use TRW (threshold random walk) to compute the
anomaly score and detect a sequence of crowds
(H1: Botnet)
Pr(Yi | H1) = θ1
Pr(Yi | H0) = θ2
(H0: not Botnet)
Likelihood that botnet detected
the i-th response crowd is dense ?
2009/5/15
Speaker: Li-Ming Chen
24
Response Crowd Homogeneity Check

Check if most of the group members have very
similar response


(currently only used for message response, IRC)
Also use TRW, but how to get Yi ?



Yi  the i-th response crowd is homogeneous ?
Use a clustering technique to obtain the largest cluster
of similar messages in the crowd
And calculate the ratio of the size of the cluster over
the size of the crowd

2009/5/15
Ratio > threshold  Yi = 1
Speaker: Li-Ming Chen
25
Outline





What is Botnet?
BotHunter – Detecting Infection Lifecycle
BotSniffer – Detecting C&C Channel
BotMiner – Protocol- and Structure-independent
Botnet Detection
My Comments
2009/5/15
Speaker: Li-Ming Chen
26
BotMiner (USENIX Security’08)

Why do we need BotMiner?


2009/5/15
Botnets can change their
 C&C content (encryption, etc.),
 Protocols (IRC, HTTP, etc.),
 Structures (P2P, etc.),
 C&C servers,
 Dialog models
 Bothunter, BotSniffer may be evaded (We need to
consider more)
Speaker: Li-Ming Chen
27
BotMiner – the Goal

Detect groups of compromised hosts within a
monitored network that are part of a botnet


Not concern the way hosts get infected
The approach is




2009/5/15
Independent of the protocol and structure used in C&C
channel
Independent of the content of the C&C communication
Low FP and FN
Efficient
Speaker: Li-Ming Chen
28
BotMiner – the Approach

Botnet is “a coordinated group of malware
instances that are controlled via C&C channels”

 monitor botnet in two planes:

C-plane (C&C communication traffic)


A-plane (malicious activity traffic)


“who is talking to whom”
“who is doing what”
 Find a coordinated group pattern in both
kinds of activities
2009/5/15
Speaker: Li-Ming Chen
29
BotMiner System Architecture
• Extract features from the raw logs
and perform clustering
• Combine results and make final
decision
Using different methods
to analyze outbound traffic
(based on Snort)
log
log
Record flows, contact activities
2009/5/15
Speaker: Li-Ming Chen
30
C-Plane Clustering
Data reduction,
(Filter out irrelevant flows)
Make clustering more efficient
Further reduce
traffic workload
4 features:
• temporal – fph, bps
• spatial – ppf, bpp
2-step clustering,
Coarse-grained clustering + fine-grained clustering
(why?)
2009/5/15
Speaker: Li-Ming Chen
31
A-Plane Clustering
• 2-layer clustering
• based on activity type and features
(more straightforward)
2009/5/15
Speaker: Li-Ming Chen
32
Cross-Plane Clustering


Idea: crosscheck clusters in the two planes to
find out intersections that reinforce evidence of a
host being part of a botnet
Ai
Aj
.h
1. Botnet score s(h) for host h
(weight)
交集占聯集的比例

2. find similarity between bots (hi) and cluster
2009/5/15
Speaker: Li-Ming Chen
33
Outline





What is Botnet?
BotHunter – Detecting Infection Lifecycle
BotSniffer – Detecting C&C Channel
BotMiner – Protocol- and Structure-independent
Botnet Detection
My Comments
2009/5/15
Speaker: Li-Ming Chen
34
Summary

Bothunter:



Botsniffer:



Vertical Correlation
Correlation on the behaviors of single host
Horizontal Correlation
Focus on centralized C&C botnets
Botminer:


2009/5/15
Extension on Botsniffer
No limitations on the C&C types.
Speaker: Li-Ming Chen
35
Botnet Detection – Part 2




Focus on detailed approaches
Focus on evaluation methodologies and results
Possible evasions and solutions in Botnet
detection
Discussion
2009/5/15
Speaker: Li-Ming Chen
36
My Comments

Divide and conquer



Attacks are anticipated to be more stealthy



Understand the detailed attack behaviors
Try to detect attacks by correlating attack features
Sophisticated, multiple stages...
Other evasion techniques
 complex detection approaches that make
more assumptions about the attack might work
well for that specific attack

2009/5/15
But, not robust (easy to evade)
Speaker: Li-Ming Chen
37