Transcript A Signal Analysis of Network Traffic Anomalies

A Signal Analysis of
Network Traffic Anomalies
Paul Barford, Jeffrey Kline, David
Plonka, and Amos Ron
Network Traffic Anomalies
 Failures
and attacks
 Detection part of everyday work for
administrators
 Data derived mainly from two sources

SNMP
• Queries to nodes; mostly counts of activity

IP flows
• More specific than SNMP
Related Work
 Statistical
detection of anomalies
 Past work on malicious (DoS, port scan)
behavior detection
 Flash crowd studies
Data

Analysis based on SNMP and IP data
 Taken from a border router at University of
Wisconsin-Madison
 Flows sampled 1 in 96 packets
 Journal of known anomalies and events was
kept




Network
Attack
Flash
Measurement
Current Practices
 Network
operators use ad hoc methods
 Rely on operator’s personal experience
 Handling SNMP data


Graph network data
Alarms for certain events
 Flow

data handling less mature
Popular tool converts into time-series data
Method
 Wavelet
analysis
 Divides the data into strata
 Low-frequency strata: slow-varying trends
 High-frequency strata: spontaneous
variations
Wavelet Processing
 Analysis/Decomposition


Break down the signal into the strata
Run different filters for the different
frequencies
 Synthesis

Inverse of decomposition
 Wavelet

algorithms
Recombine strata, but filtering out unwanted
data
Cont.
 The
technique used by the authors
synthesizes 3 separate parts of the signal
 Total amount within the parts will be longer
than the actual signal
 L – Captures long term patterns; ideal for
weekly trends
 M – Captures midrange patterns; ideal for
daily trends
 H – High frequency data capture
Anomaly Detection
 Normalize

H- and M- to a variance of 1
Compute local variability of data within a
moving window (3 hours)
 Combine
variability of H- and M Apply thresholding
IMAPIT
 Development
environment for anomaly
detection
 Used the H-, M-, and weights for both to
determine deviation scores
 Anomalies tend to have deviation over 2.0
Characteristics of Ambient Traffic

Need data free of anomalies as a calibration
Flash Crowds

Test data: New Linux release on ftp mirror
Short-lived Anomalies
Discriminator for Short-term
Anomalies
Two DoS Events
Analysis of Network Outage
Deviation Score Evaluation
 Used
logged anomalies as baseline for
evaluation

Of 39 logged anomalies, detected 38
Comparison to Holt-Winters

Holt-Winters is an exponential smoothing algorithm




Uses baseline (intercept), linear trend (slope), and seasonal
trend
Aberrations are detected by detecting a certain amount of data
outside the threshold range within a window
Different from wavelet in that the different strata are
processed separately whereas Holt-Winters is one
prediction function
Compared to an alternative using Holt-Winters algorithm



Holt-Winters detected 37 anomalies
Both missed anomalies would have been detected with a larger
window
Holt-Winters more sensitive
Conclusion
 Performs
comparably to Holt-Winters
 Deviation score detection can be effective
 Learning methods potentially used in the
future
 Study ways of classification