Transcript - TMA portal

Connect. Communicate. Collaborate
Experiences with tools for
network anomaly detection in the
GÉANT2 core
Maurizio Molina, DANTE
COST TMA tech. Seminar
Samos, 23rd Sep 2008
The GÉANT Network
Connect. Communicate. Collaborate
• DANTE operates GÉANT2
• Backbone network for National Research and Education
Networks in Europe
• 30+ NRENs, 2 global connectivity providers (Telia and
GCrossing), peerings with other research networks
(Abilene, Canarie, Clara, TEIN2, SINET…)
The GÉANT Network (IP layer)
Connect. Communicate. Collaborate
• 20 Juniper
routers
Pls see www.dante.net
• tenths of
GBit/s of
aggregated
traffic
• Main
accesses
and the
backbone
10Gbit/s
The Services
Connect. Communicate. Collaborate
• So…. Just a big pipe? No!
• Services
– Dedicated L1-L2 circuits via multiple technologies
– Performance Monitoring services (perfSONAR) NEW!
– Support for federation of National AA Infrastructures
(eduGAIN) and wireless roaming (eduROAM)
– Security Service
Very NEW!
The vision:
enhance NRENs security
Connect. Communicate. Collaborate
• NRENs have their (+ - evolved…) CERTs to deal with
security
• and DANTE can filter traffic on GÉANT upon NRENs
request….
! BUT !
• Can we be more proactive to NREN CERTs exploiting
the visibility of the GN2 core?
The vision (cont.):
enhance NRENs security
Connect. Communicate. Collaborate
• Approach: NetFlow (+ Routing data) & good processing tools
NetFlow v5 collector
• Netflow collected on all
peering interfaces
• 1 / 1,000 Sampling
• ~3k flows/s
Proof of concept: Can we
identify anomalies in the core?
Connect. Communicate. Collaborate
• Anomalies are often “hidden”
Requirements:
 High detection
rate
 Low false
positives
 Anomaly
classification
 Evidence
collection
NfSen
From “volume” to “IP feature
entropies”
Connect. Communicate. Collaborate
•“IP features entropies”
•Simple linear filter
Drilling down on peaks
-Concentration of DST
IPs and DST ports
receiving flows
-Dispersion of SRC IPs
and SRC ports
Connect. Communicate. Collaborate
• IRC server in
Slovenia, receiving
a lot of 60 bytes
syn pkts on port
6667, mainly from a
/16 Subnetwork of
an University in the
Netherlands.
• Likely a “BotNet
war”?
Drilling down on peaks
(cont.)
Connect. Communicate. Collaborate
- Concentration of SRC and
DST IPs and SRC ports
- Dispersion of DST ports
• Portscan of host in
CARNET, from 4
hosts, 29 bytes
packets
Open source tools
Connect. Communicate. Collaborate
• Results:
– anomalies are observable in the GÉANT2 core
– Novel methodologies (IP Features entropy) for their
classifications are applicable
• Limits:
– NfSen does not fuse NetFlow and Routing data
– Extensions would need to be run (and tuned) on all
ingress/egress points
– No support, no guaranteed development
Commercial tools
Connect. Communicate. Collaborate
• Test started Jun 08 (3 tools)
– Tool 1
• PCA, entropy
– Tool 2
• Large scale DDoS and Worm spread
– Tool 3
• Per host behaviour
Tool 1 (as a security tool…)
Connect. Communicate. Collaborate
• Two main novel elements
– Principal Component Analysis (PCA)
– Both Volume and IP features Entropy anomaly
detection
• Address what makes anomaly detection a complex task
– PCA: single parameter to control detection sensitivity,
even if anomalies are attributed to specific OD pairs
– Entropy: Detection of both low volume (scans) and high
volume (DoS) anomalies
Demo….
• …. Or Screenshots….
Connect. Communicate. Collaborate
Connect. Communicate. Collaborate
Connect. Communicate. Collaborate
Connect. Communicate. Collaborate
Connect. Communicate. Collaborate
Connect. Communicate. Collaborate
Connect. Communicate. Collaborate
Connect. Communicate. Collaborate
Connect. Communicate. Collaborate
Tool 2
Connect. Communicate. Collaborate
• Well-established (and expensive!) solution for detecting
“large” events
• Originally based on large volume shifts only
• Now enhanced to give alerts on “fingerprints” (e.g.
communication with C&C servers)
– Shared by (part) of the user community (50 out of 120)
• No usage of routing data
– though “zones” can be manually created via BGP
prefixes lists
• Traditional threshold based detection (although adaptive)
Tool 3
Connect. Communicate. Collaborate
• Per host behavioural analysis
• rather complex “scoring” system to distinguish normal from
abnormal behaviour. Proprietary algorithms
• Doesn’t use routing info
– though “zones” can be manually created via BGP
prefixes lists
• Potentially attractive methodology
• Concerns on scalability and accuracy with 1,000 sampling
lessons learnt and
directions for research
Connect. Communicate. Collaborate
• Manual validation is required to confirm/correct anomalies
– More automatic intelligence to help this process
– Fusion with other data sources (router logs?
Honeynets?)
• Detection space of 3 tools often disjoint
– (Standard) anomaly injection
• Operations need supported tools to support services
• If choice is among published but “not a tool” or “secret but
supported and (claiming to) work” => risk to stick to those!
– Fill the gap towards TOOLS!
Connect. Communicate. Collaborate
Thank you!
[email protected]