Transcript Internet Traffic Monitoring and Analysis : Methods and Applications

5. Passive Monitoring Techniques
Internet Traffic Monitoring and Analysis:
Methods and Applications
(1)
POSTECH
DP&NM Lab.
5. Passive Monitoring - Packet Capturing
Probe system
Probe system
Mirroring
Splitting
 Packets can be captured using Port Mirroring or Network Splitter (Tap)
How it works
Advantage
Disadvantage
Port Mirroring
Network Splitter
- Copies all packets
passing on a port to
another port
- No extra hardware
required
- Processing overhead
on router/switch
- Splits the signal and send
a signal to original path
and another to probe
- No processing overhead
on router/switch
- Splitter hardware required
Internet Traffic Monitoring and Analysis:
Methods and Applications
(2)
POSTECH
DP&NM Lab.
5. Passive Monitoring - Packet Capturing
 Difficulties in packet capturing
 Massive amount of data
 How much packet data is generated from 100 Mbps network in an hour?
 Port speed ⅹIn&Out ⅹLink Utilization ⅹ sec/hour = throughput
100 Mbps ⅹ 2 ⅹ
0.5
ⅹ 3600
= 360 Gbps
 Throughput / avg. packet lengthⅹ bytes of packet data = data size
360 Gbps / (1500 ⅹ 8) ⅹ
30
= 1 Gbyte
 Processing of high-speed packets
 Processing time for 100 Mbps network
Port speed ⅹ In&Out ⅹ Link Utilization / average packet length
= 8333 packets/sec => 0.12 msec/packet
100 Mbps
1 Gbps
1 Tbps
Data size per hour (assume 0.5 link util) 1 Gbyte
10 Gbyte
10 Tbyte
Processing Time per packet
0.012 msec
0.012 μsec
Internet Traffic Monitoring and Analysis:
Methods and Applications
0.12 msec
(3)
POSTECH
DP&NM Lab.
5. Passive Monitoring - Sampling
 If the rate is too high to capture all packets reliably, there is
no alternative but to sample the packets
 Sampling algorithms: every Nth packet or fixed time interval
1
2
3
4
5
6
7
8
9
10
11
(a) 2:1 sampling
0 msec
1 msec
2 msec
3 msec
4 msec
(b) 1 msec sampling
Internet Traffic Monitoring and Analysis:
Methods and Applications
(4)
POSTECH
DP&NM Lab.
5. Passive Monitoring - Flow Generation
flow 1
flow 2
flow 3
flow 4
 Flow is a collection of packets with the same {SRC and DST IP address,
SRC and DST port number, protocol number, TOS}
 Flow data can be collected from routers directly, or standalone flow
generator having packet capturing capability
 Popular flow formats
 NetFlow (Cisco), sFlow (sFlow.org), IPFIX (IETF)
 Issues in flow generation




What information should be included in a flow data?
How to generate flow data from raw packet information efficiently?
How to save bulk flow data into DB or binary file in a collector?
How long should the data be preserved?
Internet Traffic Monitoring and Analysis:
Methods and Applications
(5)
POSTECH
DP&NM Lab.
5. Passive Monitoring - Flow Technology: NetFlow
 Cisco NetFlow
 is an option configurable in Cisco routers that exports data on
each IP flow passed through an interface
 Cisco IOS NetFlow technology
 is an integral part of Cisco IOS software that collects and
measures data as it enters specific routers or switch interfaces
 enables to perform IP traffic flow analysis without custom probes
 3 key components in a NetFlow system
 Flow Exporter
 Flow Collector
 Network Data Analyzer (Flow Analyzer)
Internet Traffic Monitoring and Analysis:
Methods and Applications
(6)
POSTECH
DP&NM Lab.
5. Passive Monitoring - Flow Technology: NetFlow
 NetFlow Export Datagram
Header
· Sequence number
· Record count
· Version number
Flow
Record
Flow
Record
Flow
Record
Flow
Record
Flow
Record
 Version 1, Version 5, Version 7, Version 8
 Version 1: original format supported in the initial Cisco IOS
software releases.
 Version 5:
Usage
• Packet Count
• Byte Count
• Source IP Address
• Destination IP Address
Time
of Day
• Start Timestamp
• End Timestamp
• Source TCP/UDP Port
• Destination TCP/UDP Port
From/To
Application
Port
Utilization
QoS
Internet Traffic Monitoring and Analysis:
Methods and Applications
• Input Interface Port
• Output Interface Port
• Type of Service
• TCP Flags
• Protocol
(7)
•
•
•
•
•
Next Hop Address
Source AS Number
Dest. AS Number
Source Prefix Mask
Dest. Prefix Mask
Routing
and
Peering
POSTECH
DP&NM Lab.
5. Passive Monitoring - Flow Technology: NetFlow
 Version 7
 Enhancement that supports Cisco Catalyst 5000 Series switches
equipped with NetFlow Feature Card (NFFC).
 Version 8
 developed mainly to MINIMIZE output size from exporter by
adding Router-Based Aggregation schemes
type UDP datagram
ASMatrix
ProtocolPortMatrix
SourcePrefixMatrix
DestPrefixMatrix
PrefixMatrix
records/datagram
51
51
44
44
35
max udp pktsize
1456
1456
1436
1436
1428
 available on Cisco routers from IOS release 12.0(3)T
Internet Traffic Monitoring and Analysis:
Methods and Applications
(8)
POSTECH
DP&NM Lab.
5. Passive Monitoring - Flow Technology: sFlow
 sFlow is described in RFC 3176: “InMon Corporation's
sFlow: A Method for Monitoring Traffic in Switched and
Routed Networks”
 sFlow is a monitoring technology that gives visibility into
the use of networks, enabling performance optimization,
accounting/billing for usage, and defense against security
threats
 sFlow provides an effective means of embedding traffic
monitoring in high-speed switches and routers
 sFlow samples packets using statistical sampling theory
Internet Traffic Monitoring and Analysis:
Methods and Applications
(9)
POSTECH
DP&NM Lab.
5. Passive Monitoring - Flow Technology: sFlow
 sFlow Datagram Format
 is specified using the XDR standard
 XDR is a standard for the description and encoding of data.
(eXternal Data Representation Standard, RFC1014)
 version 4
 Packet Header Data
 Header Protocol (Format of sampled header)
 Frame_length
 Header bytes
 Packet IP v4 Data






 Packet IP v6 Data






Length
Protocol (IP Protocol Type)
src_ip / dst_ip
src_port / dst_port
TCP flags
tos
Internet Traffic Monitoring and Analysis:
Methods and Applications
(10)
Length
IP next Header
src_ip / dst_ip
src_port / dst_port
TCP flags
IP priority
POSTECH
DP&NM Lab.
5. Passive Monitoring - Flow Technology: sFlow
 Equipment Supporting sFlow
 Foundry Networks
 BigIron, FastIron, NetIron Series
 InMon’s sFlow Probe
 By attaching to a monitor/SPAN
port
 Gathers mirrored or tapped (using
a splitter) traffic data
 The resulting data is forwarded in
sFlow datagrams to a central sFlow
collector (for example InMon Traffic
Server) for analysis.
Source: InMon
Internet Traffic Monitoring and Analysis:
Methods and Applications
(11)
POSTECH
DP&NM Lab.
5. Passive Monitoring - Flow Technology: IPFIX
 IPFIX (IP Flow Information eXport) Working Group
 http://www.ietf.org/html.charters/ipfix-charter.html
 Background
 There are a number of IP flow export systems in
common use
 These systems differ significantly, even though some
have adopted a common transport mechanism
 such differences make it difficult to develop generalized
flow analysis tools
 Goal
 To produce a standard method for exporting flow info
from network devices, as an eventual replacement for
the various proprietary methods in use now
Internet Traffic Monitoring and Analysis:
Methods and Applications
(12)
POSTECH
DP&NM Lab.
5. Passive Monitoring - Flow Technology: IPFIX
 IPFIX Internet Drafts
 Requirements for IP Flow Information Export
 J. Quittek et al., Jan 2003 (work in progress)
 Architecture Model for IP Flow Information Export
 K.C. Norseth, G. Sadasivan, June 2002 (work in progress)
 Early stage of work….
Internet Traffic Monitoring and Analysis:
Methods and Applications
(13)
POSTECH
DP&NM Lab.
5. Passive Monitoring - Traffic Analysis
 Spatial aspect




The patterns of traffic flow relative to the network topology
Important for proper network design and planning
Identification of bottleneck & avoidance of congestion
Example: Flow aggregation by src, dst IP address or AS number
 Temporal aspect
 The stochastic behavior of a traffic flow, usually described in
statistical terms
 Important for resource management and traffic control
 Important for traffic shaping and caching policies
 Example: Packet or byte per hour, day, week, month
 Composition of traffic
 A breakdown of traffic according to the contents, application,
packet length, flow duration
 Helps to explain its temporal and spatial characteristics
 Example: game, streaming media traffic for a week from peer ISP
Internet Traffic Monitoring and Analysis:
Methods and Applications
(14)
POSTECH
DP&NM Lab.