Transcript Diapositiva 1 - Warsaw University of Technology

DiFMon
Distributed Flow Monitor
Dario Salvi
Consorzio Interuniversitario Nazionale per l’Informatica (CINI)
Naples, Italy
o Context: Internet flow monitoring
o Possible Uses: traffic profiling, Intrusion Detection
o Contribution: development of a distributed software
for flow monitoring
Flows are defined by means of some properties appliable to packet headers
For example:
1.
IP addresses (source and destination)
2.
The 5-uple (source address, destination address, source port, destination port, and protocol)
…and by means of a timeout…
The choice of the flow definition depends on needs of the application
which uses monitoring data
A Flow Monitor should:
1. Capture packets from the network
2. Associate a flow id to each packet on the basis of the chosen definition
of flow
3. As a packet arrives, update the metrics of the flow which the packet
belongs to
4. Keep in memory the metrics related to the “living” flows (not timed out)
in data structures (flow records)
5. Save the computed metrics related to each timed out flow in order to
make them available to the applications
Proposed architecture:
Application
Application
Application
1. Calculates the metrics at
each packet arrival
1. Keeps in memory the
2. Keeps in memory the
metrics
related to timed
1.
Captures
metrics
aboutpackets
each living
out flows
flow
2. Associates flow id to
3.
“Exports”
timed
out flows
2. Informs
the
application
the packet
to
the Collector
about
some “interesting”
4. Exports
some
living flows
“interesting” living flows
Collector
Flow Cache
Flow Cache
Meter
Flow Cache
The Flow Cache:
• It is the critical module, it must look up and update a flow record as a
packet arrives (for this reason it is distributed)
• Packet multiplexing is done by means of a hash function (mmh)
computed on the flow id
• Metrics can be implemented in a flexible way through an API
• Ordering of flow records relies on the Least Recently Used algorithm (on
the basis of the last acces time)
ordering
allows otpimized
timed out
The flowLRU
record
of a just-arrived
packetsearch
will be for
positioned
within the first
flows
(starting
tail probability
of the queue
and
elements
of the
queue from
with the
a high
(temporal
locality
stopping
whentailed
a not-timed
out flow
is found)
properties,
i.e. heavy
distributions
of the
packet rates)
Some implementation details:
• Communication between the modules is done using UDP
• A flow control between modules is provided
• Programming language: C
• Operating system: Linux
• Used libraries: libpcap
• Software license: GPL
• Project location: SourceForge.net
The management Protocol:
The system must be: reliable, robust and flexible.
Some assumptions:
The collector and the
meter use defined port
numbers for signalling
messages
Collector
Flow Cache
Flow Cache
Meter
Flow Cache
Modules can run on
the same / different
machines
The network connecting
system modules must be
faster
themust
monitored
Thethan
Meter
perform
network
packet capturing within
the packet interarrival
time
– Export
3 –3CONN
Req
4 –4END
– ACK
Req
5 – ACK
Collector
1 – CONN Req
2 – END Req
2 – ACK
6 – ACK
6 – ACK
Flow Cache
Meter
Start and Stop of the system:
1 – END Req
Starting
Stopping
On defined port number
On dinamically chosen port
number
2 – ACK
1 – Exporting Data
2 – ACK
Collector
1 – Captured Data
Flow Cache
Meter
Steady state operation:
On defined port number
On dinamically chosen port
number
2 – ABORT
3 – ABORT
2 – ABORT
2 – ABORT
Flow Cache
Meter
2 – ABORT
1 – ABORT
2 – ABORT
3 – ABORT
Collector
1 – ABORT
Flow Cache
Aborting (from Flow Cache): 2 – ABORT
1 – ABORT
2 – ABORT
2 – ABORT
2 – ABORT
Flow Cache
Meter
1 – ABORT
2 – ABORT
2 – ABORT
2 – ABORT
Collector
2 – ABORT
1 – ABORT
Flow Cache
Aborting (from Meter):
2 – ABORT
2 – ABORT
1 – ABORT
1 – ABORT
Flow Cache
Meter
2 – ABORT
2 – ABORT
1 – ABORT
2 – ABORT
Collector
2 – ABORT
2 – ABORT
Flow Cache
Aborting (from Collector):
Removing
Adding
3 – CONN Req
1 – DISCONN Req
4 – ACK
2 – ACK
5 – ACK
Collector
1 – CONN Req
1 – DISCONN Req
2 – ACK
2 – ACK
6 – ACK
Flow Cache
Meter
Adding/Removing a Flow Cache:
On defined port number
On dinamically chosen port
number
2 – ACK
Flow Cache’s
crash
Collector’s
Meter’s
crash
crash
1 – ALIVE Req
2 – ACK
Collector
1 – ALIVE Req
Flow Cache
Meter
Crashes:
Conclusions / future work:
• The proposed protocol is scalable with respect to the increase in the
number of the flow caches and monitored networks.
• The system is suitable to different contexts, such as security, traffic
profiling or billing where specific metrics are of interest.
• Benchmarking and robustness evaluation will be conducted.
• The LRU sorting algorithm will be compared with other ordering
algorithms.
• We are currently working on the implementation of an intrusion
detection system and a tool for traffic profiling based on the proposed
monitoring architecture.
http://sourceforge.net/projects/difmon/
[email protected]