Transcript PPT Version

Draft-molina-flow-selection-00
Maurizio Molina, <[email protected]>
Motivation, Background (1/2)
• Flow selection (i.e. meter and/or export only a
subset of the flows) may be needed for
– Reducing on purpose exporter load & traffic
– Resource limitation
• In Flow recording process: cannot store all flow recs
• In Meter: update Flow rec is lightweight; create new
Flow rec is heavyweight  limit creation rate
• In exporter: not able to export all the Flow recs
• Resource limitation may become more evident
under attacks
© NEC Europe Ltd., 2002
Network Laboratories, Heidelberg
2
Motivation, Background (2/2)
• Flow selection is considered in current IPFIX drafts,
limited to the exporting process
– Arch draft (10.1) selection criteria of flows for export
• Attempt here:
– Consider also meter & flow recording processes
– Define which information to export about flow
selection
– Why? Fundamental for collectors to adjust their level
of trust of received information
• Presentation was given in Wien
– http://ipfix.doit.wisc.edu/IETF57/Flow_sampling_ipfix.ppt
© NEC Europe Ltd., 2002
Network Laboratories, Heidelberg
3
Metering Process
• Packets may be discarded because they would lead to the
creation of a new Flow rec
– and there’s no room in the Flow recording process
– or the Flow rec creation rate must be kept low
• There are “intelligent” methods for doing so [*][**] (not in the
scope of standardization)
• What info to export?
– (A) Macro-flow of these discarded packets (#pkts, #bytes,
timestamp first, timestamp last)
• NOTE: it’s a discarding that happens after flow classification!
[*] C. Estan and G. Varghese: New Directions in Traffic Measurement and
Accounting, ACM SIGCOMM Internet Measurement Workshop 2001, San
Francisco (CA) Nov. 2001
[**] M.Molina: A scalable and efficient methodology for flow monitoring in the
internet, International Teletraffic Congress (ITC-18), Berlin, Sep. 2003
© NEC Europe Ltd., 2002
Network Laboratories, Heidelberg
4
Flow recording process
• Existing Flow recs may be discarded without being exported
to make room for new ones
– Looks strange? Yes, but [*] proofs that can be a better choice!
• What info to export?
– (B1) Macro-flow of the non-exported packets and bytes
belonging to these discarded records (#pkts, #bytes, timestamp
first, timestamp last)
– (B2) Macro flow of these discarding events (#Frec, timestamp
first, timestamp last)
– In addition, export (C) the amount of all non-exported traffic
contained in the flow recording process (#pkts, #bytes, #Frecs
containing at least 1 non-exported pk).
– Why? (next slide will clarify…)
– No timestamps, in this case, would require keeping full state….
[*] M.Molina: A scalable and efficient methodology for flow monitoring in
the internet, International Teletraffic Congress (ITC-18), Berlin, Sep. 2003
© NEC Europe Ltd., 2002
Network Laboratories, Heidelberg
5
Exporting process (1/2)
• Flow recs may not be exported
– For explicit policies (e.g. export 1 flow out of N, or only
the flows with more than x bytes, or other smarter
strategies, e.g. [*])
– For resource limitation (buffer for packet preparation,
exporting rate control…)
• Non exported records may be
– immediately discarded
– kept in the flow recording process for later exporting
[*] N. Duffield, C. Lund, M.Thorup: Learn More, Sample Less: Control of
Volume and Variance in Network Measurement http://www.research.att.com/~duffield/pubs/DLT02-optimal.pdf
© NEC Europe Ltd., 2002
Network Laboratories, Heidelberg
6
Exporting process (2/2)
• What info to export?
– (D1) Macro-flow of the non-exported packets and bytes belonging to
non exported and discarded records (#pkts, #bytes, timestamp first,
timestamp last)
– (D2) Macro flow of these discarding events (#Frec, timestamp first,
timestamp last)
• (D1)(B1), (D2)(B2) only the discarding cause is
different!
– (E) the amount of all non-exported traffic contained in the flow recording
process that was not exported, but that still stays in the flow repository
(#pkts, #bytes, #Frecs containing at least 1 non-exported pk)
– No timestamps, in this case, would require full state….
• (E) may be difficult/impossible to obtain depending on the flow
recording reading implementation
– That’s the reason of having (C): At a collector
(C)+(B)+(D)-received traffic= (E)
© NEC Europe Ltd., 2002
Network Laboratories, Heidelberg
7
Open Issues
• Is the defined info about flow selection sufficient/redundant?
• We didn’t define so far info about the flow selection process, like
– “Exporter selects 1 out of N flows” or “Exporter exports flows with more
than x bytes only” or “meter does elephant flow selection”
• Is it feasible/worth to describe it?
• How to export flow selection information?
– It is not relative to a flow, but to the behavior of a whole metering / flow
recording / exporting process
– As protocol draft suggests,
“The Options Template Record (and its corresponding Options Data Record) is used to
supply information about the Metering Process configuration or Metering Process
specific data, rather than supplying information about IP Flows”
– Option recs contain a “scope field” which is
“The relevant portion of the Exporting Process/Metering Process to which the Options Template
Record refers. Currently defined values are: can be the interface, the cache, etc.”
– Are the currently defined scope types (System, Interface, Line Card,
Cache, Template) enough for flow sampling purposes?
© NEC Europe Ltd., 2002
Network Laboratories, Heidelberg
8