Transcript Mort`s slides

Ubiquitous Instrumentation
Richard Mortier
MSR Cambridge
[email protected]
Measuring networks
• Active
– traceroute/ping/etc (ICMP)
– SNMP MIBs
– Non-critical functionality  buggy
• Passive
– NetFlow, IPFIX, sFlow
– Port spanning, VLAN spanning
– Router/switch only  poor visibility, scalability
Claims
1. Routers are just specialized hosts
•
Hardware, protocols, configuration
2. Hosts are part of the network as well
•
Transmit, receive, forward data
Claims
1.
Routers are just specialized hosts
•
2.
Hardware, protocols, configuration
Hosts are part of the network as well
•
•
Transmit, receive, forward data
The strong distinction that has grown between them
makes understanding network behaviour difficult
–
–
•
Traffic is opaque to routers (and becoming more so)
Network is black box service to hosts (and becoming more so)
Hosts provide inputs
–
•
…we should use them to understand demands
Routers provide resource
–
…their configuration implements constraints
Networking measurements
• Host-based measurement – Anemone
– Instrumented stack, kernel structures
– Per-EXE per-packet live network stats
– Unify flows with routing topology
• Distributed query system – Seaweed
– Access to Anemone flow/packet tables
– Built over Pastry, highly scalable (106 nodes+)
Measure for measure
• Ubiquitous network measurement
– Undercarriage of the Knowledge Plane
– Infrastructure for autonomic-foo?
• Concretely…
– Combining host & network data – SeaStar
• Single trust domain – how to expand?
– User-visible diagnosis – Constellation
• What do users want to know about the network?
Measure for measure
• Data gathering and management
– Existing tools primitive: tend toward manual analysis
– Need performant, type-safe, distributed processing
– Theoretical aspects of temporal, spatial sampling
• Unifying many different datasets
– Routing and configuration data
– Pcap and netflow and SNMP and … data
• Mining structure, relationships from data
– Machine learning techniques seem a very rich vein
– Robust automated processing techniques
– Distributed trust, provenance, privacy