Transcript Title Slide First Line Title Slide Second Line

perfSONAR Architecture: Design,
Usage, Extension and Next Steps
Presented by Prof. Martin Swany
University of Delaware / Internet2
05 August, 2008
26th APAN Conference
Queenstown, New Zealand
What is perfSONAR
• A collaboration
• Production network operators focused on designing and
building tools that they will deploy and use on their networks
to provide monitoring and diagnostic capabilities to
themselves and their user communities.
• An architecture & a set of protocols
• Web Services Architecture
• Protocols based on the Open Grid Forum Network
Measurement Working Group Schemata
• Several interoperable software implementations
• Java, Perl, Python…
• A Deployed Measurement infrastructure
perfSONAR Architecture
• Interoperable network measurement middleware (SOA):
•
•
•
•
Modular
Web services-based
Decentralized
Locally controlled
• Integrates:
• Network measurement tools and data archives
• Data manipulation
• Information Services
• Discovery
• Topology
• Authentication and authorization
• Based on:
• Open Grid Forum Network Measurement Working Group schema
• Currently attempting to formalize specification of perfSONAR
protocols in a new OGF WG (NMC)
• Network topology description being defined in the Network Markup
Language WG
Decouple 3 phases of a Measurement
Infrastructure
Analysis &
Visualization
Analysis &
Visualization
API
Measurement
Infrastructure
Measurement
Infrastructure
API
Data
Collection
Performance
Tools
perfSONAR Services
• Measurement Point (MP) Service
• Enables the initiation of performance tests
• Measurement Archive (MA) Service
• Stores and publishes performance monitoring results
• Transformation Service
• Transform the data (aggregation, concatenation, correlation,
translation, etc)
• Resource protector
• Arbitrate the consumption of limited resources
• Other services delegate a limited portion of the authorization
decision here
These services are specifically concerned with the job
of network performance measurement and analysis
(perfSONAR) Information Services
• Lookup Service
• Allows the client to discover the existing services and other LS
services.
• Dynamic: services registration themselves to the LS and mention their
capabilities, they can also leave or be removed if a service goes down.
• Topology Service
• Make the network topology information available to the framework.
• Find the closest MP, provide topology information for visualisation tools
• Authentication Service
• Based on Existing efforts: Internet2 MAT, GN2-JRA5
• Authentication & Authorization functionality for the framework
• Users can have several roles, the authorization is done based on the
user role.
• Trust relationship between networks
These services are the infrastructure concerned with
discovering federating the available network services
6
Example perfSonar client interaction
gLS
Where can I get more about network
Doman B/IP d,e,f and Domain A/IP a,b,c?
Useful graph
LS A, LS B
Where is link utilization for – IPs a,b,c?
a,b,c : Network A, MA A
Get Link utilization a,b,c
Here you go
LS A
a
MA A
Client
Get utilization
link utilization
d,e,f
Where is link
for
Here you go IPs
d,e,f?
d,e,f : Network B, MA B
LS B
b
e
c
Network A
MA B
d
Network B
f
perfSONAR works E2E when All Networks Participate
Many collaborations are
inherently multi-domain, so
for an end-to-end
monitoring tool to work
everyone must participate
in the monitoring
infrastructure
user
performance GUI
m1
m1
m4
Analysis tool
measurement
archive
measurement
archive
measurement
archive
m4
m1
m4
measurement
archive
m3
m3
m3
m1
FNAL (AS3152)
[US]
measurement
archive
m1
m3
m4
GEANT (AS20965)
[Europe]
m3
ESnet (AS293)
[US]
m4
DESY (AS1754)
[Germany]
DFN
8 (AS680)
[Germany]
perfSONAR Architecture Overview
• perfSONAR schema and protocol
• Understanding the structure of the schema is informative
when discussing the architecture and extensibility
mechanisms
• Information Services
• The Lookup Service and Topology Service utilize the
base model and provide system “glue” that allows for
service discovery and contextualization of both data and
services
• Extensibility and ongoing work
•
•
•
•
perfSONAR Schema
Key Goals: Extensibility, Normalization,
Readability
Break representation of performance
measurements down into basic elements
Data and Metadata
Measurement Data
• A set of of measurement events that have
some value or values at a particular time
• Measurement Metadata
• The details about the set of measurement data
Schema Normalization
• Can simplify the database representation for
many types of measurement data
• While optimizations are possible, many
measurement types can be viewed as one value
measured over time
• Assists Combination/Concatenation of
metrics
• Creating derived metrics
• Normalization helps with inferring
relationships between types of metrics
Schema Basic Elements - Metadata
• Subject (Noun)
• The measured/tested entity
• EventType (Verb)
• What type of measurement, value, or event
occurred
• Characteristic, tool output, or generic event
• Parameters (Adjectives and Adverbs)
• How, or under what conditions, did this
event occur?
Schema Basic Elements - Data
• Some sort of value - Datum
• Existence of an event might point to the
case where there no additional value
• As in “Link up/down” or threshold events
• Time
• Is extensible since various representations
are appropriate in different cases
• E.g. UNIX timestamp vs NTP time
A Message
Message
Message
Metadata
Data
An Object Store
Store
Metadata
Data
A Data is Linked to a Metadata
Metadata
<id>someId</id>
Data
<metadataIdRef>
someId
</metadataIdRef>
A Metadata may be linked to another
Metadata
<id>someId</id>
Metadata
<id>someOtherId</id>
<metadataIdRef>
someId
</metadataIdRef>
Schema Namespaces
• All measurements have some sort of Data
and Time
• All measurements can be described by the
Metadata identifying who, what and how
• The specific structures of the Data and
Metadata elements depend on the
measurement
• Approach: Consistently use Data and
Metadata elements and vary the namespaces
of the specific elements
Schema Namespaces - 2
• We encode the measurement/event type in
the namespace
• And as a standalone element
• Some components of the system can pass
Data and Metadata elements through without
understanding their specific structure
• Allows and implementation to decide whether
it supports a particular type of data or not
• Allows validation based on extended
(namespace-specific) schemata
Schema Namespaces and Extensibility
• One key to extensibility is the use of
hierarchy with delegation
• Similar to OIDs in the IETF management
world
• The NM-WG has a hierarchy of network
characteristics
• Good starting point
• However, not all tools are cleanly
mapped onto the Characteristic space
• Often a matter of some debate
Schema Namespaces and Extensibility - 2
• Organization-rooted tools namespace
addresses this
• Some top-level tools
• ping, traceroute
• Easy to add new tools in organizationspecific namespaces
• Performance Event Repository
• Add a schema and get a URI
• Add Java classes
Linking Metadata
• Metadata can be linked in series in two ways
• Merge chaining allows for elements to be reused
and a complete metadata can be built
• Operation chaining requests or describes
operations on data sets
A
A
A
B
B
B(A)
B
Operation Metadata
• Functions applied to the data have URIbased names
• Common parameters
• Ideally, a series of these metadata elements
can completely describe the provenance of
any resultant dataset
• As well as requesting selection and reduction
operations at query time
Topology Schema
• Topology schema grew from network measurement
description
• Reusable “Subject” elements for common cases
• Also reduces redundancy
• Relationships between measurement Subjects
• Same basic structure at all layers
• Networks are graphs
• Define:
•
•
•
•
•
•
•
Node
Port (Interface)
Link
Domain
Network
Path
Service
Topology
Topology - Recursive Links
Topology Schema
• Structured by layers and the same elements recurring
there
• Varied by namespaces
• Reuse visualization logic, etc.
• Validate layer- or technology-specific attributes
• 4 Layers: Base (both abstract and L1), L2, L3, L4
• Also technology-specific layers like Ethernet,
SONET/SDH
Relationships between Subjects
• To completely capture the relationships, we need to
do a few more things
• Recursive definition of links
• Logical links consist of physical links
• Ordered lists of links - Path
• Like above, but we need to introduce an Index attribute
• Networks
• Physically consist of links but that is not always the most
convenient logical view
• Special element to which Interfaces or Links belong
Relationships between Subjects
• Elements at the same layer have relationships
• A link references two ports/interfaces
• At Layer2 or Layer3
• Elements of the same sort have relationships between
themselves at different layers
• A Layer 1 Interface (physical NIC) can have one or more Layer 2
Interfaces, which can each have one or more Layer 3 Interfaces
• Node is special
• Since a Node doesn’t really have any higher-layer characteristic
independent of its Interfaces
Schema - Network Element Identifiers
• A scheme for identifying network
elements
• Each network element gets a unique
identifier
• This identifier will be included with any
measurement associated with that
element.
Network Element Identifiers
• Use Cases:
• A topology service can be used to find the
identifier for a network element
• An LS could then be queried to find all
measurements associated with that element
• Dynamic service path-finding can be
integrated with ongoing measurements
Network Element Identifiers
• Identifiers use URN notation
• Prefixed with “urn:ogf:network:”
• Consists of name/value pairs separated by
colons
• Possible field names: domain, node, port,
link, path, network
• Set of rules defined for each field to keep
identifiers compact and finite
• Referred to as NURNs
Network Element Identifiers
• Examples
•
•
•
•
•
•
•
urn:ogf:network:domain=Internet2.edu
urn:ogf:network:domain=internet2.edu:node=packrat
urn:ogf:network:domain=internet2.edu:node=rtr.seat:port=so-2%2F1%2F0.16
urn:ogf:network:domain=internet2.edu:node=rtr.seat:port=198.32.8.200
urn:ogf:network:domain=Internet2.edu:node=packrat:port=eth0:link=1
urn:ogf:network:domain=internet2.edu:link=WASH to ATLA OC192
urn:ogf:network:path=anna-11537-176
perfSONAR Protocol
• Utilizes the basic message container
• With type attribute
• Contains various data and metadata
elements
• Uses a message-level parameter element
to communicate message-level options
• Messages return a result datum with a type
system identical to that of a measurement
datum
perfSONAR Information Services
• The Lookup Service and the Topology
Service comprise the perfSONAR IS
• Topology and Service Lookup information
are linked with references and NURNs
• Not necessarily a single instance of each
information service
• These services are being utilized outside the
measurement world
perfSONAR Lookup Service
• Directory service of perfSONAR deployments
• Accept service registrations
• Handles queries for service location and capabilities
and location of available data
• Manage the lifetimes of data and services to keep
information up to date
• Web Service interface to XML Database
• Sleepycat/eXist XML Databases
• Service Info/Data kept in native formats
• Draw away complex query tasks from otherwise
'busy' services
Lookup Service
• Also use an XML based configuration/protocol
• Native storage/query mechanisms [Xpath/XQuery]
• Uses LS-specific messages
• Targeted at single domain deployment
• Single instance to manage multiple services
• Client components and applications use the LS
to find services
perfSONAR Topology Service
• Provides a queryable repository for obtaining
topology information about a domain
• Can obtain the entire network
• Xquery interface allows the construction of
complex queries about the network
• Topology is specified according to the topology
schema
• Various Uses
• as a “Subject” repository for measurement
information
• Topology discovery for dynamic circuit pathfinding
Distributed/Global Lookup Service
• Federation of individual LS instances into a global
system
• “Meta”-lookup phase allows a query to find the
specific LS that has relevant information
• Or perhaps the relevant LSes that have said info
• The specific query is sent directly to the LS in
question
Distributed Lookup Service
• Service and measurement metadata is
“summarized” for propagation to distant
domains
• IP addresses in service and measurement
metadata are compressed into network/netmask
pairs in the same way that routes are advertised
(CIDR-style)
• These summarized metadata elements are
advertised to external “scopes”
• A “scope” is a set of LSes that are related by e.g.
being in the same administrative domain (although
multiple scopes within a single domain are
possible)
Global Lookup Service
• Currently deployed solution involves a static
set of “root” Lookup Services
• This works very much like DNS with peer to
peer information propagation
• Summarization is based on the stored
eventTypes and serviceTypes
Using perfSONAR
• How do I share my data using perfSONAR?
• perfSONAR MAs can publish data gathered with MRTG, Cacti
• http://wiki.perfsonar.net/jra1-wiki/perfSONAR_Getting_Started
• [email protected]
• Download one or more distributions
• MDM Release from GEANT2
• Most services in Java
• Available as RPMs
• perfSONAR-PS
• Services in Perl
• Available as RPMs, via CPAN, Knoppix “live cd”
• Download various services “a la carte”
• Register with the Global Lookup Service…
perfSONAR Services
• Measurement Archive (MA) Services
•
•
•
•
•
Both RRD and SQL, java and perl
FlowSA MA (java)
HADES MA (perl)
Layer2 Circuit Status MA (java and perl)
perfSONAR-BUOY
• Measurement Point (MP) Services
•
•
•
•
•
•
CLI MP
SNMP MP
Telnet/SSH MP
Flow Subscription
BWCTL
E2EMon
• Lookup/Topology/Authentication Services
Extending perfSONAR
• How can I extend perfSONAR?
• Definition of metric schema
• If you are publishing a new type of data, schema
definition is the first step
• Reuse or re-implement protocol processing
• Examples in Java and Perl
• Register with the Lookup Service
• Defining a new service type
• Analysis modules are extended by assigning a URI,
defining parameters
Ongoing Work
• Deployment and refinement of the global LS
• Recently funded by the US National Science
Foundation
• Deployment of (“live CD”) appliances
• Initially to support LHC networking
• Integration with dynamic circuit networks like
Internet2’s DCN, GEANT2’s AutoBAHN and
ESnet’s SDN
• Leverage common components for dynamic,
visible network
Joining perfSONAR
•
•
•
•
•
How can I join this effort?
Join the mailing lists at perfsonar.net
Participate in OGF working groups
Participate in Internet2 working groups
Let us know that you are interested!
Acknowledgments
• US National Science Foundation, OCI-0721902
• Collaborators:
•ARNES
•BELNET
•CARNET
•CESNET
•CYNET
•DANTE
•DFN
•ESnet
•FCCN
•Fermilab
•GARR
•GEANT
•Georgia Institute of
Technology
•GRNET
•HEAnet
•Indiana University
•Internet2
•ISTF
•POZNAN
•RedIRIS
•Renater
•RNP
•SLAC
•SURFnet
•SWITCH
•UNINETT
•University of Delaware
end
• Questions?
• Visit www.perfsonar.net