Transcript Consortium of International Research and Education Networks

I
N
D
IRNC Measurement
I
A
N
A
U
N
I
John Hicks
HPCC Engineer
Indiana University
V
E
R
S
I
T
Y
18th APAN Meeting – Cairns 4-July-2004
I
N
D
Overview
I
A
N
A
U
N
I
V
E
R
S
I
T
Y
• Global Federated Measurement
Infrastructure
• Similarities with the Abilene
Observatory
• Tools
• NLANR MNA
• Work with APAN JAPAN
I
N
D
Global Federated Measurement Infrastructure
I
A
N
A
U
N
I
V
E
R
S
I
T
Y
• The Consortium of International Research and
Education Networks (CIREN) is name of Indiana
University’s response to the NSF’s International
Research Network Connections (IRNC) program
solicitation.
• CIREN’s goal is to connect the Asian Pacific
Advanced Network (APAN) in Asia, the GEANT
network in Europe and the CLARA network in
Latin America to the US high performance
infrastructure (Abilene and the various Federal
networks).
• CIREN US node locations include the East and
West coast as well as the Midwest.
I
N
D
Measurement Infrastructure
I
A
N
A
U
N
I
V
E
R
S
I
T
Y
• An important part of CIREN is to develop a
Global Federated Measurement Infrastructure.
• In order to effectively monitor the performance
and utilization of a global high performance
networks, we need to bring together existing and
emerging measurement and analysis tools and
frameworks.
• This environment needs to provide measurement
data to network engineers and researchers.
• The data needs to be stored and published in
standard data formats agreed upon by the
measurement community.
• We will start with the schemas developed by the
GGF NMWG.
I
N
D
Measurement Infrastructure (Cont.)
I
A
N
A
U
N
I
V
E
R
S
I
T
Y
• Data will be collected and published through the
Global NOC at Indiana University and databases
will be replicated at strategic location to provide
timely access worldwide.
• Through a publicly available web interface, we
would like to provide a “one stop shop” for all
your measurement needs.
• The idea is to provide a facility that will let
network engineers and researchers visit one
location to decompose a network path, segment
by segment, in order to determine performance
characteristics and help debug network problems.
I
N
D
Measurement Infrastructure (Cont.)
I
A
N
A
U
N
I
V
E
R
S
I
T
Y
• We will deploy hardware (similar to the
Abilene Observatory hardware) to strategic
location on the network and work closely
with Internet2, APAN, GEANT, and
CLARA to coordinate measurement
efforts.
• We are already working with Internet2 and
APAN to accomplish this goal.
• Internet2 is also working with GEANT.
I
N
D
Similarities with the Abilene Observatory
I
A
N
A
U
N
I
V
E
R
S
I
T
Y
• Like the Abilene Observatory, this program
supports the collection and dissemination of
network data. It provides Network Engineers with
an operational view of the network.
• It also provides researchers a platform to conduct
experiments and collect data from a HighPerformance network.
• Rack space, power, and connectivity will be
provided to network researchers wanted access to
high performance international networks.
• Data from these projects must be made publicly
available in some form.
I
N
D
Co-locating a measurement project
I
A
N
A
U
N
I
V
E
R
S
I
T
Y
• This project will reserve space for researchers to
deploy equipment in the CIREN Rack.
• Formal procedures will be announced when
appropriate.
• Requirements are similar to the Abilene
Observatory:
– Description of the Project including
participants and duration
– Space, Network Power Requirements
– System Information
– Security
I
N
Co-locating Caveats
D
I
A
N
A
U
N
I
V
E
R
S
I
T
Y
•
•
•
•
•
•
Commodity Routes will not available
48V DC Power (most places)
23” Racks
Address Space
Security
“Lights Out” Remote Operation
I
N
D
Measurement Racks
I
A
N
A
U
N
I
V
E
R
S
I
T
Y
• In each CIREN node there is a one rack dedicated
to the measurement project.
The rack is dedicated for:
• NOC Administrated measurement Machines
• Co-located Machines
• DC Power Controllers for Observatory machines
No Routing equipment is in the Rack
I
N
D
I
A
N
A
U
N
I
V
E
R
S
I
T
Y
Typical configuration (front)
I
N
D
I
A
N
A
U
N
I
V
E
R
S
I
T
Y
Typical configuration (back)
I
N
D
Advanced services
I
A
N
A
U
N
I
V
E
R
S
I
T
Y
• Since the Machines are directly connected to the
Backbone there are many advantages to test
“Advanced” Services
• Native V6
• Native V4 and V6 Multicast
• 1Gb ports directly connected to the backbone
• 9000 MTU
• Alternative TCP project testing, such as FAST
TCP and Highspeed TCP, will also be supported.
I
N
D
Monitoring
I
A
N
A
U
N
I
V
E
R
S
I
T
Y
• The state of the machines will be closely
monitored.
• Nagios/AlertMon monitor and display
alerts if any machine or service is down
• Ganglia Cluster Toolkit is used for system
(load,mem,disk usage, etc..) monitoring
I
N
Tools
D
I
A
N
A
U
N
I
V
E
R
S
I
T
Y
•
•
•
•
•
•
•
•
Netflow
Owamp (One-Way Latency)
BWCTL
SNMP Interface Statistics
Multicast Beacon
Ping/Traceroute V6 Destination
NLANR AMP
NLANR PMA
I
N
D
Netflow
I
A
N
A
U
• Sampled (100:1) Netflow will be sent from all
CIREN Routers to one of the local measurement
machines. The flows are sent to researchers and
also cashed locally and retrieved to a central
storage using rsync.
N
I
V
E
R
S
I
T
Y
• The Netflow records will be anatomized by
masking the low-order 11 bits of the IP address.
Unanatomized data is not stored.
I
N
D
Netflow Data
I
A
N
A
U
N
I
V
E
R
S
I
T
Y
• Netflow Data will be available ether as a
direct feed from the measurement
Machines or as a download from the
centralized storage area at the Global NOC
• More information will be available about
retrieving netflow data when appropriate
I
N
D
Owamp
I
A
N
A
U
N
I
V
E
R
S
I
T
Y
• One-Way Latency Measurements using a mesh of
nodes in each CIREN Node.
• Owamp tests one-way latency between machines
using both IPv4 and IOv6. The measurement
machines will be synchronized with Praecis Ct
CDMA Clocks connected to each machine.
• Data is collected from each machine and
processed in a centralized database.
I
N
D
BWCTL
I
A
N
A
U
N
I
V
E
R
S
I
T
Y
• BWCTL is a command line client
application and a scheduling and policy
daemon that wraps Iperf.
• Supports (and requires) versions 1.7.0 or
2.0.b of Iperf.
• Full IPv6 support.
• Supports dynamic TCP window size
determination.
I
N
D
SNMP Interface Statistics
I
A
N
A
U
N
I
V
E
R
S
I
T
Y
• SNMP Collection will be done in a distributed
way on the measurement machines. Data for local
routers and switches are captured and copied back
to a central repository.
• The data collected is a Hi-Resolution (10 sec)
capture of interface and environmental statistics
• In addition a SNMP router proxy similar to the
Abilene router proxy is in the works to allow
access to query SNMP variables on the router
I
N
D
Multicast Beacon
I
A
N
A
U
N
I
V
E
R
S
I
T
Y
• Mesh of all CIREN Router Nodes, running
on measurement machines
• Modified version of NLANR Multicast
Beacon
• Saves data into RRD Database
• Graphs of Delay, Loss and Jitter statistics
• Multicast Group 233.1.2.3
I
N
D
NLANR AMP
I
A
N
A
U
N
I
V
E
R
S
I
T
Y
• The Active Measurement Project (AMP) has a joint
research/engineering focus with site-to-site active
measurements and analyses conducted between locations
connected by high performance networks.
• The AMP monitors measure the "heartbeat" of the high
performance connection (HPC) networks, dissecting the
networks in over 15,000 ways every minute.
• The performance of AMP monitors are well known and we
would like to work with our international partners to extend
the deployment of AMP machines.
I
N
D
NLANR PMA
I
A
N
A
U
N
I
V
• The goal of the PMA project is to deliver new insights into
the operation, behavior, and health of the Internet, for the
benefit of network users and operations.
• Passive header trace data provides the means to study
workload profiles for a number of strategically located
measurement points in high speed environments.
• Passive monitors are available up to OC192 but they are
prohibitively expensive.
E
R
S
I
T
Y
• We will support the PMA project by provide an environment
to deploy passive monitors.
I
N
D
Work with APAN JAPAN
I
A
N
A
U
N
I
V
E
R
S
I
T
Y
• We are currently working with the APAN-JP NOC
to establish a measurement environment across
TransPAC.
• The measurement effort will be consistent with
that of the Abilene Observatory (i.e. PiPEs,
including bwctl and owamp and other codes).
• Other measurement projects, discussed in a later
talk, will also be deployed.
• This will provide valuable measurement data for
network engineers and researcher using APAN,
TransPAC, and Abilene.
I
N
D
Work with APAN JAPAN (Cont.)
I
A
N
A
• Strong relationships are required to make
this global measurement environment
successful.
U
N
I
V
E
R
S
I
T
Y
• A special thanks goes to the APAN-JP
NOC for their support and eagerness to
move the development and deployment
forward.
I
N
D
Thank you
I
A
N
A
John Hicks
U
N
I
V
E
R
S
I
T
Y
Indiana University
[email protected]