Transcript Internet2 Engineering and Abilene Update

Internet2 Engineering
and Abilene Update
Westnet Meetings :: Boulder
Guy Almes <[email protected]>
29 June 2001
Outline of Talk
A
Internet2 Engineering Objectives
A
Abilene History and Status
A
Engineering Update
• Multicast, IPv6, QoS
• Issues in End-to-End Performance
Internet2 Engineering
Objectives
A
Provide our universities with superlative
networking:
• Performance
• Functionality
• Understanding
A
Make superlative networking strategic for
university research and education
The End to End Challenge
A
Support advanced networking end to end
A
Performance
• 100 Mb/s across the country normative
• several multiples possible in some cases
A
Functionality
• Multicast
• Quality of Service
• IPv6
• Measurements
Abilene Update
Original Abilene
Objectives -- 15-Apr-98
A
A
Provide high-quality, widely available
Interconnect among participating
gigaPoPs/universities
Connect to Internet2 members via the
vBNS and to other key research/
education sites via Internet2/NGI-class
federal and non-US nets
Abilene Objectives, cont.
A
A
A
A
[Support] advanced functionality
Maximize Robustness
Minimize Latency
Provide Capacity to Avoid Congestion
Key Attributes
A
12 Router Nodes
• Cisco 12008 Routers
• Qwest collocation
A
OC48 Interior Circuits connect them
• Packet over Sonet in all cases
A
Access: 52 total
• OC3, OC12, and some OC48
• via any Qwest Sonet PoPs (Access Nodes)
• ATM and POS both supported
Abilene Partners
A
Qwest: Sonet and Collocation
A
Nortel: OC192 Sonet ADMs
A
Cisco: 12008 Routers
A
Indiana University: NOC
Abilene core
Seattle
New York
Cleveland
Indianapolis
Sunnyvale
Washington
Denver
Kansas City
Los Angeles
Atlanta
Houston
The Qwest plow laying:
- two conduits
- one with 96 fibers
- one empty
Abilene Router Nodes
being assembled
Summer 1998
Qwest
'right of way'
PoP
60 Hudson
an early carrier hotel
Abilene Connections
by (roughly) summer 2001
Current Status
A
A
About 50 Connections
Growing number of OC3-to-OC12 POS
upgrades
• e.g., NYSERnet Buffalo
• but also NoX, Univ Pennsylvania, Florida
A
185 Participants in 50 States and DC
• Puerto Rico soon
Sponsored Education Group
Participants –Other Possible
Increasing Routing
Complexity
A
Abilene International Transit Network
• with CA*net3 and StarTap
A
Abilene policy on Federal Labs
• differentiate Federal Labs from BB agencies
Abilene Traffic Matrix
Destination
Transit
Regular
Regular
Participant
Corporate
Federal
Abilene
Non-AITN
Federal
Participant Participant
Lab/BB
ITN Peer
Peer
Network
Y
Y
Y
Y
Y
Y
Participant
Y
N
Y
Y
Y
Y
Federal
Lab/BB
Y
Y
N
Y
Y
N
Y
Y
Y
Y
N
N
Y
Y
Y
N
N
N
Y
Y
N
N
N
N
Corporate
Source
Peering
Abilene
ITN Peer
Non-AITN
Peer
Federal
Network
International Peering
Seattle
CA*net3, (AARnet)
STAR TAP
APAN/TransPAC, CA*net3, IUCC,
RENATER, REUNA, SURFnet, SINET,
TAnet2 CERnet, (HARnet)
OC12
Sunnyvale
New York
DANTE*,
JANET,
NORDUnet,
SURFnet
CA*net3
(SINET)
Los Angeles
SingAREN, SINET
San Diego
CUDI
El Paso
(CUDI)
OC3-12
Miami
(REUNA, RNP2,
RETINA)
Early NoF Planning
A
A
A
Commitment to ongoing needs of the
Internet2 infrastructure beyond 2003
Leverage growing DWDM/fiber
provisioning with many 10-Gb/s ls
Needs:
• Leverage Backbone/GigaPoP/Campus structure
• Serious attention to international/federal peering
• Current advanced services now normative
The Houston Flood
A
Tropical storm Allison hit Houston hard
• 26 inches in 24 hours
A
Abilene was effected by this
• Houston Router Node went down Saturday morning
• No news until mid-day Monday
• "Technicians could not access this facility until late
this afternoon when it was determined that the
Abilene equipment there had been damaged beyond
repair."
The Houston Flood, cont.
A
Then (!), on Tuesday morning, it came up
• First, just a few circuits
• Operational for most Texas/Louisiana connectors
• Last circuit came up Thursday
A
Networking, like baseball, is a "game of
inches"
Advanced Functionality
A
Multicast
A
IPv6
A
QoS
Internet2 Multicast
A
Multicast Working Group
• Kevin Almeroth, Univ California Santa Barbara, chair
A
Encouraging more pervasive high-quality
deployment of native IP multicast
throughout the Internet2 infrastructure
A
Fighting fires
A
Keeping an eye on SSM
A
Clarifying the application story
Internet2 Multicast
Architecture
A
PIM-SparseMode
• multicast routing within an Autonomous System
• quite scalable
• notion of rendezvous points
A
MBGP
• between Autonomous Systems
A
MSDP
• Source Discovery
Longer-term WG Issues
A
A
Scalability (what happens if it does catch
on?)
Exploring the role of Source-Specific
Multicast
Could SSM be Enough?
A
'Classic' Multicast
• Group <g> has global significance
• A user creates, joins, sends to g
• Others can join, then send to and/or listen to g
• MBGP, PIM-SM, MSDP triad
A
Source Specific Multicast
• Group <g> has local significance
• A user 's' creates, sends to <s,g>
• Others can subscribe to, then list to <s,g>
• No need for MSDP (or allocation of <g> values)
Implications of SSM
A
Simplify Multicast Routing / Addressing
• No need for global class-D address allocation
• No need for source discovery
A
Complicates 'few-to-few' applications
• Define all the members of the application-level group
• Both a burden and an opportunity
A
Allows better Security, Scalability
A
Requires new version of IGMP
Multicast Summary
A
Full functionality supported now
A
Deployment steadily increasing
A
Some international peering, e.g., CA*net3
A
Performance excellent
A
Scalability?
A
Applications?
Internet2 IPv6
A
IPv6 Working Group
• Dale Finkelson, Univ Nebraska, chair
A
A
A
Build the Internet2 IPv6 infrastructure
Educate campus network engineers to
support IPv6
Explore the Motivation for IPv6 within the
Internet2 community
IPv6 Infrastructure
A
vBNS and Abilene both support IPv6
A
Abilene IPv6 with IPv6/IPv4
• Four 'backbone' nodes: Cisco 7200
"
Atlanta, Pittsburgh, Denver, and Indianapolis
• Managed by the Abilene NOC
A
IPv6 WG: address allocation and
engineering coordination
Education / Training
Goals
A
IPv6 hands-on workshop
• Lincoln, Nebraska; 17 May 2001
• starting from scratch, build an IPv6 network,
including routers, hosts, DNS tools and various
transition tools, ending up with a functional IPv6
network fully interconnected to the global Internet.
A
Materials from this workshop will be
available to enable gigaPoPs and others to
use in their own workshops.
Explore IPv6 Motivation
A
Why should our users, campus decisionmakers, and community generally care
about IPv6?
• we like Steve Deering
• IPv6 preserves the classic end-to-end transparency of
the Internet architecture
• improved support for mobility
• key for IPsec
• key for the scalability of the Internet
A
The answers must be pragmatic.
Internet2 QoS
A
Quality of Service Working Group
• Ben Teitelbaum, Internet2 staff, chair
A
QBone Premium Service
A
Scavenger Service
A
Architectural and ad-hoc projects
QBone Premium Service
A
For a given bit/second rate, minimize:
• Delay and variation in delay, and
• Loss
A
And support
• Interoperability of separately designed/managed IP
networks (e.g., Abilene, gigaPoP, ESnet, campus)
• Interoperability of different (compliant) equipment
A
This is hard and very important
Abilene Premium Service
A
Goal
• Make APS a reference implementation of QBone
A
Status
• CAR policing + uncongested ABES
A
Seven participating connectors
A
Summer 2001:
• Turn on PQ and stochastically detect illegal EF traffic
via NetFlow monitoring
A
Fall 2001:
• Engine-3-based real policing
Scavenger Service
A
Suppose there were a less-than-bestefforts IP service within Internet2?
• users can mark their packets LBE
• best-efforts traffic generally routed before LBE traffic
• what bottom-feeding applications would emerge?
• much easier than Premium Service
Other Abilene items
A
DDoS Detection Technique Development
• collaboration of Asta Networks and the NOC
• reflects open measurement/management stance
• similar relation anticipated with Arbor Networks
A
High Performance Demos
• Fall Member Meeting; Austin in October
• SC'2001; Denver in November
Issues in
End-to-End Performance
The Current Situation
A
Our universities have access to an
infrastructure of considerable capacity
• examples of 240 Mb/s flows
A
End-to-end performance varies widely
• but 40 Mb/s flows not always predictable
• users don't know what their expectations should be
A
Note the mismatch
What are our Aspirations?
A
A
Candidate Answer #1:
Switched 100BaseT + Well-provisioned
Internet2 networking ® 80 Mb/s
But user expectations and experiences
vary widely
What are our Aspirations?
A
A
Candidate Answer #2:
Lower user expectations and minimize
complaining phone calls
There is a certain appeal I suppose...
What are our Aspirations?
A
A
Candidate Answer #3:
Raise expectations, encourage aggressive
use, deliver on performance/functionality
to key constituencies.
Not the easy way, but necessary for
success
Why should we Care?
A
"We" as the university community.
A
"We" as campus networking specialists.
A
"We" as networking professionals.
A
"We" as the (broad) Internet2 project.
A
Low aspirations are dangerous to us.
End to End Performance
Initiative
A
Goal:
• To create a ubiquitous, predictable, and well-
supported environment in which Internet2 campus
network users have routinely successful experiences
in their development and use of advanced Internet
applications, by focusing resources and efforts on
improving performance problem detection and
resolution throughout campus, regional, and national
networking infrastructures.
Threats to
End to End Performance
A
A
BW = C x packet-size / ( delay x sqrt(packet-loss ))
(Mathis, Semke, Mahdavi, and Ott, CCR, July 1997)
Context:
•
Network capacity
•
Geographical distance
•
Aggressive application
Threats to
End to End Performance
A
Fiber problems
• dirty fiber
• dim lighting
• 'not quite right' connectors
Threats to
End to End Performance
A Fiber problems
A
Switches
• horsepower
• full vs half-duplex
• head-of-line blocking
Threats to
End to End Performance
A Fiber problems
A Switches
A
Inadvertently stingy provisioning
• mostly communication
• happens also in international settings
Threats to
End to End Performance
A Fiber problems
A Switches
A Inadvertently stingy provisioning
A
Wrong Routing
• asymmetric
• best use of Internet2
• distance
Threats to
End to End Performance
A Fiber problems
A Switches
A Inadvertently stingy provisioning
A Wrong Routing
A
Host issues
• NIC
• OS / TCP stack
• CPU
Perverse Result
A
A
'Users' think the network is congested or
that the Internet2 infrastructure cannot
help them
'Planners' think the network is
underutilized, no further investment
needed, or that users don't need high
performance networks
Promising Approaches
A
Work with key motivated users
A
'Shining a flashlight' on the problem
A
Measurements
A
Divide-and-Conquer
A
Understanding Application Behavior
A
Getting it right the first time
Internet2 End-to-End
Performance Initiative
A
Very recently hired / deployed staff
• Cheryl Munn-Fremon, initiative director
• Russ Hobby, chief technical architect
• George Brett, chief information architect
A
$1.5M budgeted by Internet2
Internet2 End-to-End
Performance Initiative
A
Distributed measurement infrastructure
• Enable rapid effective understanding of why an
instance of end-to-end performance is limited
• Make the work of PERT members rewarding
• Enable initiation of tests by PERT members
A Teams of performance analysis specialists (PERTs)
A Dissemination of best practices
Internet2 End-to-End
Performance Initiative
A Distributed measurement infrastructure
A
Teams of performance analysis specialists
(PERTs)
• members at campuses, gigaPoPs, backbones
• socially and technically coordinated
• committed to effecting radical change
A Dissemination of best practices
Internet2 End-to-End
Performance Initiative
A Distributed measurement infrastructure
A Teams of performance analysis specialists (PERTs)
A
Dissemination of best practices
• Identify key techniques, tools, and 'best practices'
• Make them common
• Work toward widespread / routine excellent user
experiences
• Improve the reputation / status of network engineers
Anticipated Partners
A
NLANR: DAST, MOAT, and NCNE
A
Web100 Project
A
Abilene partners
A
Leading campuses and gigaPoPs
A
Internet2 corporate members
Internet2 Measurements
A
Measurement Working Group
• Matt Zekauskas, Internet2 Staff
A
Define architecture:
• Usage
• Active Measurements of Performance
• Passive Measurements
A
A
Uniform Access to Results
Contributing to Measurement
Infrastructure for the E2EPerf
Applications for
Measurements
A
A
A
A
A
End-to-end Performance Debugging
Verification of QoS Performance
Characteristics
Support for Operations
Forward engineering of new
infrastructure
Supporting research, e.g., by university
computer scientists
Active Measurements
within Abilene
Surveyors with:
Active delay/loss measurements
Ad hoc throughput tests
Application to
Performance Debugging
Application to
Performance Debugging
Divide and Conquer
A
A
Systematically identify/isolate the
network segment at fault
Can we make this systematic and
(eventually) automated?
Access to Key Resources
A
Optical telescopes in Hawaii
A
CRAFT Project
A
PACI Supercomputer Facilities
A
CERN
Working Groups as
Opportunities
A
We intend the WGs to be effective as:
• means for interested engineers to 'sink their teeth
into' hard Internet2 engineering problems
• means for disseminating best practices etc to the
Internet2 membership
A
New Engineering Area of Internet2 web
site due up by 14-Feb-01
Internet2
and Stephen F Austin
A
Can we defeat distance as a barrier to:
• human collaboration?
• effective access to key instruments / data sources?
A
For very large research universities, this
is somewhat important, but it is key for
smaller ones!
Applications Communities
A
General notion: distributed sets of
researchers who collaborate at a distance
• High Energy Physics (CERN, MIT, Caltech)
• Space Physics & Aeronomy Research Collaboratory
• Geospatial Information Systems community
A
These groups explore why advanced
Internet2 infrastructure is important