Transcript 20060614-cdw-nettworkdesign-rrs

Internet2 Network Design
Rick Summerhill, Director
Network Research, Architecture, and Technologies
Internet2
Community Design Workshop
15, 16 June 2006
Indianapolis, IN
Contents
•
•
•
•
•
•
•
•
•
•
2
Strategic Objectives
Background
Requirements
Architecture
Network Usage Policy
Engineering
Network Research
Advisory Groups
Partnerships
Agreement Summary
Community Design Workshop – June 15-16, 2006
Rick Summerhill
Strategic Objectives
Strategic Objectives
• Ensure community control of the underlying network infrastructure
• Control those elements that allow development of new network
capabilities
• Leverage the capabilities of a global telecommunications leader
• Providing carrier class reliability and expanded breadth of services,
along with a broad set of partnership options
• Capitalize on the latest technological advancements in networking
• Create an asset that benefits the entire community - researchers,
universities, regional optical networks, industry, government, K12, and the international community
3
Community Design Workshop – June 15-16, 2006
Rick Summerhill
Background
Background - I
• A defining goal of Internet2 is to support the US
research universities’ need for scalable, sustainable,
high-performance networking
• In 1998, the Abilene network was created using 2.5
Gbps SONET circuits as part of a partnership with
Qwest Communications, Cisco Systems, and Nortel
Networks
• The original agreement with Qwest was later extended
to October of 2007 and the Abilene network was
upgraded to a 10 Gbps network
• A further extension for one year was possible if notification
was made by March 31, 2006, an extension that was recently
declined
4
Community Design Workshop – June 15-16, 2006
Rick Summerhill
Background
Background - II
• Abilene is an IP packet based network similar in
design to the original NSFnet, but with much greater
capabilities and bandwidths
• Today there is great Interest in Hybrid networks
• Greater availability of circuit based capabilities
• Greater interest world wide to bring circuit based services
closer to the edge
• Most RONs now have developed this capability already
• The HOPI project has experimented with the basic ideas for
the last two years
5
Community Design Workshop – June 15-16, 2006
Rick Summerhill
Background
Process – I
• With pending end of MoU with Qwest,
Internet2 began complete examination of all
the possibilities
• Think out of the box - look at new ideas
• Consider different network options - IP, hybrid
networks, etc.
• Considered a variety of different carrier and
existing fiber options. For example,
• How does one share wavelengths on an existing
system?
• Several viable options were available
6
Community Design Workshop – June 15-16, 2006
Rick Summerhill
Background
Process – II
• In the end, one option stood out - the
Dedicated Wave System (DWS) from Level3
• A Hybrid Network, not just an IP network and
waves
• Supports a wide variety of services and
capabilities, from highly experimental to production
level
• Control at layer 1
• Uses equipment and fiber dedicated to Internet2
• Internet2 board has approved the DWS for the
Internet2 network and the contract has been
executed.
7
Community Design Workshop – June 15-16, 2006
Rick Summerhill
Requirements
Requirements
• Throughout the process, the requirements of the
community were considered - the process did not
occur in a vacuum
• Requirements documents produced during the spring
of 2005 were considered
• The Group A report, produced by a joint engineering
committee from NLR and describing a wide ranging set of
network attributes
• The Abilene Technical Advisory Committee (TAC) Report
outlining a need to provide for new experimental services
along with production services like IP
• The MORPHnet document (1997) - Multi-Modal
Organizational Research and Production Heterogeneous
Network
8
Community Design Workshop – June 15-16, 2006
Rick Summerhill
Requirements
Requirements – II
• Input from the Quilt Optical workshops and discussions
with the RONs
• Research examining a wide variety of different types of
capabilities
• DRAGON and CHEETAH - NSF funded projects examining
dynamic provisioning
• Science related projects such as UltraLight and eVLBI
• OptIPuter and TeraGrid
• HOPI Testbed
• Input from the International community
• GEANT2 providing hybrid types of capabilities
• GLIF and lightpath capabilities for scientific work
9
Community Design Workshop – June 15-16, 2006
Rick Summerhill
Requirements
Process Revisited
• The actual process of negotiating with Level3 was done by a
relatively small number of Internet2 staff
• The negotiations could not have been done with a large community
group
• The process created a framework for the Internet2 network and
enabling community involvement
• The process used the knowledge gained from the above
requirements documents and discussions, and was done in the
spirit of developing a network to serve the community by providing
a wide range of capabilities for advancing the state of the art of
networking
• Within the above framework, there are many details yet to be
understood and ideas to be developed
• That is the purpose of this meeting, starting the process and …
• Leading to network deployment, future planning, and continuing
community involvement
10
Community Design Workshop – June 15-16, 2006
Rick Summerhill
Architecture
Architecture Design Goals – I
• Develop an innovative optical system on a national
footprint to serve the broad research and education
community
• The goal is to support all Internet2 members and to enhance
the ability to serve a wider community
• Develop a hybrid network capable of providing pointto-point services together with an IP network
• Every connector (RON) connects a NewNet backbone
ring across a metro location not requiring extensive
backhaul
• The community retains complete control of the layer 1
optical system including provisioning and switching of
wavelengths
11
Community Design Workshop – June 15-16, 2006
Rick Summerhill
Architecture
Architecture Design Goals – II
• Internet2 does not have to concentrate on
reliability and sparing - Level 3 is responsible
for an SLA.
• The community focus is on networking, not on
managing devices like amplifiers
• The system is capable of supporting network
research in wide variety ways
• Minimal Conditions of Use (CoU), allowing full
participation from the entire community in
providing new services and capabilities
12
Community Design Workshop – June 15-16, 2006
Rick Summerhill
Architecture
The System - I
• System uses dedicated fiber from the carrier
• Significant financial advantage if owned (via IRU)
footprint is used
• Built on Infinera platform providing innovative
optical technology
• Simple and convenient add/drop technology
• Simple and convenient wave setup
• Demonstrated high reliability in initial period of
operation on the Level3 network
• Economics of Infinera system are disruptive in the
market place
13
Community Design Workshop – June 15-16, 2006
Rick Summerhill
Architecture
The System - II
• Architecture has maximum flexibility. Every
RON can access every wave on the system if
needed
• System includes grooming capabilities lightpaths can be built over Ethernet or SONET
• Can take advantage of advanced SONET
cpabilities like GFP, VCAT, and LCAS
• Capable of lightpath provisioning to the campus
14
Community Design Workshop – June 15-16, 2006
Rick Summerhill
Architecture
Lightpath Capabilities
15
Community Design Workshop – June 15-16, 2006 Rick Summerhill
Rick Summerhill
Architecture
System - III
• RON connectivity expected to evolve to 2 x 10
Gbps connections
• 10 Gbps IP connection
• 10 Gbps point-to-point connection (capable of STS1 granularity lightpaths provisioned in seconds),
most likely provision using Ethernet (GFP based)
• Hybrid capabilities
• Expect 20 - 24 connectors
• Simple and consistent connection scheme
• Promoting aggregation
• Need input and discussion on exceptional cases
16
Community Design Workshop – June 15-16, 2006
Rick Summerhill
Architecture
System - IV
• The following diagrams represent
preliminary thoughts on topology
• Exact locations may vary, but fiber is
expected to be on the Level3 footprint
• One of the goals of this meeting is to clarify
and optimize these locations
17
Community Design Workshop – June 15-16, 2006
Rick Summerhill
Architecture
Layer 1 Topology
PROVISIONAL TOPOLOGY – SUBJECT TO DISCUSSION
18
Community Design Workshop – June 15-16, 2006 Rick Summerhill
Rick Summerhill
Architecture
IP Network
• IP network built on top of optical system
• High reliability - architecture provides a variety of
protection options
• Commodity service offering - standard connection
will include commodity services
• Current plan is to continue to use highly reliable
Juniper routers, but open to new technologies
• Will likely use fewer routers, emphasizing point-topoint capabilities and hybrid networking
• Potential near term option of 40 Gbps
• Community input is important these issues
19
Community Design Workshop – June 15-16, 2006
Rick Summerhill
Architecture
Layer 1 Topology with IP Network
PROVISIONAL TOPOLOGY – SUBJECT TO DISCUSSION
20
Community Design Workshop – June 15-16, 2006 Rick Summerhill
Rick Summerhill
Architecture
Architecture Summary
• Initially provisioned with ten 10-Gbps
wavelengths (100 Gbps)
• Capacity: up to 80 wavelengths (0.8 Tbps)
• Scalability: Potential for 40 & 100 Gbps
wavelength support
• Reliability: Carrier provides standard service
assurances for these wavelengths
• Provide a variety of different types of reliability
• Flexibility: Support for dynamic provisioning
and wavelength switching
21
Community Design Workshop – June 15-16, 2006
Rick Summerhill
Architecture
Node Architecture
•Advanced optical DWDM
equipment
•Grooming capabilities to provide
sub channels
•Support for other networks
having a national footprint
•Simplified and standardized
interface to connectors, exchange
points, and other global research
and education networks
•2 x 10 Gbps interfaces
•Example displayed is the Denver
optical node, showing
connections required by the
collocated IP router node and an
additional network
PROVISIONAL ARCHITECTURE – SUBJECT TO DISCUSSION
22
Community Design Workshop – June 15-16, 2006 Rick Summerhill
Rick Summerhill
Architecture
Node Architecture
•Node without IP router
•IP connection to remote router
•Same basic architecture
PROVISIONAL ARCHITECTURE – SUBJECT TO DISCUSSION
23
Community Design Workshop – June 15-16, 2006 Rick Summerhill
Rick Summerhill
Architecture
Node Architecture
•Node without network connector
•Support for Other Network
•Same basic architecture
PROVISIONAL ARCHITECTURE – SUBJECT TO DISCUSSION
24
Community Design Workshop – June 15-16, 2006 Rick Summerhill
Rick Summerhill
Architecture
Example Circuit Services - I
• Services on NewNet
• Short Term Dynamically Configured STS-1 Granularity
Deterministic Circuits using two waves on the full footprint,
with framing either SONET or GFP mapped Ethernet. There
is no additional cost to use these capabilities. They are
included in the base connectivity Costs. The may be call
blocking in the use of these capabilities.
• Long Term Circuits configured as above. There is an extra
cost for these circuits, proportional to bandwidth, and charged
on a day-by-day basis. The shortest term is one day, and
there is no call blocking for these capabilities.
• Long Term full waves configured and provisioned for a
minimum of a year.
25
Community Design Workshop – June 15-16, 2006
Rick Summerhill
Architecture
Example Circuit Services - II
• Additional Carrier Services
• WaveCo services provisioned by Level3 as
part of the agreement.
26
Community Design Workshop – June 15-16, 2006
Rick Summerhill
Architecture
Example Projects on NewNet - I
• Dynamic services
• Creating lightpaths across multiple administrative
domains in seconds
• Development of “Domain Controllers”
• Support for applications - setup of “Application
SpecificTopologies”
• Dynamic inclusion of optical paths into IP
networks
• TCP data transfers
• File transfers with long segments not having to
require congestion control
27
Community Design Workshop – June 15-16, 2006
Rick Summerhill
Architecture
Example Projects on NewNet - II
• Network research projects deploying
programmable Ethernet capable chipsets
at each optical node using “NetFPGA”
cards from Stanford
• Service trial with GEANT2 on dynamic
provisioning of 1 GigE circuits across
Internet2 and GEANT2
28
Community Design Workshop – June 15-16, 2006
Rick Summerhill
Architecture
Internet2/GEANT2 Service Trial
• Develop inter-domain 1 GigE services between hosts
or clusters of hosts in Europe and the US
• Develop the technology to provide real services
• Domain controllers and interoperability
• Develop policies and cost models for providing such
services
• Interested in enlisting a couple of RONs and
campuses willing to participate
• The following diagram illustrates the problems
29
Community Design Workshop – June 15-16, 2006
Rick Summerhill
Internet2/GEANT2 Service Trial
30
Community Design Workshop – June 15-16, 2006
Rick Summerhill
Network Usage Policy
Network Usage Policy – I
• The use of the network facility shall be consistent with the broad
set of non-profit objectives of the U.S. research and education
community and its affiliates
• Internet2 will seek to maximize the ability of the network to
support the broad collaboration requirements of the Internet2
membership and their affiliates to interact with the broadest scope
of domestic and international networks and collaborating
organizations
• Each organization utilizing the facility will be responsible for
complying with all technical and operational standards developed
for the network, and will avoid actions that adversely impact the
performance or stability of the facility itself or those experienced
by other users
31
Community Design Workshop – June 15-16, 2006
Rick Summerhill
Network Usage Policy
Network Usage Policy – II
• The facility shall not be used for illegal,
classified, life-safety, or unrelated commercial
activities
• Additionally, Internet2 will seek to maximize to
its members the value of IP transit by
expanding connectivity through peering with
commercial networks including content
providers (e.g., Google, Akamai) and other
regional and national service providers (i.e.,
tier 2 ISPs)
32
Community Design Workshop – June 15-16, 2006
Rick Summerhill
Engineering
Engineering - I
• Building on the experience from projects such as
Abilene, HOPI, and NLR, operational support is
expected to fall into three broad categories:
• Control plane development, deployment, and support. The
Mid-Atlantic crossroads through the DRAGON project has
focused on dynamic resource allocation.
• Application support, enabled by targeting key applications for
the research community, applications such as eVLBI. The
North Carolina Research and Education Network has focused
on this area for the past year through the HOPI project
• Engineering, monitoring, and management has been
provided by the Global Network Operations Center at Indiana
University
33
Community Design Workshop – June 15-16, 2006
Rick Summerhill
Engineering
Engineering - II
• These groups were originally selected as
the HOPI testbed support center
• Internet2 expects to continue these
relationships for the new network
34
Community Design Workshop – June 15-16, 2006
Rick Summerhill
Network Research
Network Research - I
• The Internet2 network is an ideal platform for
network research - the ability to support both
highly experimental projects along with
production based services is a key objective
• The Abilene observatory will be expanded to
include the new capabilities of the network
• Data collection at all layers of the network, with
datasets made available to network researchers
• Support for collocation of equipment in optical
nodes
35
Community Design Workshop – June 15-16, 2006
Rick Summerhill
Network Research
Network Research - II
• Internet2 has supported individual network research
projects whenever possible. Examples include
• The router clamp
• MPLS tunnels for researchers - these can now be supported
at layer 1
• Support for the 100x100 project
• Flexibility is key feature of the new network
• Smaller projects at lower bandwidths can be supported on
variable footprints
• Projects can also be supported for variable lengths of time
• Projects like VINI and GENI can be supported by the
new network
36
Community Design Workshop – June 15-16, 2006
Rick Summerhill
Advisory Groups
Advisory Groups - I
• Internet2 currently has four councils: Network
Planning and Policy Advisory Council (NPPAC), the
Applications Strategy Council (ASC), the Industry
Strategy Council (ISC), and the Network Research
Liaison Council (NRLC). The latter in particular will
play a major role because of the developmental nature
of the network (Note that the Internet2 board is in the
process of reviewing the Internet2 governance)
• A network executive committee consisting of senior
executives from Internet2, Level3, and others including
Infinera
37
Community Design Workshop – June 15-16, 2006
Rick Summerhill
Advisory Groups
Advisory Groups - II
• A technical advisory committee that reports on
a regular basis to NPPAC
• The advisory committee could have detailed
working groups or other advisory groups (e.g., the
HOPI design team) organized to provide advice on
specific subjects
• It could commission working groups to work on
particular aspects of the network
• Input from the community is crucial - a later
session on this will seek further input
38
Community Design Workshop – June 15-16, 2006
Rick Summerhill
Partnerships
Partnerships
• The agreement with Level3 and the many capabilities
of the network provide an opportunity for strong
partnerships to form. For example,
• The partnership with Level3 to examine new service
capabilities
• Joint work on control plane investigations
• Examination of new optical capabilities with Infinera and
others
• International partnerships, for example related to service trials
• Federal government partnerships, continuing and expanding
on collaborations with other networks such as Esnet
• Continued work with the original Internet2 partners, Qwest
communications, Cisco Systems, and Nortel Networks
39
Community Design Workshop – June 15-16, 2006
Rick Summerhill
Agreement Summary
Additional Agreement Information
• Continuation of FiberCo
• Introduction of WaveCo
• Essentially, a “circuit” version of FiberCo
• Provides a platform for collaboration with
the corporate and academic research
communities to examine new services
and ideas
• More information will be presented later
40
Community Design Workshop – June 15-16, 2006
Rick Summerhill
41
Community Design Workshop – June 15-16, 2006
Rick Summerhill