Transcript Abilene - Internet2

Internet2: A Tutorial
Part 2 of 4
17th Brazilian Symposium
on
Computer Networks
Paul Love, Internet2
Chair, I2 Topology WG
[email protected]
Abilene
Overview
•
•
•
•
•
•
•
Abilene Background
Review of Abilene Architecture
1999 Schedule
Launched Network
First Half 1999 Expectations
Engineering Issues
Additional information
http://www.internet2.edu/abilene/
Abilene: Objectives
• Provide high-quality, widely available
Interconnect among participating
gigaPoPs/universities
• IP as the bearer service
• Goals
•
•
•
•
Reliability
Functionality
Latency
Capacity
SBRC99 Salvador, Bahia, Brazil
25-28 May 99
Abilene: Objectives cont
• Additional, separate networks to
• Test advanced network capabilities
• Final proposals being evaluated this month
• For network research - still TBD
• Support advanced functionality as it
evolves
• Multicast, QoS, IPv6
SBRC99 Salvador, Bahia, Brazil
25-28 May 99
Abilene Team
•
•
•
•
•
•
•
•
Overall direction by Internet2
Qwest
Cisco
Nortel (Northern Telecom)
Indiana University
National Center for Network Engineering
Open to other contributors
Collaborate with related efforts in
network or applications research
SBRC99 Salvador, Bahia, Brazil
25-28 May 99
Key Attributes
• IP over Sonet initially
• Qwest OC-48 Sonet capacity and
collocation sites
• Nortel OC-192 Sonet equipment
• Cisco GSR 12000 routers including
support for
• QoS
• Native multicast
SBRC99 Salvador, Bahia, Brazil
25-28 May 99
Abilene and Other Networks
• Important for Abilene to interconnect with
other high performance networks
• First I2 backbone: vBNS
• Federal Agency NGI networks (ESnet, NREN,
DREN, ...)
• Non-US Research & Education Networks
• CANARIE, NORDUnet, SURFnet, TransPac, etc
• Peering in hand or plans in process
• vBNS, ESnet, CANARIE, DREN, NREN, NORDUnet,
SURFnet, etc
• Advanced functionality across I2 & our peers
• Native multicast
• QoS across multiple interconnected networks
SBRC99 Salvador, Bahia, Brazil
25-28 May 99
*RSN
Broader Internet2, NGI, and
International Advanced Nets
Initial NGIX sites
Possible CA*net3 peering sites
StarTap
StarTap Plus
SBRC99 Salvador, Bahia, Brazil
25-28 May 99
Notes on I2, NGI, and
International Interconnections
• NGIXes are the foundation for peering I2
backbones & with NGI nets
• Abilene will connect with OC12
• StarTap is the foundation of international
interconnections - universal solution
• New York is first example of StarTap Plus
• Minimize latency
• Provide redundancy
• Abilene doesn’t provide transit
SBRC99 Salvador, Bahia, Brazil
25-28 May 99
Example: New York City
• SURFnet has OC-3 circuits and routers at
the Teleglobe POP at 60 Hudson Street
• NORDUnet the same
• And each connects from New York to the
StarTap with DS3
• Details on the next slides
SBRC99 Salvador, Bahia, Brazil
25-28 May 99
New York City
ISP
Europe
SURFnet
NORDUnet
Teleglobe
Qwest
Abilene
Chicago
SBRC99 Salvador, Bahia, Brazil
25-28 May 99
New York City
ISP
Europe
SURFnet
NORDUnet
Teleglobe
Qwest
Abilene
Chicago
SBRC99 Salvador, Bahia, Brazil
25-28 May 99
CA*net3
Calgary
Regina
St. John’s
Winnipeg
Vancouver
Charlottetown
Fredericton
Montreal
Ottawa
Chicago
SBRC99 Salvador, Bahia, Brazil
25-28 May 99
Toronto
Halifax
Abilene NOC
• Located at Indiana University
• Publish statistics & online information
resources
• Coverage can be looked at in three broad
areas
• Administration
• Engineering
• Operations
SBRC99 Salvador, Bahia, Brazil
25-28 May 99
Administration
• Security Management
• Communications
• Overall Management and Coordination
SBRC99 Salvador, Bahia, Brazil
25-28 May 99
Engineering
•
•
•
•
Configuration
Testing
Data collection
Traffic Analysis
SBRC99 Salvador, Bahia, Brazil
25-28 May 99
Operations
•
•
•
•
•
24x7 Monitoring
Reporting
Change and Problem management
Information base
Single point of contact
• Network problems
• Abilene connection process
SBRC99 Salvador, Bahia, Brazil
25-28 May 99
Online Traffic Map
For more information
on the Abilene NOC
• www.abilene.iu.edu
• Abilene traffic map at
http://hydra.uits.iu.edu/~abilene/traffic
SBRC99 Salvador, Bahia, Brazil
25-28 May 99
Couple of Notes on Costs
• Cost recovery on UCAID’s part
• Set each year for following year
• Participant institution
• Connector - based on BW
(can be connector and not participant)
• nb There is also a small Qwest access fee
• Long-distance part of access line is
considered part of the ‘backbone’
• Thus, number/location of core
nodes does not affect costs borne
by connecting sites
SBRC99 Salvador, Bahia, Brazil
25-28 May 99
Qwest Fiber Network
http//www.qwest.com/network/netmaps.html
SBRC99 Salvador, Bahia, Brazil
25-28 May 99
Schedule - part 1
•
•
•
•
•
•
•
•
Design work: Mar-98 and ongoing
Rack design: May-98 to Jul-98
Initial assembly / testing: Jul/Aug-98
Router Nodes / Interior Lines: Jul/Aug-98
Demo network installed: Sep-98
Remainder installed: Nov/Dec-98
Production begins: 29-Jan-99
Launch Event in DC: 24-Feb-99
SBRC99 Salvador, Bahia, Brazil
25-28 May 99
Abilene Sep 98 Demo Network
Seattle
Eugene
Minneapolis
Westfield
Boston
New York
Cleveland
Detroit
Salt Lake City
Pittsburgh
Lincoln
Sacramento
Oakland
Indianapolis
Newar
Trent
k
on
Philadelp
Wilmington
hia
Columbus
Washington
Denver
Kansas City
Raleigh
Albuquerque
Nashville
Los Angeles
Atlanta
Anaheim
Phoenix
Dallas
New Orleans
Router Node
Houston
Miami
Access Node
Star Tap
SBRC99 Salvador, Bahia, Brazil
25-28 May 99
New Haven
Abilene Architecture: Core
• Router Nodes located at Qwest PoPs
•
•
•
•
•
Cisco 12008 GSR
ICS Unix PC: IPPM and Network Mgmt
Cisco 3640 Remote Access for NOC
100BaseT LAN and ‘console port’ access
Remote 48v DC Power Controllers
• Initially, ten Router Nodes
• Seattle, Sacramento, Los Angeles, Denver,
Kansas City, Houston, Indianapolis, Cleveland,
New York City, Atlanta
• And, coming this Fall - Washington, DC
• Interior lines connect core nodes
• OC-12 and OC-48 Sonet
• IP-over-Sonet interfaces
SBRC99 Salvador, Bahia, Brazil
25-28 May 99
SBRC99 Salvador, Bahia, Brazil
25-28 May 99
Abilene Architecture: Access
• Access Nodes
• Located at Qwest PoPs
• Sonet: Connects Local to Long-distance
• Currently about 135 Access Nodes:
• This list grows as the Qwest Sonet plant grows
• Access lines connect from core node to
gigaPoP or member
• IP-over-Sonet or IP-over-ATM possible
• OC-3 and OC-12 typical
• Others as possible & appropriate
• GigE, ...
SBRC99 Salvador, Bahia, Brazil
25-28 May 99
Abilene + some Access Nodes
Seattle
Boston
Eugene
Minneapolis
Westfield
New York
Cleveland
Newar
Trent
k
on
Philadelp
Wilmington
hia
Detroit
Salt Lake City
Chicago
Pittsburgh
Lincoln
Sacramento
Oakland
Indianapolis
Columbus
Washington
Denver
Kansas City
Raleigh
Albuquerque
Nashville
Los Angeles
Atlanta
Anaheim
Phoenix
Dallas
New Orleans
Houston
Miami
Router Node
Access Node
SBRC99 Salvador, Bahia, Brazil
25-28 May 99
New Haven
Launch Network
•
•
•
•
•
Some Sonet Rings not completed
Mix of OC-12 and OC-48 interior Lines
NOC fully operational
Some Access Lines not completed
Connections at two NGIXes
• NGIX/Central (OC3)
• NGIX/West (OC12)
• OC12 into Union Station for demos
SBRC99 Salvador, Bahia, Brazil
25-28 May 99
Schedule, part 2
•
•
•
•
•
•
•
Production began: 29-Jan-99
Launch Event in DC: 24-Feb-99
Member Meeting: 28-Apr-99
Native Multicast deployed in core: Jun-99
Completion of OC-48 Core: TBD
Addition of DC Router Node: 3Q99
Seattle Member Meeting: 11/15-Oct-99
SBRC99 Salvador, Bahia, Brazil
25-28 May 99
First Half 1999 Expectations
• Measurements of Delay, Loss, Utilization
• Gradual
• Increase in OC-48 Interior Lines
• Completion of (Qwest) Sonet Rings
• Accesss to measurements and other
engineering data
• Participation in QBone
• Upgrade of NGIX/Central to OC12
• Native Multicast
SBRC99 Salvador, Bahia, Brazil
25-28 May 99
Second Half 1999
Expectations
• Gigabit Ethernet connection(s)?
• OC48 connection?
• Connection to NGIX/East
• Somewhere in Washington, DC, area
• OC12
SBRC99 Salvador, Bahia, Brazil
25-28 May 99
Abilene Core: Current
Seattle
New York
Cleveland
Indianapolis
Sacramento
Denver
Kansas City
Los Angeles
Atlanta
Houston
SBRC99 Salvador, Bahia, Brazil
25-28 May 99
Abilene: Current plus On Order
Seattle
New York
Cleveland
Sacramento
Denver
Indianapolis
Kansas City
Los Angeles
Atlanta
OC12 still being installed
Houston
OC12
OC48
OC12
OC3 SBRC99 Salvador, Bahia, Brazil
25-28 May 99
Outline of Engineering Issues
• Routing:
• OSPF, BGP4, Routing Arbiter Database
• Multicast
• PIM-Sparse Mode, MBGP, MSDP
• Measurements
•
•
•
•
Surveyor: One-way delay and loss
Traffic utilization
End to end flows with gigaPoP help
OC3MON -- passive measurements
• QoS - QBone
SBRC99 Salvador, Bahia, Brazil
25-28 May 99
GigaPoP Architecture
Acknowledgments
• Steve Corbató - University of Washington &
Pacific/Northwest Gigapop
• Ron Hutchins - Georgia Institute of Technology &
Southern CrossRoads Gigapop
• Ken Lindahl - University of California, Berkeley
• Rick Summerhill - Executive Director, Great Plains
Network
• David Wasley - CENIC Director of Projects/CalREN-2
Gigapop
SBRC99 Salvador, Bahia, Brazil
25-28 May 99
GigaPoP Idea
• From George Strawn of NSF - Predates
Internet2
• Idea was one of the catalysts for Internet2
• Aggregation point
• Economies of scale
• Member sites
• Attached networks
• Value added services shared by members
• Not limited to university/I2 members
• (High-speed) local traffic stays local
SBRC99 Salvador, Bahia, Brazil
25-28 May 99
GigaPoPs
• Variety of services and styles
• Technical and organizational differences
• Mixture of technologies
• Some things must be the same
•
•
•
•
IP as a common bearer service
Inter-GigaPoP routing policy and design
Measurement
Trouble tickets among network operation
centers
SBRC99 Salvador, Bahia, Brazil
25-28 May 99
Backbone nodes
Robust Model
Peering connection
Aggregation points/
GigaPoPs
Agency Networks
Customer links
SBRC99 Salvador, Bahia, Brazil
25-28 May 99
With thanks to Ron Hutchins
GigaPoPs can be Hierarchical
UNC
Vanderbilt
Duke
NCSU
MCNC
UAH
UAB
GCATT
UA
SoX
GigaPoP
GT
vBNS
GSU
Peachnet
FSU, UF,
Miami
SBRC99 Salvador, Bahia, Brazil
25-28 May 99
With thanks to Ron Hutchins
Internet2 GigaPoPs
As of November 1997
Denotes GigaPoP
SBRC99 Salvador, Bahia, Brazil
25-28 May 99
Internet2 GigaPoPs
As of 19 May 1999
Denotes GigaPoP
SBRC99 Salvador, Bahia, Brazil
25-28 May 99
GigaPoPs - What Flavors Do
They Come In
• Two broad sets of choices
• Level
• Topology
• Level
• Layer 1 (optical)
• Layer 2 (usually ATM)
• Layer 3
• Topology
• Star - aka Gigapoint
• Distributed
• Combination - figure “8”
SBRC99 Salvador, Bahia, Brazil
25-28 May 99
Star or Gigapoint
• Advantages
• No internal transit
issues
• Disadvantages
• Longer backhauls
for both members
& service providers
Member links
ISP/HPNSP links
SBRC99 Salvador, Bahia, Brazil
25-28 May 99
Distributed
• Advantages
• Optimize backhauls
• Take advantage
existing connections
• Politics
• Disadvantages
• BW between nodes
can be bottleneck
Member links
Intra-GP links
ISP/HPNSP links
SBRC99 Salvador, Bahia, Brazil
25-28 May 99
Combination
• Advantages
• Optimize backhauls
• Take advantage
existing connections
• Politics
• Minimize internal transit
• Disadvantages
• BW between hubs
can be bottleneck
• Longer backhauls
SBRC99 Salvador, Bahia, Brazil
25-28 May 99
Member links
Intra-GP link
ISP/HPNSP links
Example - Non-ATM
Level 2 GigaPoP
P/NW GigaPoP Design
With thanks to Steve Corbató
SBRC99 Salvador, Bahia, Brazil
25-28 May 99
Example - Combination
GigaPoP
Great Plains Network
• Participating States
•
•
•
•
•
•
N & S Dakota
Nebraska
Kansas
Oklahoma
Missouri
Arkansas
• Topology
With thanks to Rick Summerhill
SBRC99 Salvador, Bahia, Brazil
25-28 May 99
Example - Distributed
GigaPoP
CalREN-2 Topology Plan (5/98)
Corporation for Education Network Initiatives in California
CalREN-2 Network
Internet2
(vBNS )
Sacramento
UCB, UCD, UCSF,
UCSC, Stanford, UCOP
UC Davis
UC Berkeley
UCOP, Oakland
UC San Francisco
Stanford
(Planned but not funded)
Merced
UC Santa Cruz
Northern California
GigaPOP
Fresno
Monterey
USC, ISI, UCLA, UCI, Caltech,
CSU, JPL, UCR, UCSB
OC-12c
OC-48
Planned N-S
Interconnect
UC Santa Barbara
UCLA JPL
CSU
ISI
Southern California
GigaPOP
Caltech
USC
UC Riverside
UC Irvine
Internet2
(vBNS )
UC San Diego
San Diego GigaPOP (Future)
SBRC99 Salvador, Bahia, Brazil
25-28 May 99
UCSD & SDSC
With thanks to David Wasley
CalREN-2 Topology
UC Davis
UC
Berkeley
Stanford
ESnet
CalREN2 North
UC Santa
Cruz
UC San
Francisco
Abilene
UC OP
vBNS
Abilene
UC Los
Angeles
ISI
USC
CSU
JPL
CalREN2 South
UC Irvine
(future)
NREN
(future)
Caltech
UC
Riverside
OC-48
UC Santa
Barbara
UC SD
SDSC
OC-12c
ESnet
vBNS
SBRC99 Salvador, Bahia, Brazil
25-28 May 99
With thanks to Ken Lindahl
CalREN-2 North: Physical Topology
CBR
UCSC
S
UCD
S
CBR
ESNet
Abilene
CBR
Stanford
R
S
UC Berkeley
R
S
CBR
vBNS
S
CBR
S
R
UC SF
OC-12 PoS
OC-12 ATM
R
UC OP
CBR
CBR
campus
border router
SBRC99 Salvador, Bahia, Brazil
25-28 May 99
With thanks to Ken Lindahl
CalREN-2 North: Logical Topology
UCSC
UCD
CBR
CBR
ESNet
Abilene
CBR
Stanford
R
UC Berkeley
R
CBR
vBNS
CBR
R
UC SF
OC-12 PoS
OC-12 ATM
OC-3 VC
R
UC OP
CBR
CBR
campus
border router
SBRC99 Salvador, Bahia, Brazil
25-28 May 99
With thanks to Ken Lindahl
Example - Star/Gigapoint
SoX: Layer 3 Example
R
R
R
R
R
R
Georgia GigaPoP
R
R
Logical Layer 3 Diagram
vBNS
R
R
Peering using BGP
R
R
R
R
R
R
ISP
ISP
SBRC99 Salvador, Bahia, Brazil
25-28 May 99
With thanks to Ron Hutchins
SoX: Layer 2 Example
Router
PNNI
“External Gigapop” PNNI
vBNS
A
Layer 2 diagram
GigaPOP Switch
vBNS
Router
PNNI
Router
GT Network
Router
ISP
B
PNNI
Router
PNNI
Dept. Net
Georgia GigaPOP
SBRC99 Salvador, Bahia, Brazil
25-28 May 99
With thanks to Ron Hutchins
Combined Services
ISP 1
*
*
*
HPNSP
ABR/UBR PVC
Burst to Full OC-3
SoX
Gigapop
Campus
Network
ISP 2
*
*
Campus 1
VBR PVC
Preempt to 10Mb/s
vBNS
ISP
ATM “Pipe”
ISP 3
*
Campus 2
3
SBRC99 Salvador, Bahia, Brazil
25-28 May 99
4
With thanks to Ron Hutchins
The End
SBRC99 Salvador, Bahia, Brazil
25-28 May 99