Transcript NLR Layer2/Layer3 BOF NLR status update

NLR Layer2/Layer3 Users BOF
NLR status update
Philadelphia Internet2 Member Meeting
19 September 2005
Brent Sweeny, Indiana University
National LambdaRail design
SEA
POR
SYR
BOI
STA
OGD
SVL
PIT
DEN
KAN
SLC
NYC
CLE
CHI
WDC
RAL
LAX
PHO
ALB
SAN
ELP
TUL
NLR owned fiber
SAA
NLR WaveNet PoP
NLR WaveNet & FrameNet PoP
NLR WaveNet, FrameNet & PacketNet PoP
9/19/2005
ATL
DAL
PEN
JAC
BAT
HOU
NLR Layer2/Layer3 users BOF
2
Generic NLR L1, L2 and L3 PoP Layout
Colo
West
15808
15454
East
6509
NLR demarc
CRS-1
DWDM
10G wave, link or port
1G wave, link or port
9/19/2005
NLR Layer2/Layer3 users BOF
3
NLR Layer 1
SEA
8
8
POR 8
8
8
BOI
SYR
4 4
4
OGD
SVL
8
4 8
8
8
LAX
SLC
DEN
4
4
8 4
4
4
PHO
4
8
4
4
Level3 fiber
WilTel fiber
Cisco 15808 terminal
Cisco 15808 OADM
Cisco 15454 terminal
Cisco 15454 OADM
9/19/2005
8
CHI
CLE
4
PIT
8 4
8
RAT
RAL
TUL 4
ALB
8
DAL44
ELP
4 4
SAA
PEN
4
4 4
4 4
4 4
4
4
NYC
88
8 8
4
4
4
4 4
8
KAN
8 8
4
STA
4
8
8
WDC
8
8
ATL
8
4
JAC
BAT
HOU
NLR Layer2/Layer3 users BOF
4
Layer 1 Phase 2 Deployment
SYR
4 4
OGD
DEN
4
SLC
4
4
4
4
CLE
KAN
4
4
4
LAX
4
4
4
4
PHO
4
8
4
4
Level3 fiber
WilTel fiber
Cisco 15808 terminal
Cisco 15808 OADM
Cisco 15454 terminal
Cisco 15454 OADM
9/19/2005
WDC
RAT
TUL 4
ALB
4
4
4 4
8
NYC
DAL44
ELP
4 4
SAA
PEN
4
4 4
4 4
4 4
4
JAC
BAT
HOU
NLR Layer2/Layer3 users BOF
5
Layer 1 baseline

Opportunity to connect into lambda fabric



Early examples:





Point to point
Other endpoint could be anywhere
HOPI
Ultralight
iGRID
SC05 (Supercomputing)
Experiments support center (for all layers)
9/19/2005
NLR Layer2/Layer3 users BOF
6
Layer 2 Network Design
SEA
CHI
PIT
SVL
DEN
NYC
CLE
KAN
WDC
RAL
LAX
ALB
TUL
ATL
ELP
JAC
BAT
Cisco 6509 switch
SVL
HOU
10GE wave
10GE managed wave
Yellow sites are done
9/19/2005
NLR Layer2/Layer3 users BOF
7
Layer 2 installation status
Switch installations completed:
Sunnyvale
Cleveland
Jacksonville
El Paso
Denver
Pittsburg
Atlanta
Houston
Kansas City
Raleigh
Los Angeles
Chicago
Washington DC
Tulsa
New York City scheduled for this week (Sept 20-21).
Baton Rouge waiting for Katrina side-effects to calm down. (Was
scheduled
for the week of August 29, now tentatively Oct 11-12.)
Albuquerque scheduled for mid-Oct.
Interconnections being completed now.
9/19/2005
NLR Layer2/Layer3 users BOF
8
Layer 2 service baseline

1GE Connection into local 6500


Access to “national peering VLAN”
Additional Options:
Dedicated point to point Etherrnet, Nx1GE
 Best-effort point to multipoint (no dedicated bw)


Soon:
 10GE ports
 Dedicated point-to-multipoint
9/19/2005
NLR Layer2/Layer3 users BOF
9
Goal
 Provide
circuit-like options for users who
can’t use, can’t afford, or don’t need, a
10G Layer1 wave via point-to-point layer2
VLANs.
 Experiment
capabilities.
9/19/2005
with large-scale layer2
NLR Layer2/Layer3 users BOF
10
Dimensions of NLR Layer2 capabilities






point-to-point or multipoint
public or private
bandwidth guaranteed or best effort
resilient (spanning tree) or nailed up
temporary or permanent
experimental or stable/productionoriented
Some of the above dimensions are either/or, some
are more spectrum-like.
The following slides show some expected uses of
NLR layer 2, and how these dimensions may
relate to each type of project.
9/19/2005
NLR Layer2/Layer3 users BOF
11

National Peering Fabric

Multipoint, public, best-effort, resilient,
permanent, stable
NLR allocated addresses, peer with any other
member across the layer 2 fabric policy-free.
 Possible to have more than one (say by MTU).
 Ready to go today.


Backup connections to networks such as
Abilene, commodity providers, etc.

Point-to-point, private, permanent, stable
Could be best-effort or guaranteed.
 Could be nailed-up or resilient.
 Load-balance or leave idle until needed.
 Ready to go today, though we only have pricing for
the guaranteed
nailed-up case.
9/19/2005
NLR Layer2/Layer3 users BOF
12


To enable a flexible topology for NLR
layer3

Best-effort, private, temporary, experimental
We have 8 layer3 nodes, but the topology between
them can be made much more interesting by creating
various connections over the layer2 network.
 Enables layer3 experimentation.


To provide members with a second path
into the NLR layer3 network

Point-to-point, private, permanent, experimental
Connect to a second node on the layer3 backbone.
 Load-balance or leave idle until needed.
 Included in membership.

9/19/2005
NLR Layer2/Layer3 users BOF
13

Temporary connections for special
projects

Guaranteed, private, temporary, stable
For remote instrumentation, where the member only
has the remote resource reserved for a limited
window.
 For conferences, demos, and other special events.
 Provides a low latency/jitter path if needed.
 Nailed-up if latency is critical, probably resilient if not.
 Could be point-to-point or multipoint.
 Technically, this is possible today, but we have no
pricing model for it.

9/19/2005
NLR Layer2/Layer3 users BOF
14

Bootstrapping circuit-like research

Point-to-point, private, guaranteed, temporary,
nailed-up, experimental
To enable a researcher to get started while waiting
for funding or provisioning of a layer1 circuit.
 Similar to a special event, but more experimental, a
probably a stronger need for it to be nailed-up.
 Technically, this is possible today, but we have no
pricing model for it.


Provide control plane network for optical
experiments

Permanent, resilient, experimental

9/19/2005
A topology could be created for the oob management
network needed for some dynamic optical networking
experiments (GMPLS, etc.)
NLR Layer2/Layer3 users BOF
15

Cluster/Grid LAN

Multipoint, private, guaranteed, experimental
To enable remote clusters to appear on the same
LAN.
 It is not known if spanning tree would be wanted.
 It could evolve into a more production-like
requirement.
 Technically, this is possible today, but we have no
pricing model for it.


Experiment directly with Layer 2

Could be of any type (experimental, obviously)
Web-based provisioning, direct user requests, etc.
 Concern about interaction with more production-like
requirements.

9/19/2005
NLR Layer2/Layer3 users BOF
16
Layer 3 Network
SEA
CHI
NYC
DEN
PIT
WDC
LAX
PHO
ALB
RAL
TUL
ATL
BAT
JAC
HOU
Cisco CRS-1 router
LAX
10GE wave
Yellow sites are installed
9/19/2005
NLR Layer2/Layer3 users BOF
17
Layer 3 installation status
With the work this week in NYC, all layer3
router installations are complete.
Waves are being built between routers and
for layer3 backhaul now.
9/19/2005
NLR Layer2/Layer3 users BOF
18
Layer3 service baseline

Each member gets two routed connections







“local” 10GE
VLAN backhauled to 2nd node
BGP peering with NLR L3 network
IPv4 unicast
IPv4 multicast (MBGP/PIM/MSDP)
IPv6 unicast (multicast later)
An ‘experimental’ (changeable, changing) L3
network
9/19/2005
NLR Layer2/Layer3 users BOF
19
Layer 3 coming services







Likely eventual logical routers
More 1GE options
More 10G options
Pre-emptable connections
MPLS
More user control—scheduling, testing, etc
User access to measurement data
9/19/2005
NLR Layer2/Layer3 users BOF
20
NLR Engineering/Support
Organization
A very distributed, coordinated organization:




Service desk at Indiana
Layer1 NOC and engineering at CENIC
Layer2/3 NOC and engineering at Indiana
Experiments support center at North Carolina
9/19/2005
NLR Layer2/Layer3 users BOF
21
NLR User Resources




http://noc.nlr.net
[email protected]
[email protected]
[email protected]
9/19/2005
NLR Layer2/Layer3 users BOF
22
NLR Layer 2/3 - discussion
What do users want/need?
Tools?
User groups?
Instruction/workshops?
Monitoring and measurement ability? Specify?
Full routes? Control over route propagation?
Control over protocols and timers?
What would you do with logical routers?
Direct access to login and configure routers?
Commodity access or ISP collaboration?
Collaboration with projects like PlanetLab and WAIL? Which
others?
9/19/2005
NLR Layer2/Layer3 users BOF
23
Summary from July ‘05 (Joint Techs BOF)




Costs
More deeply-technical descriptions of the architecture,
service offerings
L2 and L3 service-design docs
Standardized interface into resources
And some thoughts about possibilities:






"matchmaking" service, across all layers?
Logical routers: separate forwarding/control planes,
possibly enabling customer logical networks (RONs)
Customer edge control
Native non-IP, non-Ethernet protocols?
Commodity internet peering?
What’s NLR trying to be? (“dancing-elephants or dancinggerbils”, “facile/flexible”)
9/19/2005
NLR Layer2/Layer3 users BOF
24