Transcript Network Architectures

Network Architectures
OARTech
Paul Schopis
October 13, 2005
Topics
• TFN/OARnet background
• General MPLS Description
• What problem are we trying to solve
anyway?
• Early Experiments at ITEC
• TFN implementation
OARnet Background
• Founded in 1987 as part of the Ohio
Supercomputing Center
• 90+ higher ed member institutions
• Board of Regents funding
• OSTEER advisory council
• Internet2 GigaPOP
Third Frontier Network
• Phase 1: replace backbone with dark
fiber
• Phase 2: connect 17 universities to
network with dark fiber or gig circuits
• Phase 3: connect other universities and
colleges
• Phase 4: connect other partners
Dark Fiber Acquisition
• RFP issued during Summer of 2002
• Dark fiber was strongly preferred, but
leased services considered
• Vendors who bid dark fiber were
required to offer a minimum of a single
pair of fiber over their network
Dark Fiber Acquisition
• Determined that leased lambdas were
too expensive and not widely available
• Selected a bid from Spectrum Networks
for single pair of fibers
– American Electric Power (AEP)
– Williams Communications (Wiltel)
– American Fiber Systems
Spectrum
• We had various responses
• The providers in the Spectrum offer bid
individually
• No price increase for using Spectrum as
integrator
• SBC and others reported no bid bit desired to
bid on future last mile
– inter-lata issue
Dark Fiber Acquisition
•
•
•
•
$4.6 M for 20 year IRUs
$342K/yr for maintenance
1600+ route miles
Truewave, SMF-28, LEAF or Terra Light
Fiber
• Aerial and buried
TFN Financing
• $21M investment
• Financing from Ohio State University
– Loan for fiber ($7M)
– Short-term financing ($2M)
• Financing from state capital budget
($8.5M)
– Equipment
– Last mile to 17 institutions
Community
• We desired to make this a true
community owned network
• Committees with schools participating in
decisions and recommendations
Equipment
• Cisco 15454 integrated solution (DWDM)
– all of the amps, mux/demux etc. integrated
•
Multi Service Transport Platform (MSTP)
– ITU G.709 compliant
• Cisco routers (GSR 12000) and switches
• Juniper M7i routers
Last Mile
• RFP issued in Dec 2003 for last-mile
connectivity to all higher education and
K-12 sites
• OC3, gig circuits and10 gig circuits
• We did make contact with local fiber
providers on backbone bid ex. Buckeye
Telesys
General MPLS Description
General MPLS Description
Packet have a 20 bit label that routes it
along a “Label Switched Path”. Values range
from 0 to 1,048,575.
0 through 15 are reserved for special uses.
Some label ranges have special meanings
for specific vendors.
General MPLS Description
•0 IPv4 Explicit Null Label - No label
stacking, must POP label
•1 Router Alert Label - delivered to local
router for local processing
•2 IPv6 Explicit Null Label - Same rule as
IPv4 except forwarded to IPv6 routing
instance.
•3 Implicit Null Label - Control protocol (LDP
or RSVP) request for down stream router to
POP Label
General MPLS Description
Choosing the next hop can be thought of as the
composition of two functions. The first function
partitions the entire set of possible packets into a set
of"Forwarding Equivalence Classes (FECs)". The
second maps each FEC to a next hop.
In many ways an IP prefix is a FEC
IP routing protocols are the mechanisms to map IP
FECs to a next hop.
General MPLS Description
What are the advantages of MPLS?
General MPLS Description
MPLS forwarding can be done by switches which are
capable of doing label lookup and replacement, but
are either not capable of analyzing the network layer
headers, or are not capable of analyzing the network
layer headers at adequate speed.
General MPLS Description
Since a packet is assigned to a FEC when it enters
the network,the ingress router may use, in
determining the assignment, any information it has
about the packet, even if that information cannot be
gleaned from the network layer header. For
example,packets arriving on different ports may be
assigned to different FECs. Conventional forwarding,
on the other hand,can only consider information
which travels with the packet in the packet header.
General MPLS Description
A packet that enters the network at a particular
router can be labeled differently than the same packet
entering the network at a different router, and as a
result forwarding decisions that depend on the
ingress router can be easily made. This cannot be
done with conventional forwarding, since the identity
of a packet's ingress router does not travel with the
packet.
General MPLS Description
Sometimes it is desirable to force a packet to follow
a particular route which is explicitly chosen at or
before the time the packet enters the network, rather
than being chosen by the normal dynamic routing
algorithm as the packet travels through the network.
This may be done as a matter of policy,or to support
traffic engineering. In conventional forwarding,this
requires the packet to carry an encoding of its route
along with it ("source routing"). In MPLS, a label can
be used to represent the route, so that the identity of
the explicit route need not be carried with the packet.
General MPLS Description
Some routers analyze a packet's network layer
header not merely to choose the packet's next hop,
but also to determine a packet's"precedence" or
"class of service". They may then apply different
discard thresholds or scheduling disciplines to
different packets.MPLS allows (but does not require)
the precedence or class of service to be fully or
partially inferred from the label. In this case, one may
say that the label represents the combination of a
FEC and a precedence or class of service.
What problem are we trying to
solve anyway?
The Problem
• Goal create an Abilene Premium Service
• Need to create “Virtual Wire” ( Smells a lot
like a light path)
• Need predictable bandwidth
• Need to meet DiffServ EF requirements
• Need to be able to signal request for
resources
• Needed admission control
The Solution
• DiffServ Code Point
• Queuing mechanisms High Priority
• Policy on edge to mark and forward via
high priority queue
• Admission control for LSP (MPLS
Tunnels) via marked packets that
conformed to requirements
The Solution
• LSPs anchored to WRED Queues on
WAN side
• All CPE side used High Priority
• Tested across multiple BGP Domains
• Tested QPPB for discovery of QoS
resources
The Solution
• Used RSVP to signal request for “subpool” reservation, e.g. guaranteed BW
• Resulted in primitives being
incorporated into DSTE-MPLS
• Results used to write RFC 3270
AS 3
AS 3
AS 1
AS 2
AS 4
AS 3
AS 3
AS 1
AS 2
AS 4
AS 3
AS 3
AS 1
AS 2
AS 4
AS 3
AS 3
AS 1
AS 2
AS 4
TFN implementation
TFN implementation
• Needed to migrate to new network
• Needed to provide services such as
multicast and IPv6
• Needed to solve fish problem
• Executed test plan based on Abilene
test plan
Legacy POP Design
I1
I1
ATM
I2
ATM
I2
ATM
I1&I2
POP
Campus
Legacy POP Design
I1
I1
ATM
I2
ATM
I2
ATM
I1&I2
POP
Campus
BGP for
Route diff
New OARnet Design Goals
•
•
•
•
Reduce Costs
Reduce Complexity
Reduce Maintenance Fees
Deliver Services
MPLS Requirements
•
•
•
•
•
•
•
CPE device
PE Provider Edge
P Provider Core LSP Switching Router
We can collapse P and PE to one device
Need CPE for Label to IP binding
I1 will be standard routing
I2 will be Label Switched with BGP multihop to find correct
path
• Must deliver advanced services to I2 community
– IPv6, Multicast, Jumbo Frames etc.
New Architecture
PE/P
CPE
CPE
GigE
Aggregator
POP
GigE
Campuses
New Architecture
MPLS for I2 Routes LFIB
IP for I1 Routes FIB
Red = LDP tagged
CPE
PE/P
CPE
GigE
AS600 Aggregator
AS3112
BGP Multihop
LDP Exchange with
Core
BGP Multihop
LDP Exchange with
Core
GigE
Campuses
New Architecture
CPE
PE/P
LR 1
GigE
AS600 Aggregator
AS3112
LR 2
GigE
Campuses
Rate Cap Architecture
Red = I2 Cap
Blue = Commodity Cap
Green = Intra State Cap
PE/P
CPE
CPE
GigE
AS600 Aggregator
AS3112
GigE
Campuses
Some Implementation Issues
• Had to come up with more robust
naming convention
– Old ALP1, SWALP1
• Required DNS overhaul
– Pseudo CILLY code
• CLMBN-R0, CLMBN-E0, CLMBN-O1, CLMBNOT1
Questions?