Transcript Slide 1

A new national network
Andrew Mackarel
Andrea Tognola
Dave Wilson
Agenda
• Evolution of the network
• Procurement process
• The new hardware and layout
• Transition and timeline
Evolution of the network
Star topology
1994-1999
First IP network UCC
star topology
Single core router in UCD
(then a second for BGP)
DIT
Forbairt
MCI
UCD
Typical speeds: 64k-2M,
clients and peers
HEA
NCIR
VCIL
TCD
Ebone
National Backbone
2000-2007
External peers and upstreams
• Bring the network
out of Dublin
• Avoid duplication & cost
• Add resilience
• Tendered for PoP sites
– Won by client sites
• Typical speeds
Clients
– clients up to ~10M, upstreams N*2M
Multi-pop backbone
• This model has scaled to 1Gbit/s
• Overlaid on National Backbone Extension
• Dark fibre network providing ethernet
point-to-point
– Bring network
to the client
– Avoid duplication
– Reduce cost
– Increase flexibility
– Add resilience
• These goals now addressed by NBE!
National Backbone
Scaling limit
External peers and upstreams
• >1Gbit/s requires new
hardware
• Software upgrades long
and intensive
• Feature upgrades
require new hardware…
Clients
Collapsed backbone
2008 onward
• New Best Current Practice:
Concentrate, then duplicate
– Concentrating reduces the hardware,
and with it failure incidence and
scaling cost
– Duplicating provides
resilience, and keeps
network consistent
• Model now being followed by many NRENs
Procurement
Procurement
• Chose procedure
– Competitive dialogue, open, restricted
• PIN notice followed by open procedure
– PIN notice sought input on type of equipment
and architecture, and set expectations
– Proceeded to open RfT based in part on
the input received in response to PIN
• RfT issued December 2006
Procurement
• Responses arrived February 2007
• Thorough evaluation including
on-site testing of candidate equipment
– Single vendor solution chosen
– Biggest single contract in HEAnet
• Extensive environmental requirements
checked with colo sites
• Contract signed August 2007
• Delivery October 2007
• Implementation ongoing as we speak
To implementation
• Detailed transition plan being developed
– HEAnet
– Lan Communications & Cisco
– Being customised with each client's input
• Starting with HEAnet internal
infrastructure
• Identification of project to clients now
– Contact taking place to agree individual
transition plans over the coming weeks
The hardware
and new architecture
Hardware
• Chosen equipment: Cisco CRS-1
• Scalablity
– up to 100Gbit/s interfaces
– 1.5+ Terabits per second total
• Full support for current and
future services
• New operating system platform: IOS-XR
Features
• Support for critical new features
– IPv6 multicast, 4byte ASN, …
• Excellent high availability
– Software is modular
– Possibility of hitless upgrades!
• Top of the field for future growth
– 10Gbit/s connections a matter of course
– Scale to 100Gbit/s peer, multi-10Gbit/s client
Network Architecture
• Two routers, both performing
core and access functions
– One in Citywest, one in Kilcarbery Park
• Where possible, client gets connectivity
via NBE to both routers
– Primary/backup connections
– Resilience a function of the underlying NBE
• Ethernet only, burst up to 10Gbit/s
per interface
Bandwidth scaling
Today
12000
10000
8000
6000
4000
2000
Effective limit of old equipment
0
1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014
Mbit/s
Moore's law
10Gig connections to INEX, GÉANT and
General Internet now being commissioned
The transition
and timelines
Transition
• Careful consultation with each client
• Minimise disturbance to connections
• Where possible, bring up new connections
before deleting old
– BGP policy will assist in this (thank you!)
• Preferentially route traffic over new
network for a test period
– Can revert by shutting down new links
• Transition plan customised for each
client's needs
Transition
Transition
• NBE resilience rollout in parallel
• Clients not directly connected to NBE
should still peer with both routers
– BGP preferred, to protect from fibre cut
or reboot of one or other router
• All clients now transitioning to Ethernet
– Extra flexibility of design & speed over
ATM and serial links
– Connect directly with NBE
– Use VLANs to provide second peering session
Transition
• Testing during November 2007
• 4Gbit/s interconnect with current network
• Slow start with handpicked connections
– HEAnet internal infrastructure first
(MNS - ftp.heanet.ie, videoconf, streaming)
– INEX and backup GÉANT connections
– One client of each technical category,
by agreement, full support from all partners
• Plan: production traffic by Christmas
– Transition in earnest starts in January 2008
– Majority March 2008, decommission June
Impact
• Most transitions should be hitless or
near-hitless
– Will discuss each connection in detail
• If HEAnet manages your CPE
– We will contact you for scheduling
– We will manage the transition
• If you manage your own CPE
– We will contact you for scheduling
and planning of the changes
Thank you!
Questions?
Bonus slides
Timelines
• NOW
• Late Nov
• End Nov
• Early Dec
• Mid Dec
• Jan-Feb
• Feb-Mar
-
Installation
Acceptance tests
HEAnet services
First clients and peers
Acceptance and freeze
for Christmas period
- Next 10 clients
- Remaining clients
and completion
Tasks for transition
• Inform customer of new network design
• Identify local liaison at site
• Determine Local requirements, software
changes, patching
• Agree time schedule for parallel
running/test
• Implement switchover to new connection
• Confirm new connection confirms to
requirements
• Terminate old connection
New IP Backbone
Rollout phase
Nov 2007 – June 2008
Current topology “Rednet”
ar1-cwt
ar2-cwt
ar1-dcu
cr1-cwt
cr1-gal
ar1-tcd
cr1-kp
cr1-lim
cr1-cork
1) Interconnection with the core (Nov 07)
1
CR1-CWT
CR2-CWT
NTL
ESBT
CWT-PE1
KP-PE1
CR1-KP
CR2-KP
1
2) Interconnection with Bluenet (Nov)
1
CR2-CWT
CR1-CWT
2
2
NTL
ESBT
IP link via Bluenet
CWT-PE1
KP-PE1
2
CR1-KP
CR2-KP
1
3)Rehome ESBT dark fibre (Dec)
1
CR2-CWT
CR1-CWT
2
CWT-PE1
ESBT
ESBT
NTL
X
3
KP-PE1
2
CR1-KP
CR2-KP
1
4)Peer with Route Reflectors (Jan 08)
1
4
RR1
CR2-CWT
CR1-CWT
2
CWT-PE1
ESBT
ESBT
NTL
X
3
KP-PE1
4
2
RR2
CR1-KP
CR2-KP
1
4)Peer with Route Reflectors (w/o Bluenet)
4
RR1
CR2-CWT
CR1-CWT
4
RR2
CR1-KP
CR2-KP
5) Dealing with ATM and commodity (Feb 08)
RR1
5
CR2-CWT
AR3-CWT
X
AR2-CWT
CR1-CWT
5
RR2
CR2-KP
AR1-KP
X
CR1-KP
Transition phase 2008
AR3-CWT
CR1-CWT
CR2-CWT
CWT-PE1
GEANT
Clients
ESBT
INEX
GBLX
via Bluenet
New IPTr
KP-PE1
CR1-KP
CR2-KP
AR1-KP
New IP Backbone
Client’s access setup
overview
Client with telco connection to a central PoP
-> NUI Maynooth
Port-Channel1
X
AR1-DCU
G6/2.28
G1/0/1
Eircom
cpe2-nuim
DWDM
nuim-sw1
BLUENET
kp-sw1
dcu-sw1
North
CR2-KP
IP link & BGP
IP link & BGP
BLUENET
cwt-sw1
NEW!
HEAN103306
nuim-sw2
CR2-CWT
Port-Channel1
BT
nuim2
CWT-PE1
HEAN105011
X
G7/5
X
AR1-CWT
South
AT 200711 Slide 39
Client access via Bluenet and cancelled Telco
-> DIT Aungier St (planned)
X
G6/2.22
AR1-TCD
IP link & BGP
G1/0/1
A1/0.28
G0/2
cpe1-dit
G1/0/1
aungiersw1
DCC
fiber
G2/0/10
tcd-sw1
BLUENET
KP-PE1
CR2-KP
?
IP link & BGP
BLUENET
cwt-sw1
CWT-PE1
CR2-CWT
X
Eircom
A3/1.8
AVC502353
atmsw1-cwt
X
AR1-CWT
AT 200711 Slide 40
Client with leased line to Citywest, singlehomed
-> EPA, Wexford
?
F0/0
IP link & BGP
cpe1-epa
BLUENET
kp-sw1
KP-PE1
CR2-KP
?
IP link & BGP
BLUENET
cwt-sw1
CWT-PE1
CR2-CWT
X
HEAN101018
10Mb/s BLI
static
F9/2
X
AR1-CWT
AT 200711 Slide 41
BGP client with full resilience. Will use Bluenet
-> UL (not resilient though)
IP link & BGP
BLUENET
KP-PE1
CR2-KP
IP link & BGP
trunk
BLUENET
X
ul.lim.client
lim-sw1
CWT-PE1
LIM-PE1
CR2-CWT
G0/2
REDNET
CR1-LIM
AR1-LIM
AT 200711 Slide 42
IoT: Limerick IT
IP link & BGP
AR1-GAL
BLUENET
X
KP-PE1
CR2-KP
IP link & BGP
trunk
BLUENET
CWT-PE1
X
cpe1-lit
lit-sw1
lim-sw1
LIM-PE1
CR2-CWT
G0/2
AR1-LIM
AT 200711 Slide 43
IoT: Tipperary Institute
HEAN0111
2Mb/s
S3/2
X
AR1-GAL
converted leased line
BLUENET
KP-PE1
gal-sw1
TippInst
CR2-KP
Clonmel
IP link & BGP
BLUENET
converted leased line
CWT-PE1
cwt-sw1
TippInst
CR2-CWT
Thurles
IP link & BGP
HEAN0109
STM-1
A3/1.44
X
AR1-CWT
AT 200711 Slide 44
IoT: Limerick IT
IP link & BGP
AR1-GAL
BLUENET
X
KP-PE1
CR2-KP
IP link & BGP
trunk
BLUENET
CWT-PE1
X
cpe1-lit
lit-sw1
lim-sw1
LIM-PE1
CR2-CWT
G0/2
AR1-LIM
AT 200711 Slide 45