Transcript Diapositiva 1

Anella Científica:
A virtualized research network
Maria Isabel Gandía Carriedo
Communications Department, CESCA
Terena Networking Conference
Universidad de Málaga, 11/06/2009
Agenda
 Anella Científica
• Introduction
• Functionalities
• Evolution
 The new core of the network
•
•
•
•
•
Topology and equipment
Technology: virtualization in the core
The management
During the deployment
After de deployment
 Examples of use
Agenda
 Anella Científica
• Introduction
• Functionalities
• Evolution
 The new core of the network
•
•
•
•
•
Topology and equipment
Technology: virtualization in the core
The management
During the deployment
After de deployment
 Examples of use
About CESCA and Anella Científica
 Public consortium
 Created in 1991
 Formed by:
Commercial
Internet
• Generalitat de Catalunya
• Fundació Catalana per a la
Recerca i la Innovació
• 9 Catalan universities
• Consejo Superior de
Investigaciones Científicas
 Anella Científica created in 1993
 CATNIX created in 1999
About CESCA and Anella Científica
Anella Científica is the high-speed
communications network that
connects the universities and
research centres in Catalonia
(“Scientific Ring”)
About CESCA and Anella Científica
CESCA, as the manager of the Regional Research
and Education Network (RREN) in Catalonia and
as a Local Internet Registry (LIR) has:
• Addresses for the connected institutions:
– IPv4: 84.88.0.0/15
– IPv6: 2001:40B0::/32
• An Autonomous System (AS):
– AS13041
CESCA and Anella Científica provide services...
 For all the universities and research community
 not only Ethernet
 For all Catalan universities and researchers, not only in
Barcelona area  not everyone has fibre
 Even for enterprise researchers  not only RedIRIS
members
 For special projects and regular traffic  special circuits,
dedicated fibres, QoS
 (Unicast || Multicast) && (IPv4 || IPv6)
The Anella Membership Policy
Own connection
A
B
C
A.1
Public and Private
Universities
B.1
Manager of R+D+i
Program with Public
Funding
C.1
Technological and
Scientific Parks
A.2
Public Research
Organizations
& LSF
A.3
Research Centres
and Institutes
A.4
Hospital Research
Departments
B.2
Institutions with
Relevant Digital Contents
for the Scientific and
Technical Community
B.3
Entities Participating
in R+D+i Projects
B.4
Entities of Special
Interest
C.2
Other Hospital
Departments
Anella Científica: connected institutions
A.1
A.2
A.3
A.4
B.1
Linked
UB
BSC
ASPB
CHV
CIDEM
BGSE
UAB
CESCA
CAR
CSPT
FCRI
CESC
UPC
CIEMAT
TERMCAT
FCRB
UPF
CELLS
CTTC
IDIBELL
B.2
EUPMT
UdG
CSIC
CTFC
Guttmann
BAdM
FIGTP
CRG
Puigvert
BC
FUB
URV
2-02
CETI
UdL
C.1
FBM
SantPau
UnescoCAT
IMIM
UOC
ParcUdG 20-01
i2CAT
VHebron
CConsultiu
RI
URL
PRBB
ICC
Dexeus
CBUC
UVic
ICIQ
UIC
ICFO
C.2
Liceu
UAO
IdeG
FHAG
VINSEUM 01-01
UdA
IDESCAT
XarxaTecla
ESMUC
IEC
EUSS
IEEC
CatSalut
INEFC
IFAE
XTEC
CDP
B.4
21-04
IRTA
IGC
PAM
≥ 1.000
≥ 100
≥ 10
≤8
16-02
Anella Científica: projects
 PIC participates in LHC (10 Gbps)
 i2CAT participates in several European projects:
FEDERICA, Phosphorus,… (10 Gbps)
 UPC-CCABA participates in EuQoS, MUPBED,… (1 Gbps)
 New card and 10 Gbps for PASITO
 Two temporal increases of BW for FCRB at 100 Mbps
(DVTS)
 Liceu transmits the Opera Oberta course
Anella Científica: Evolution
Time
period
1993-1998
Name &
logo
ANELLA
CIENTÍFICA
Points of
access
Technology
BW (Mbps)
Dedicated
circuits
6-8
Ethernet
over DQDB
10 (34 Mbps
core)
No
1998-2001
15-19
IP/ATM
34-155 (622
Mbps core)
Yes
2001-2003
19-37
IP/ATM
34-155 (2,5
Gbps core)
Yes
2003-2008
37-76
Ethernet
over WDM
10-10,000
Yes
2008-2011
76-...
IP/DWDM
10-10,000
...
Yes
Changes in the network
 Anella Científica had a central node with separated L3
equipment for institutions type A+B and C.
 Level 2 and 3 equipment was the same for most of them.
 There were several L2 projects involved.
 In 2008, both the network and the equipment had to be
renewed.
 Two Calls for tenders to be submitted in July 2007:
one for the network and one for the equipment.
 Survey among our users to know their needs.
The survey for the last version of Anella Científica
 Goal: plan the evolution of the RREN (Regional Research
and Education Network) and prepare the call for tenders.
 Survey was sent in December 2006 to 62 institutions.
 We asked about:
•
•
•
•
Connection needs
Evaluation of our services
Evaluation of our dissemination activities
Planning of new deployments
Results of the survey: BW & points of access
2006
2008
2011
2013
73
68
69
70
4
11
17
23
100 - 500 Mbps
16
16
17
14
10 - 100 Mbps
32
30
24
22
 10 Mbps
21
11
11
11
5,242
11,956
21,625
47,740
Points of access
 500 Mbps
Aggregated BW (Mbps)
 Sustained and progressive growth
 Regular traffic connections
•
•
Numbers of points of access didn’t
increase considerably
Growth with the access of the Health
and Culture Departments
 Special projects
2006
2008
11
12
 10,000 Mbps
0
1
1,000 – 10,000 Mbps
4
5
100 – 1,000 Mbps
2
2
 100 Mbps
5
4
4,613
19,211
Local connections
Aggregated BW(Mbps)
Conclusions of the survey
 Requirements for the new Anella Científica:
• Increase the BW
• Asymmetric and flexible model to cover different needs
• Adaptable topology, specially for relevant projects
 More reliability: redundancy in the core nodes,
in the accesses, etc.
 Evolution towards a distributed network:
• Two points of presence interconnected
• L2/L3 equipment redundant
 Connections:
• Own fibre
• Rent fibre to a operator
• Bandwidth from an operator
Anella Científica: 2006
10 Gbps
1 Gbps
100 Mbps
34 Mbps
10 Mbps
Operator
C. Nord
Internet
≤ 8 Mbps
Anella Científica: 2008
A
B
C
1. Public and private non-profit Universities
2. Official Bodies of Research
3. Other non-profit Research centres
4. Hospital Research centres
1. Official bodies of R+D management
2. Relevant Digital contents institutions
3. R+D+i participants
4. Special interest for R+D institutions
1. Science and technological parks
2. Other hospital units
Operator
C. Nord
Telvent
Internet
Agenda
 Anella Científica
• Introduction
• Functionalities
• Evolution
 The new core of the network
•
•
•
•
•
Topology and equipment
Technology: virtualization in the core
The management
During the deployment
After de deployment
 Examples of use
Previous topology
Local
connections
Special
projects
Operator
Nortel
Level 2 (RedIRIS)
Catalyst 6513
Level 2/3
A+B institutions
Cisco 7200
Level 3
C institutions
C. Nord
Telvent
Juniper M320
Level 3 (RedIRIS)
Internet
Topology: what we asked for
Local
connections
Special
projects
Operator
RedIRIS
Level 2
Annexus
CESCA-CN
CESCA-T
Level 2
Level 2
Level 3
Level 3
Telvent
RedIRIS
Level 3
Level 3
equipment
The equipment we asked for
 It had to:
• Be modular, scalable, with redundant power supplies, manageable
via SNMP,…
• Support BW reservation, QoS, MPLS, multicast, IPv6, SPAN ports
• Have good performance
• Permit 802.1q VLAN, jumbo and baby giant frames, VLAN stacking
• Have filtering mechanisms like access-lists
 Virtualization was an improvement
The equipment we asked for
 The old equipment could be re-used, improved, its cards
could be replaced… It could be used in any of the nodes,
for level 2, level 3 or both.
 We explained all its characteristics (in fact, the same we
were asking for!)…
 …Maybe that was the reason why in the 3 offers we had,
the equipment we were offered was from the same vendor,
with different configurations, but always with two new
equipment and reusing the previous switch/router.
 We chose the more advantageous: separated L2 and L3
equipment in one node, same equipment in the other node.
Final topology
Local
connections
Special
projects
Operator
Nortel
Level 2 (RedIRIS)
DWDM
10 Gbps
Catalyst 6513
Level 2
Giganet
2 x1 Gbps
Catalyst 6509
Level 2/3
Catalyst 6509
Level 3
C. Nord
Juniper M320
Level 3 (RedIRIS)
Telvent
The new core network
 Core with two nodes linked with 10 Gbps DWDM + 2Gbps
 CESCA-CN
• Segmentation of functionalities level 2 and 3:
• Catalyst 6513 Sup 720-3B (L2)
• Catalyst 6509 Sup720-3BXL (L3)
 CESCA-T
• Catalyst 6509 Sup720-3BXL (L2/L3)
 Use of Virtual Routers
 Easy creation of new virtual routers (projects,
management, pre-production, …)
 Separation of routes
 Layer 3 Balancing/backup
 More flexibility, redundancy and reliability
Why virtualization?
 It permitted to have many different routers in each node for:
•
•
•
•
Different types of institutions (A, B and C).
Pre-production environment.
Special projects.
An institution with many points of access that needed a separated
router for its Campus LAN.
 It allowed us to take an ISP approach more than a Campus
approach.
 We had the backup control on the L3 network. Some parts
of the L2 network are under the control of the Operator.
 Having more nodes would be easier.
Changes
C. Nord
C. Nord
Telvent
Virtualized core network
MPLS
@REDIRIS
CATNIX
@REDIRIS
CATNIX
@CESCA A, B
@CESCA C
INET-ALPI
OSPF
@CESCA A, B
CESCA-CN
@CESCA C
INET-ALPI
Core
Network
CESCA-T
Logical architecture
Providers
ISP A
REDIRIS BCN
ORANGE BCN1
@REDIRIS
@CESCA, A, B
ISP B
ISP C
CATNIX
@CESCA, C
INET-ALPI
REDIRIS VAL
@REDIRIS
Core
network
@CESCA, A, B
CATNIX
@CESCA, C
INET-ALPI
CESCA-T
CESCA-CN
Institutions
ORANGE BCN2
@CESCA, A, B
@REDIRIS
@CESCA, C
INET-ALPI
Routing IPv4 (example)
eBGP Anella RedIRIS-VAL
RedIRIS
València
iBGP institutions
OSPF infrastructure
eBGP Anella
RedIRIS-CAT
RedIRIS
Barcelona
CESCA-CN
eBGP Anella
CATNIX
BGP RedIRIS
CATNIX
Redundancy
• Between nodes of CESCA
• with RedIRIS (or Orange)
• at CATNIX
CESCA-T
eBGP CESCA-CN
Institution
eBGP Anella
CATNIX
eBGP CESCA-T
Institution
Institution
The management (I)
 It is more complex
•
•
•
•
More equipment involved
More dynamic routing (with providers and institutions)
More interfaces to connect with the operator (from 2 to 20)
More VLAN (from 150 to 270)
 New syntax, remember not to forget VRF!
 Some “false friends”…the legacy commands…
• show ip bgp summary (there are no neighbours!) -> show ip bgp
vpnv4 vrf <NAME> summary
• ping 10.1.1.1 (oops! It doesn’t work) -> ping vrf <NAME> 10.1.1.1
The management (II)
 The real friends:
• Aliases!
alias exec bgp-ri-adv sh bgp vpnv4 unicast vrf <NAME> neighbors
10.1.1.1 advertised-routes
alias exec pir ping vrf <NAME1>
alias exec pia ping vrf <NAME2>
alias exec par ping vrf <NAME3>
• Includes, but do not abuse…
show vlan | inc <INSTITUTION>
show int desc | inc IP6
…
show ip bgp vpnv4 all | inc 10.1.1.0 (No!!)
• Work with a text file, not with the configuration directly 
During the deployment (I)
 In the Catalyst 6500, by default, all the interfaces have the
same MAC address!!!
cs-6506-24a#show interfaces | include line | address
Vlan1 is down, line protocol is down Hardware is Cat6k RP Virtual Ethernet, address is
00d0.bcf1.ee5c (bia 00d0.bcf1.ee5c) Internet address is 14.18.2.182/16
Vlan2 is down, line protocol is down Hardware is Cat6k RP Virtual Ethernet, address is
00d0.bcf1.ee5c (bia 00d0.bcf1.ee5c)
If the same equipment has more than one interface on the
same VLAN (for instance, one with IPv4 and one with IPv6)
there are random flaps, packet loss, etc.
During the deployment (II)
 Address-family ipv4 multicast is not supported in the VRF:
router(config)#router bgp 13041
router(config-router)#address-family ipv4 multicast vrf TEST
^
% Invalid input detected at '^' marker.
Static routing in only one VRF for multicast.
 VRF IPv6 are not recommended:
router(config-router)# vrf upgrade-cli multi-af-mode
...
Experimental command, not supported by Cisco.
We keep IPv6 in the physical router rather than in a VRF.
Other opened issues
 Tagging of the routes per-VRF and not per-prefix.
Supported by Cisco, but not much documentation:
• mpls label mode vrf <vrf> protocol bgp-vpnv4 per-vrf
 It would be desirable to “enter the configuration” of each
VRF instead of writing the VRF each time you write a
command.
 The name of the VRF can not be changed!!
After the deployment
 Two bugs affect the IOS causing a memory leak and
increasing the CPU (workaround: avoid “show run”
commands ).
 Maximum number of routes is approximately 106.
If the routes increase considerably, we may face a problem.
You can not have so many full-routings.
Agenda
 Anella Científica
• Introduction
• Functionalities
• Evolution
 The new core of the network
•
•
•
•
•
Topology and equipment
Technology: virtualization in the core
The management
During the deployment
After de deployment
 Examples of use
Case study: UAB
UAB–CS
UAB (IFAE/PIC)
UAB (Bellaterra)
DWDM 10 Gbps
Giganet 1 Gbps
HUVH (unitat docent)
Giganet 100 Mbps
Giganet 500 Mbps
FUB
FIGTP
ADSL 4 Mbps
Ethernet 20 Mbps
C. Nord
UAB-IGOP
Telvent
ADSL 4 Mbps
CESC
PRBB (unitat docent)
Giganet 100 Mbps
Giganet 100 Mbps
HSP (unitat docent)
Giganet 500 Mbps
i2CAT
HSP (Casa Convalescència)
Giganet 500 Mbps
VLAN from each point of access to CESCA-CN & CESCA-T (internet)
VLAN from Bellaterra to CESCA-CN for multicast
VLAN each point of access to Bellaterra forn internal routing
VLAN with i2CAT for the Electronic Learning Singing project
VLAN PIC for the LHC project
Case study: IRTA
IRTA-MV
IRTA-CT
IRTA-CM
IRTA-CO
IRTA
IRTA-VA
IRTA-CA
IRTA-MO
@REDIRIS
INET-ALPI
IRTA-SC
CESCA
IRTA
 IRTA has 9 points of access with different
Intra-institution connections technologies, from ADSL to FastEthernet
Internet access
 The point of access in Barcelona makes NAT
and filters all the connections, and all of them go to
the internet through it
Thanks for your attention!
Questions? Suggestions?
[email protected]