Transcript Ullmann_Xwin

Provision of networks for the
LHC experiments in Germany
- Status 03/2006 K. Schauerhammer, K. Ullmann, (DFN)
2. / 3.3.2006
Meeting GridKa TAB
Karlsruhe
Contents
I. Technical options X-WiN and Geant2
II. Architecture LCG - Network (T0 - T1)
III. Performance Measurements
DFNInternet
IV. Connecting T2 centres in Germany
Seite 2
I.
Technical options X-WiN and Geant2
Seite 3
X-WiN - status as of 1. 1. 2006
• Technical concept backbone (after Europe wide
tender)
–
–
–
–
backbone consists of (dark) fibre and leased circuits
operational responsibility: DFN
more (than before) bought in (partial) services
much more (than before) DFN PoPs
• Access-Network very similar as in the past
• Economic concept
– backbone put together from different service offerings from
the market and integrate that under DFN responsibility
– most effective usage of competition on the market
Seite 4
X-WiN (optical platform)
KIE
Dark fibre
50km to
Enschede
(Surfnet/NL)
Leased circuits
HAM
BRE
BIE
MUE
BRA
LEI
MAR
BIR
GIE
FRA
WEI
50km to
Kehl/Straßburg
(Renater/FR)
80km to
Frankfurt/O
(Pionier/PL)
DRE
JEN
CHE
BAY
ESF
HEI
FZK
120km to Zürich
(Switch/CH, GARR/IT)
ADH
ILM
GSI
KAI
MAG
ZIB
HUB
GOE
KAS
FZJ
TUB
POT
HAN
DUI
AAC
GRE
ROS
DES
ERL
REG
STU
GAR
43 DFN PoPs
20.02.2006
Seite 5
Optical platform based services
• Toolbox for the provision of
– DFNInternet (2Mbps to 10 GE)
– VPN-Services based on optical links
– services like DFNVC, DFNPKI, DFN-CERT,
DFNRoaming, DFNNews etc. unchanged
• New cost structures for optical networks
enable economic solutions for specialised
services (Optical Private Networks / OPNs)
i.e. Grids
Seite 6
General targets X-WiN design
• More performance
performance increase by a factor of 4 since 01/06
more performance available (for the same price)
as of 01/07
• More flexibility
no volume charging (and limitation)
Ethernet as additional access technology
Hybrid PoPs enable VPNs
• More availibility
during design for backbone implicitly taken into
account
Seite 7
Availibility backbone by means of ...
KIE
• strongly protected
physical topology
ROS
DES
HAM
BRE
– protected fibre paths
BIE
MUE
– meshed fibre ring structure
DUI
– more mathematical FZJ
AAC
BIR
optimisation will show,
how
to meaningful complement FRA
GSI
physical topology
HEI
FZK
TUB
POT
HAN
BRA
MAG
ZIB
LEI
HUB
ADH
DRE
JEN
CHE
ILM
BAY
ESF
ERL
REG
STU
GAR
Seite 8
Availibility backbone by means of...
• Fault tolerant logical topology,
especially for DFNInternet
• Well-known optimisation for
– logical links between PoPs
– best mapping to physical topology
• Optimisation targets, especially
– minimal delay
– minimal loss of connectivity in case of severe
failures
– best („smooth“) utilisation in case of failures
Seite 9
X-WiN DFNInternet as of 01/06
Logical IP backbone
Seite 10
Availibility backbone by means of...
Redundant global connectivity
worldwide Internet
At least two accesses to global IP
networks, more redundant carriers
Full throughput in case of failures
Additional roughly 100 direct peerings
X-WiN
Redundant global connectivity
by means of independant paths
and PoPs assures protection
against problems with Carriers
worldwide Internet
(e.g. network failures or
insolvency)
User
Seite 11
Dark Fibre
• Technical definition
– ITU-T conform and "WDM-applicable"
– some minimal requirements on technical
parameters
• for example attenuation
• Service:
– Support : 24/7 hotline and debug
– provision of colocation space
– End-to-end operational responsibility
– usual time for reaction
– integration into DFN defined processes
Seite 12
Equipment to „light“ Dark Fibre
• Technical definition:
– equip with (several optional) digital interfaces
– provision of a management system
• Service:
– includes installation and maintenance
– support: 24/7 hotline and debug
– usual time for reaction
– integration into DFN defined processes
Seite 13
Operations
• Definition
– Monitoring of management data
– pursue well defined procedures
•
•
•
•
failure identification (devices? - links?)
alarm of responsible support groups
supervision of debug
if necessary escalation
– all procedures DFN defined
– regular (i.e. monthly) reporting
– open for integration of new devices
Seite 14
Operational model X-WiN platform
Operations Group
alarms and triggers support
(links, devices)
Access to management data
of DFN „owned“ devices
dark fiber
PoP B
PoP A
Leased circuit
PoP C
Seite 15
GÉANT2 footprint
EE
2
IE
DK
LV
2
LT
NL
2
2
UK
3
BE
1
DE
PL
2
3
3
3
1
1
2
PT
3
1
CZ
LU
FR
SK
CH
3
2
2
RU
MT
2
IT
ES
1
2
RO
AT
HU
2
2
SI
2
BG
TR
HR
GR
1
IL
CY
Seite 16
GÉANT2 Hybrid POP
to NREN
10G N x 10G
Leased Service N x 10G 
N x 10G 
Dark Fibre
GÉANT2 POP
Dark Fibre
Seite 17
Gigabit Ethernet p2p Service
GÉANT
Border
GÉANT
Border
NREN A
GÉANT2
Interconnect:
N x 10GEth
Interconnect:
N x physical GEth
NREN B
GEth (GÉANT2 transport)
GEth (NREN transport)
Physical GEth
VLAN/10GEth
(LAN or WAN)
Seite 18
II. Architecture LCG - Network (T0 - T1)
Seite 19
Architecture LCG network (1)
• Assumptions, design criteria:
– bandwidths 10Gbit/s per T1 resp. multi 10Gbit/s
for T0
– continuous stream of data
– „keep it simple“ (as possible)
– organise transport if possible in layer 2
– security already considered during design
• system should be protected from unauthorised access
• access from “trusted sources” not limited
Seite 20
Architecture LCG Network (2)
• Optical Private Network, consists of
dedicated 10G links from T0 and any T1,
• T1 access in two options:
– “Light path T1”
– “Routed T1”
• Back-up for T0-T1 is part of the design
• T0 interface very likely 10GE LAN-PHY
Seite 21
Scalability of the architecture
• Options for transfer:
– 10 Gbit/s ~ 1014 byte/d or 100 Tbyte/d
– eleven 10G links -> more than 1 Pbyte/d or
roughly 0.5 Exabyte/a
• If one 10G link per T1 is not sufficient:
– Installation of just another 10G link T0 - T1 if
possible on physically separated fibre path
– architecture covers this completely
Seite 22
LCG T0 - T1 OPN
Seite 23
III. Performance Measurements
DFNInternet
Seite 24
Measurements DFNInternet
Taken from:
Issues in setting up a 10Gbit
connetion between GridKa
and CERN openlab
Forschungszentrum Karlsruhe GmbH
Institute for Scientific Computing
P.O. Box 3640
D-76021 Karlsruhe, Germany
http://www.gridka.de
Bruno Hoeft
Seite 25
WAN 2003/4 -- Gigabit GridKa – CERN
10 Gbps
Frankfurt
Géant
Geneva
1000
DFN
2.4 Gbps
98% of 1Gbit
Karlsruhe
2x 1 Gbps
GridFTP
server
GridFTP tested
over 1 Gbps
0
CERN
GridFTP
server
Seite 26
10Gigabit WAN SC GridKa – CERN
10 Gbps
Frankfurt
DFN
10 Gbps
Karlsruhe
10 Gbps
GridFTP
server
Géant
Geneva
- Bandwidth evaluation (tcp/udp)
- MPLS via France
(MPLS - MultiProtocol Label Switching)
- LBE (Least Best Effort)
- GridFTP server pool HD to HD
Storage to Storage
- SRM
CERN
10 Gbps
GridFTP
server
Seite 27
Evaluation of max. throughput
Mbit/s
7000
6300
9 nodes each site
-8 * 845 Mbit
-1 * 540 Mbit
higher speed at one stream
is resulting in a packet loss
5600
4900
4200
3500
2800
2100
1400
700
18:00
20:00
Seite 28
IV. Connecting T2 centres in Germany
Seite 29
LCG network services in Germany
• T0-T1 problem
– 10 Gbit/s over European VPN (GridKa - CERN)
– backup via CNAF or SARA
• T1-T2 problem
– within DE no problem as most LHC sites are
close to X-WiN PoPs
– for T2 in DE rough guess of traffic flows are
needed (source, destination, transferred volume
per day, peak requirements?)
Seite 30
LCG Sites DE and
X-WiN
ROS
DES
HAM
BRE
ZIB
MUE
HAN
BIE
DOR
DUI
KOE
KAS
WUP
LEI
DRE
BOC
SIE
CHE
BIR
AAC
BON
FRA
MAI
X-WiN-Nodes
BAM
GSI
MAN
ERL
HEI
FZK
STU
REG
AUG
GAR
FRE
TUM
LHC User Locations
at X-WiN-Nodes
LHC User locations
to be connected to
X-WiN nodes
Seite 31