Transcript DV-11-018-GEANT_and_EGIx

GÉANT and EGI: enabling seamless
distributed computing infrastructures through
high-speed, dedicated networking
Richard Hughes-Jones, Domenico Vicinanza, DANTE
EGI User Forum 2011
11-14 April 2011, Radisson Blu Hotel Lietuva, Vilnius
connect • communicate • collaborate
The background: what changed in
the past 15 years…
Situation:
Increasing cross-border collaboration between researchers
across the world
Sharing experiences, resources and facilities
= sustainability
Science, research and education uses high-capacity
computing to:
model complex systems
process and analyse experimental results
and deliver them in real time in different locations around
the globe
connect • communicate • collaborate
The need for Computing GRIDs
Response:
Computing grids for e-Science
Research and Education (R&E) Networks
Synergy
Together, they allow:
to respond to the requirements of the most demanding
scientific disciplines
to link and combine the power of distributed computation
and the reliability of dedicated networks.
connect • communicate • collaborate
EGI infrastructure lives in a
Multi-Domain network environment
WN1
UI
CE
WN2
Domain 3
WN3
Domain 1
Others
domains
RB
WMS
Domain 2
SE
Domain 4
connect • communicate • collaborate
In more detail…
GÉANT/NRENs interconnect NGIs
National
R&E
network
GÉANT
National
R&E
network
Regional
network
Regional
network
Campus
network
EGI/NGI site
EGI/NGI site
Campus
network
connect • communicate • collaborate
GÉANT: the pan-European network
for R&E communities
GÉANT is the pan-European data network dedicated to the
research and education community
GÉANT (with the European NRENs) connects 40 million
users at over 8,000 institutions across 40 countries.
An ideal infrastructure for scientific and research
applications with demanding network needs:
High capacity (stable sustained bandwidth)
Low latency (or one-way delay)
Low jitter (latency or one-way delay variation)
Complex Applications are being built on top the network
Collaborative tools, videoconferencing
Distributed computing (GRIDs)
connect • communicate • collaborate
GÉANT topology
• 15+ NRENs
interconnected
within the Dark Fibre
(DF) “cloud”
• The others, via
“lambda” and SDH
circuits
• Hybrid architecture
(routed IP traffic and
switched specialpurpose traffic)
connect • communicate • collaborate
Reliability
Research projects using GRIDS rely on EGI/NGIs
infrastructure
EGI/NGIs is a distributed computing facility
 It relies on the network connections between:
– computing elements
– storage elements
– workload management servers
– logging and bookkeeping servers
– grid user interfaces
The reliability of EGI/NGIs is directly related to the
reliability of GÉANT and the NREN networks
connect • communicate • collaborate
High Speed Global Network
GÉANT global connectivity and projects
connect • communicate • collaborate
GÉANT coverage and EGI footprint
GÉANT coverage:
Europe
International links to networks in
– North America
– Latin America
– Africa
– Middle East
– Southern Caucasus
– Central Asia and the Asia-Pacific region
The footprint of the GÉANT and the NREN networks
enables a seamless, reliable infrastructure for distributed
grid computation.
connect • communicate • collaborate
Support: The GÉANT Project
Liaison Team
GÉANT project is working in collaboration with EGI
to constantly improve the research experience of
scientists
Dedicated team in GÉANT to Projects Liaison and Support
Focuses on providing assistance and support to projects,
initiatives and organisations (such as EGI)
to facilitate and encourage a productive use of Research
and Education Networks
helping users to understand their network needs
supporting projects having connectivity issues
connect • communicate • collaborate
CONNECTIVITY SERVICESNECTIVITY
SERVICES
connect • communicate • collaborate
Dedicated connectivity services
1/2
Demanding projects require dedicated, specialised network
connectivity services
GÉANT offers three connectivity solutions:
Routed IP
– GÉANT IP
Point-to-Point Circuits:
– GÉANT Plus
– GÉANT Lambda
connect • communicate • collaborate
Dedicated connectivity services
2/2
GÉANT IP – high bandwidth connectivity for millions of
academic users through NRENs via the shared GÉANT IP
backbone network.
GÉANT Plus and GÉANT Lambda
– point-to-point circuit services
– dedicated bandwidth and guaranteed QoS.
GÉANT Plus – flexible allocation (155Mb/s to 10Gb/s)
– user-dedicated point-to-point connections
GÉANT Lambda – full 10 Gbps wavelengths
– to support projects with particularly demanding
network requirements
connect • communicate • collaborate
MONITORING
connect • communicate • collaborate
Multi-domain Monitoring for EGI
GRID computing in a geographically distributed
environment
 Multi-domain monitoring solution
EGI Network Support Group is liaising with GÉANT about
perfSONAR
http://perfsonar.geant.net
connect • communicate • collaborate
perfSONAR for EGI
perfSONAR is:
Naturally multi-domain:
Seamlessly covers multiple network domains
Open
NGIs may choose to install what they need for their
troubleshooting
NGIs may develop high-level tools using APIs
Based on a OGF standard, open protocol
Allowing a flexible access to network measurements
Free
No fee so it naturally fits in any open GRID business
model
connect • communicate • collaborate
perfSONAR for EGI
perfSONAR enables:
proactive monitoring of critical links between GRID
components
Making EGI NOC able to intervene to detect deteriorations
before disruption
Improving overall perception of the EGI/NGI resources
connect • communicate • collaborate
Examples:
Some GRID projects already liaising with
GÉANT
HIGH ENERGY PHYSICS
ART/HUMANITIES
BIO-MEDICAL SCIENCES
connect • communicate • collaborate
LHC Optical Private Network
built with GÉANT Lambdas
ATLAS
CMS
ALICE
lhcb
connect • communicate • collaborate
Volcano Seismogram sonification:
EGI + GÉANT synergy example
The idea: making audible the volcano seismograms
The ear better than the eye in detecting changes
Transforming the volcano seismograms into music
correlate volcano stages with melodies and/or sounds
The final goal: the “signature tune” of an eruption
Volcano scores are also musically interesting!
They inherit the richness of the nature
Huge amount of computing power to calculate sounds
GRID computing (EGI)
Advanced networking for reliable data transfer from
seismic stations in the world to computing nodes
connect • communicate • collaborate
Volcano sonification application:
The Mountain, music from 4 volcanoes
The Mountain: world premier 10-11 Sept 2009, The Kennedy Center, Washington
DC, USA
4 volcanoes: Etna, Tungurahua, Pinatubo, Mayon
Data transferred across GÉANT network, TEIN3, ALICE2 intercontinental links
Computation on EGEE III (now EGI) infrastructure
Music by Domenico Vicinanza; choreography by Jason Garcia Ignacio
Info: www.volcanodance.org
connect • communicate • collaborate
Ancient Instrument Reconstruction
and real-time network performances
New artistic/cultural events based on remote collaborations
fusing experiences and cultural backgrounds
Live, real-time distributed performance
Stockholm, 1-2 December 2009
Two continents, two cultures 9,300 Km apart
– joined through a music and dance performance
– made possible by advanced connectivity
Two ancient instruments virtually reconstructed
– made possible by distributed computing
connect • communicate • collaborate
The intercontinental link between
London and Singapore
GÉANT-TEIN3 network link:
2.5 Gbit/s optical connection
Madrid - Singapore
spanning almost 10,000 Km.
TEIN3: third generation of the
Trans-Eurasia Information Network
(TEIN)
High-capacity Internet for research
and education communities across
Asia-Pacific.
connect • communicate • collaborate
Two special instruments on stage
Two very special musical instruments on stage: the Epigonion and the
Barbiton
Ancient Greek instruments reconstructed by the ASTRA project using
physical modelling
Computer virtual models based on archaeological findings
Sound reconstruction and 3-D modelling on EGI Infrastructure
The Epigonion (a)
and the Barbiton (b)
Photo courtesy of F. Baghino and
F. Ugozzoli, VisArc Studio, Parma,
Italy for ASTRA
connect • communicate • collaborate
On stage, from Stockholm
Dancers real-time video
from Kuala-Lumpur
via GÉANT-TEIN3 networks
Epigonion and Barbiton
Reconstructed on EGI
GÉANT Launch performance from Stockholm.
Epigonion+Barbiton reconstructed on EGI, played by the first musician
on the left
Malaysian dancers choreography projected in the upper right corner,
behind the musicians
connect • communicate • collaborate
Video available on www.geant.net
neuGRID/outGRID
neuGRID: new user-friendly Grid-based research eInfrastructure for bio-medicine
Collection and archiving of large amounts of imaging data
Computationally intensive data analyses of the images
Advanced visualisation of the results.
Large sets of brain images to identify neurodegenerative
disease markers, for example, for Alzheimer’s disease
through the analysis of 3D magnetic resonance brain
images
The Grid infrastructure is based on EGI GLite middleware.
connect • communicate • collaborate
Collaboration between GRIDs and
R&E Networks
The outGRID project aims at supporting three e-infrastructures
neuGRID in Europe
CBRAIN in Canada
LONI at UCLA in the USA
to converge into one unique worldwide facility thanks to
Distributed Computation (EGI) & International Connectivity (GÉANT)
GÉANT Project Support team is liaising with outGRID to improve network
performances across the Atlantic
connect • communicate • collaborate
Conclusions
Computing grids and Research & Education (R&E) Networks are
increasingly growing
to respond to the evolving requirements of demanding scientific and
education disciplines.
Synergy: power of geographically distributed computation + reliability of
advance networks dedicated to the research community.
Advanced R&E networking provides the reliable connectivity which
enables
scalable, sustainable distributed grid computing infrastructure in
Europe and other parts of the world.
GÉANT supports EGI projects through its Liaison and Support team
providing assistance and support to facilitate and encourage
productive use of the network.
connect • communicate • collaborate
Thanks!
Questions?
NRENs
GÉANT
EGI
NGIs
Sustainable,
Collaborative
Research
connect • communicate • collaborate
Extra slides for Questions
connect • communicate • collaborate
GÉANT IP
The GÉANT IP service offers NRENs access to the shared European IP
backbone.
IPv4 and IPv6 support
VPN, Multicast, Premium IP
Robust high-bandwidth solution to the international connectivity
requirements of the majority of academic users.
Resilient service in the case of hardware failure or fibre cuts
Advanced routing equipment to ensure fast recovery from unexpected
events.
GÉANT IP access is available to NRENs at capacities of up to 20 Gbps,
subject to technical and commercial considerations.
connect • communicate • collaborate
GÉANT Plus and GÉANT Lambda
GÉANT Plus
User access to point-to-point circuits of between 155 Mbps and 10 Gbps across
an existing pre-provisioned network.
Dedicated sub-wavelength point-to-point circuits configured over a network of
dark fibre links and TDM (Time-Division Multiplexed) switches.
Circuits can be established to many European NRENs
Allows NRENs to configure transatlantic circuits to the GÉANT point-ofpresence in New York (connecting to Internet2, ESnet and USLHCnet).
GÉANT Lambda
It provides private, transparent 10 Gbps wavelengths between any two GÉANT
NRENs connected to the GÉANT dark fibre cloud.
Available to pan-European projects and data intensive users via NRENs with
access to GÉANT dark fibre.
A GÉANT Lambda is presented to the NREN as a transparent wavelength on
which they can then develop their own higher-level network layers.
connect • communicate • collaborate
The new perfSONAR (1/2)
Since past eight months perfSONAR is undergoing a major restructuring
What’s new:
Use-case driven design
– User-centric rather then technology-centric
Improved testing ( improved quality)
– New, open, shared testing policies and results
Agile/Scrum based software development
– Fast release cycles (3 months)
Product management function
– To act as an user advocate and make sure to correctly steer the
development
New website being designed
First (new) perfSONAR release foreseen for Spring 2011
connect • communicate • collaborate
The new perfSONAR (2/2)
Interoperability with US and Canada network monitoring
New perfSONAR designed around interoperability principles,
enabling:
– intercontinental network troubleshooting in multi-domain
environment,
– Dynamic circuit monitoring and
– SLA verification
Closer interaction with the user community
User panel to provide feedback and input to continuously improve
the produce
connect • communicate • collaborate
List of perfSONAR functionalities
interesting for EGI
Functionality or metric provided
GRID Node-Node Link availability
Available bandwidth
(To calculate weights for WMS)
One Way Delay and Jitter
(To determine GRID Node-Node Link quality and to
calculate weights for the WMS)
Measurement Points Discovery
perfSONAR component
E2EMon MP
RRDMA
Historical: BWCTL MA
On-demand: BWCTL MP
Historical: HADES MA
On-demand:To be developed
LS
(To be able to determine what CEs, SEs, RBs are
perfSONAR enables and able to deliver network
measurements)
Authentication/Authorization
AS
(Ideally based on X509 Certificates)
X509 compliance:To be developed
perfSONAR User Interfaces
perfsonarUI, WebCNM
Traceroute (from MP to any GRID Node) To be developed
Ping (from MP to any GRID Node)
To be developed
connect • communicate • collaborate
Data Sonification: Dancing on
Volcano Scores!
Data sonification is the representation of data by sound
signals (melodies, sounds, …)
It can be considered as the acoustic counterpart of data
graphic visualization
10
15
15
10
8
4
1
1
4
connect • communicate • collaborate