Transcript GRID services

Laboratório de Instrumentação e Física Experimental de Partículas
LIP Computing Overview
Jorge Gomes
LIP workshop 2010
1
Computing at LIP
• Comprehensive set of activities:
–
–
–
–
–
LIP-LX
Internal infrastructures and services
International infrastructures and services
National initiatives
International initiatives
Research and application projects
• Activities scope:
–
–
–
–
–
Goes far beyond the LIP internal context
Mostly focused in distributed computing
National and international responsibilities
Technical
Political
LIP workshop 2010
NCG
2
Projects
• Participation in EU funded projects
–
–
–
–
–
Research, development and production
Continuous participation since 2001
Contribution to new technologies
Pilot and demonstration projects
Production quality infrastructures
• Worldwide LHC Computing Grid
– CERN project
– Generic computing infrastructure and
services for all LHC experiments
– Memorandum of Understanding (MoU)
– Portuguese obligation to provide a Tier-2
• National projects
– Initially complementary funding to EU
– Funded through the national grid initiative
LIP workshop 2010
Between 2008 and 2009:
•
•
•
•
•
•
•
•
Int.Eu.grid
EGEE-II
EGEE-III
WLCG
Grid Middleware
Main node for grid computing
EGI
IBERGRID
3
~ 290 centres
~ 55 countries
~ 150,000 CPUs
~ 28+41 PetaBytes
~ 17,000 users
~ 200 VOs
~ 330K jobs/day
Archeology
Astronomy
Astrophysics
Fluid Dynamics
Civil Protection
Chemistry
Earth Sciences
Condensed matter physics
Finance
Fusion
Geophysics
High Energy Physics
Life Sciences
Multimedia
Material Sciences
…
Largest grid computing infrastructure
WLCG operations depend on EGEE
LIP workshop 2010
4
Worldwide LHC Computing Grid (WLCG)
•
The WLCG collaboration:
–
–
–
–
–
–
–
•
4 experiments
15 PetaBytes per year
34 countries
170 datacenters
12 large datacenters (T0, T1)
38 federations smaller centres (T2)
Regional grid infrastructures
EGEE, OSG, Nordugrid
Computing MoU:
– Between CERN and governments
– Agreed in October 2005
LIP operates the Portuguese WLCG federated Tier-2
The Portuguese Tier-2 supports ATLAS and CMS
LIP is member of the WLCG RRB and GDB
LIP workshop 2010
6
WLCG Tier-2 Centres Usage
• 32 Tier-2s
• One per country
• Data for:
–
–
–
–
ALICE
ATLAS
CMS
LHCB
• In December the
Portuguese T2
was the 7th
WLCG largest
CPU provider
LIP workshop 2010
7
Creating Infrastructures
• The Portuguese national grid initiative (NGI) was created in 2006:
– It is coordinated by UMIC and LIP
– Aims to:
• Support distributed computing for complex problem solving
• Integrated Portugal in major international distributed computing infrastructures
• The Portuguese NGI is creating a national grid computing infrastructure:
– Provide resources to be shared by the research communities
– Integrate existing resources
• The project “Main Node for Grid Computing” is the corner stone:
– Provisioning of core processing and storage resources for sharing
– Central services to enable a national grid and its integration in major
international distributed computing infrastructures
Core Resources
Infrastructures and
initiatives
INGRID
EGI
IBERGRID
WLCG
Existing Resources
• The WLCG MoU is being fulfilled in the context of(EGEE,
theint.eu.grid,
Portuguese
NGI
...
EELA,
•
INGRID projects ...)
The WLCG Tier-2 is a customer of the infrastructure shared resources
Other resources
LIP workshop 2010
Users
9
NGI Core Infrastructure
– Main Node for Grid Computing (NCG)
• New National Grid Initiative computer centre
• Managed by LIP on behalf of the Portuguese National Grid Initiative
– LIP computing resources
• Lisbon
• Coimbra
– Everything was built with funds to be returned to the EU
• Extremely strict deadlines and rules
FCCN
Layer 3
backbone
Main
node
LIP
Braga
3 Gbps
10 Gbps
LIP
Lisbon
10 Gbps
Layer 2
LIP workshop 2010
1 Gbps
LIP
Coimbra
10
LIP Upgrades and Improvements
• Lisbon and Coimbra
–
–
–
–
–
–
–
–
–
–
–
–
–
–
–
–
Computing farm
Storage systems
Room improvements
Power distribution
AVAC
UPS
Environment sensors
Fire detection
Fire extinction
Network core switching
Access switching
Wide area connectivity
Management network
Support systems
Virtualization platform
Tape backup
LIP workshop 2010
12
Main Node for Grid Computing
• Built by a consortium for the housing of
the National Grid Initiative core services:
– LIP, FCCN, LNEC
• Located:
– LNEC campus in Lisboa
– Near the FCCN Network Operations Centre
• Objective:
– House core services for the Portuguese NGI
– House grid resources for sharing
– House grid resources for other organizations
• Project started in 2007:
– Civil engineering and architecture
– Electrical Power and cooling
– Networks and Computing
LIP workshop 2010
14
Main Node for Grid Computing
LIP workshop 2010
15
Other Activities
• LIP computing services
–
–
–
–
Management of LIP systems, services and networks
User support
Web development and design
Coimbra computing services are being restructured
• Authentication Authorization and Security
– LIP CA grid computing authentication for Portugal
– Member of EUgridPMA / IGTF
– Security team for Portuguese grid sites
• New domains
– Voluntary computing
– Clouding computing
– High performance computing
• More systematically got into
– Benchmarking and tuning (processors, networks and storage)
– Virtualization and fault tolerance
LIP workshop 2010
18
European Grid Initiative (EGI)
• The European Grid Initiative (EGI) will replace EGEE
– Ensure long term sustainability for grid computing infrastructures in Europe
• Grid infrastructures are really needed and their future must be guaranteed
• Moving away from the model based on short term projects and individual research institutes
– New organizational model with two layers:
• National Grid Initiatives managed and funded by the governments
• European Grid Initiative as a top integration layer (country + EU funded)
• EGI headquarters established in Amsterdam
• Transition already taking place this year overlapping with EGEE-III
• Several funding projects submitted to the EU to complement funding
LIP workshop 2010
19
European Grid Initiative (EGI)
• EGI design project
– Policy board chairman
– Portuguese representative in the policy board
• EGI council
– Portuguese representative
• EGI InSPIRE
– Integrated Sustainable Pan-European Infrastructure for Researchers
in Europe
– Submitted to the EU in last November
– Main EGI project for infrastructure integration and operations
– LIP as partner representing the Portuguese NGI
• EGI International bids
– LIP together with IFCA/CSIC won the bid for coordinating the EGI
middleware rollout at European level
LIP workshop 2010
20
Announcement
TCP/IP and networking training
Aimed at the LIP computing group personnel
But open to everybody that will be willing to attend/suffer
It will be organized as two training sessions per week
It will take several weeks ...
It will start in January
More details to be announced next week
LIP workshop 2010
21