Transcript Grid activities in the Czech Republic

Grid activities
in the Czech Republic
Jiří Kosina, Miloš Lokajíček, Jan Švec
Institute of Physics of the Academy
of Sciences of the Czech Republic
http://www.fzu.cz/
HEP Experiments
in the Czech Republic (1)
●
D0, STAR, ATLAS and ALICE (main experiments
only) computing activities:
–
–
●
Institutions
–
–
–
●
detector simulations, reconstruction
data analysis in running projects - main participants are
Institute of Physics AS CR, Institute of Nuclear Physics
AS CR
Charles University in Prague
Czech Technical University in Prague
Facilities
–
EDG test-bed now being converted to LCG
Institutions involved in the Grid
activities
●
CESNET (http://www.cesnet.cz/)
●
Charles University in Prague (http://www.cuni.cz/)
●
●
Czech Technical University in Prague
(http://www.cvut.cz/)
Institute of Physics AS CR (http://www.fzu.cz/)
CESNET(1)
CESNET z.s.p.o.
●
●
●
association of legal entities created by Czech universities and Academy of
Sciences of the CR
Czech scientific network provider
–
shareholder of the Dante
–
member of TERENA
–
Internet2 international partner
Computing and networking strategic research projects
–
Optical networks and their development
–
IPv6 implementation in the CESNET2 network
–
Multimedia transfers
–
MetaCenter
–
Voice services in the CESNET2 network
–
QoS in high-speed networks
CESNET(2)
●
International projects
–
–
–
–
●
GEANT
DataGrid -> EGEE
SCAMPI
6NET
Other projects
–
Infrastructure and technology for on-line education ,
Distributed contact center , Smart NetFlow analyser ,
Storage over IP , Presentation , RT system in CESNET,
Securing CESNET2 local networks , Time
synchronization and NTP servers, Platforms for video
transmission and production, MEDIMED
CESNET(3)
●
●
●
●
1.2Gbps connection to GEANT (over HW 10 Gbps)
1+1Gbps connection to NIX (Peering with Czech
ISPs)
800Mbps connection to USA (Telia)
2.5 Gbps CzechLight connection (now used for
other tests)
–
Reserved optical connection 1 Gbps
Institute of Physics <-> CERN
–
Connected to our linux router, switching to “backup” line
through Internet managed by BGP protocol
–
Possibility to connect to StarLight in the future
CESNET(4)
●
European DataGrid project
–
WP1 workload management
–
WP7 network monitoring
–
Certification authority established for EDG
●
issuing certificates to Czech academic subjects
Institute of Physics AS CR - FZU
●
●
●
D0 experiment participation
ATLAS – main contribution to the hadronic
calorimeter TILECAL and Inner Detector
(pixel sensors production and test of strip
and pixel detectors), power supplies design
and production
Computing
–
–
Mass simulation for D0, reconstruction, data
analysis
ATLAS – participation to Data Challenges,
Atlas-LCG
Institute of Physics AS CR – FZU
(2)
●
Computing projects
–
–
EDG from 1 Jan 2001
●
WP6 – testbed
●
CESNET network support important
●
Plan to continue in EGEE
LCG – GDB
(LHC Computing Grid – Grid Deployment Board)
●
LCG deployed in October 2003 – CERN press
release
Institute of Physics AS CR – FZU
(3)
●
Dedicated server farm
for HEP and Grid
computing
–
34x dual 1.13Ghz PIII,
1TB disk array,
currently experimenting
with new 10 TB disk
array
Institute of Physics AS CR – FZU
(4)
●
Current status
–
●
Construction of the new computing room in the institute
●
Designed for 150 kW electric power
●
UPS, cooling, engine-generator
●
Construction finished by end 2003 with 50% capacity
●
Full capacity next year
Application for triple capacity upgrade of the current
farm (30 double Xeon units, 20 TB disk space)
for the 2004. Hope in positive result.
Job Management
●
PBS Pro 5.2
LCG1 -> OpenPBS (will merge soon with PBS
Pro)
●
●
queues
–
shortq
– d0
–
normalq
– atlas
–
longq
– alice
– hightest
golias:~$ qsub -q atlas run.sh
D0 MC simulations
●
SAM station
Sequential Access to data via Metadata
●
UPS, UPD
management of software products on local systems (UPS)
downloading products from product distribution servers
(UPD)
●
D0-RunII
software rel. p14.02.00, p14.05.01
EDG, LCG
●
●
●
We have installed LCG1 software and
currently running LCG1-1_1_1 version of the
software.
LCG – middleware software to provide
environment for distributed computing for
LHC
Member of WP6 work package – farm is
running EDG 1.4 software, for testbed
purposes
LCG (1)
●
●
●
LCG – LHC (Large Hadron Collider)
Computing Grid
The accelerator will start operation in 2007
12-14 PetaBytes of data will be generated
every year (20 milion Cds). It is assumed that
analyzing this will require approximately
70.000 CPUs
LCG(2)
●
●
●
●
Based on EDG software – project funded by
EU, finishing by the end of this year
GRID infrastructure allows every member to
submit his job (along with JDL file) to the
GRID, and after the computing is finished (no
matter where), get the output
In JDL you can specify requirements specific
for task (SW versions, # CPUs, etc)
Distributed storage of data.
LCG(3)
●
●
●
Installation of LCG software is done through
LCFGng toolkit – useful to manage wide
variety of configurations for different hosts
Joining sites are provided with pregenerated
skeleton of configuration files in order to
simplify installation procedure (it's necessary
to modify them)
Installation is done simply by enabling PXE
boot and rebooting the nodes
10TB disk array and Linux
●
●
We have 10 TB disk
array. Problem with
Linux 2.4 – only 2TB
block devices
supported
Patch for LBD exists
for 2.4 kernel, but it
collides with another
patch needed for
LVM.
10TB disk array and Linux
●
Kernel 2.6 (not yet stable) supports both LBD
and device mapper (needed for LVM). After
some hacking of NFS code (our patches are
incorporated in 2.6.0-test9 kernel) we have
successfully created 6 TB partition (XFS,
ext2)
Institutions involved – contacts
●
●
●
●
Jan Svec <[email protected]> - network and
system management, D0 experiment (grid)
Jiri Kosina <[email protected]> - installation of
grid software at FZU, ALICE experiment,
network and system management
Jiri Chudoba <[email protected]> ATLAS experiment
<[email protected]> - CESNET's PC
clusters
Summary – Grid infrastructure
●
●
●
Basic Grid infrastructure has been
established during last year and further
expansion due to research or commercial
needs in future shouldn't be problematic
CESNET provides good international network
connectivity, plan tests of CzechLight optical
network
Preparing for Data Challenges of LHC
experiments in 2004