Transcript pdjp-mar12-kd - Indico

PDJD Development Plan
Kaushik De
Univ. of Texas at Arlington
Kickoff Meeting, CERN
March 8, 2012
Overview
 ProdSys worked very well for past 5 years
 But we need to build new system for 2013 and beyond

Discussed in ProdSys splinter meeting Feb. 2012
https://indico.cern.ch/conferenceDisplay.py?confId=168318

Need better integration with PanDA brokerage and job dispatch



Dynamic job definition
Modular design of new system

Job request separate from job definition

Post/error processing as separate step
Used for central, group and user tasks - scalability
 Must not interfere with 2012 data run

GDP (Rod, Andrej, Armen…), Sasha, Dmitry will continue to support
the current system
Kaushik De
Mar 8, 2012
2
What is PDJD
 A very bad name for a new project

Please think of new name, or else we get stuck like PD2P
 Core components

Not to be decided today – will evolve as we design new system

Job request system – script driven, but UI also supported

Dynamic job definition – integrated with PanDA

Job retries – integrated with PanDA

Post/error processing – task completion

DB backend - prodDB/PandaDB overlap and optimizations

…
Kaushik De
Mar 8, 2012
3
Development Plan
 The PanDA development model

Start with simple implementation

Evolve with experience
 Phase I

Review current design

Develop overall plan for new system
 Phase II

Specify interfaces and core components

How do tasks become jobs which match sites?
 Phase III

Develop and test
 Deploy, support and maintain
Kaushik De
Mar 8, 2012
4
Schedule
 Phase I – review and plan

About 2 months
 Phase II – design and specifications

About 2 months
 Phase III – coding

About 3 months
 First deployment

After summer ~ September/October
Kaushik De
Mar 8, 2012
5
People
 Steering Group

The usual suspects: Kaushik, Alexei, Rod
 Lead developers



Sasha – Task Request system
Tadashi – Dynamic Job Definition
TBD – post/error processing
 Experts/developers – PanDA+ProdSys teams






Dmitry – ProdSys
Mikhail – users/authentication
Marcin – DB
Valeri - monitoring
Alden – schedconfigDB/AGIS
Paul - pilot
Kaushik De
Mar 8, 2012
6