Transcript Requirements - Indico

F2F JRA4 Meeting
CERN
16 June 2004
www.eu-egee.org
Network Requirements
Javier Orellana
EGEE is a project funded by the European Union under contract IST-2003-508833
Outline
• Components and Requirements
• NA4 Requirements
• Other Requirements:
 HEP
 Biomedical
 Generic
• Requirements from mw Clients
<F2F JRA4 Meeting, CERN> <16 June 2004> - 2
Components and Requirements
Consumer (User Application)
Requirements
End User Requirements
Operations
Grid Middleware
Middleware Requirements
Computer
Element
NE-A
NE-B
Network
Element
Storage
Element
NE-X
Network
(GEANT+NRENs)
<F2F JRA4 Meeting, CERN> <16 June 2004> - 3
NA4 : User Interface Requirements
• R1.2 User Interface configuration.
The user interface should be easy to configure. A pointer to an information
service server should be sufficient for the user interface to get all the
configuration information it requires. A default information service is
expected.
• R1.3 Application Programming Interface.
The application programming interface to middleware services should be
available in C++ and Java. The installation should be lightweight. It should
cover all services accessible from the user interface host and the worker
node: jobs submission, data manager, information system, etc.
If needed, the network related information has to be included in the user interface
<F2F JRA4 Meeting, CERN> <16 June 2004> - 4
NA4: Network Requirements
• R6.1 Communications encryption
It should be possible to encrypt data prior to communication
between sites to prevent third party listening.
• R6.2 Outbound connectivity
 Applications will need to access services external to the grid
middleware. It should be possible to establish a communication
between a grid node and non-grid hosts on any port.
 AWG UC#externalConnection
• R6.3 Guaranteed bandwidth
Some applications (e.g. interactive) have a need for a guaranteed
network bandwidth.
<F2F JRA4 Meeting, CERN> <16 June 2004> - 5
NA4 : Operation Requirements
• R8.2 User control
 It should be possible to grant access to an authorized user and
to revoke a user.
 HEPCAL UC#gridauth
 HEPCAL UC#gridrevoc
• R8.6 VO management (creation, modification and
deletion)
 VO management should allow the creation of user groups within
a VO. Each user group should have the possibility to choose
whether or not to share resources with the other VO groups.
• ie, NREN allowing just 25% of bandwidth to Grid traffic.
<F2F JRA4 Meeting, CERN> <16 June 2004> - 6
…Is that all ?
• Maybe not. More requirements can be needed
 What about Network Performance Monitoring? (Nothing is
included in the “Information Service” requirements)
 Are end users and/or operations interested in diagnostic tool?
 No mention about possible requirements from the mw clients to
the network element
 Guaranteed network bandwidth?, maybe also other network
resources like L2 paths or Premium IP. Is that important for the
end users?
Still key to talk to end users and experts !!!!
<F2F JRA4 Meeting, CERN> <16 June 2004> - 7
HEP Case *
• Context : Massive number of datasets globally distributed
 ie. BaBar currently ~1PetaByte
 Hundreds of Analysis Sites
 Need for managing global resources optimally
 The system needs to be managed and monitored globally end to end
• Requirements:
 100 GBytes extracted from 100 TeraByte data in less than 10 minutes
=> ~1Gbps bandwidth
 End to end Quality of Service
 In 3 years, extract 100TBytes from 100 PBytes => ~100Gbps bandwidth
• Lamda flows between end points
• Capacity Planning
• Key to have Network Performance Monitoring to optimize network usage
* Source: High-Performance Network Planning Workshop, US DoE
<F2F JRA4 Meeting, CERN> <16 June 2004> - 8
Biomedical Case *
• Outpacing number of researchers
 Highly effective AA solutions
 Policy-based control and sharing of Grid resources (including network
resources)
 Clear need for VO, resource allocation policies and accounting
mechanisms (including network resources)
 Heavy use of distributed large-scale relational database => end-to-end
solutions for database integration and federation
• Requirements:
 Databases ~TeraBytes and soon PetaBytes => ~1Gbps and ~100Gbps
 Accurate Database replicas with high integrity => highly reliable and
accurate transport mechanism, enhancing current ones and QoS
mechanisms
 Multicasting
* Source: High-Performance Network Planning Workshop, US DoE
<F2F JRA4 Meeting, CERN> <16 June 2004> - 9
In general, other Requirements *
• Authenticated data streams for easier site access through firewalls
• Robust access to large data files => multiple network paths with
•
•
•
•
•
•
backups and redundancy
Adequate bandwidth and bounded latency for analysis and
visualization of remote datasets (Quality of Service guarantees)
Other applications require ~500Mbps sustained bandwidth
Co-scheduling of computer and network resources
End-to-end Quality of Service Management
Network caching (rapid access to intermediate locations)
Monitoring and problems diagnostic
* Source: High-Performance Network Planning Workshop, US DoE
<F2F JRA4 Meeting, CERN> <16 June 2004> - 10
From mw Clients perspective *
• Data Replication Manager:
 Data replicated, picket up from source S and delivered to D by time T
 Data replicated with deadline, need to be delivered before time Tf
 Data replicated from S to destinations D > 1, multicast?
• Workload Manager:
 Workload Manager needs to ensure that remote data is accessible for
the time window CPUs have been allocated
 In case remote data is unknown, then at least WM needs to guarantee
“on average” access to the dataset
• Remote Archive Shadowing or Mirroring (keep remote shadow
copies of the same dataset)
 Keep datasets synchronised that change slowly
 Need for changing all at once
• High Performance Computing Virtual Backplane:
 Two end point linked with a maximum latency boundary
* Paul Mealor, Peter Kunszt and Peter Clarke comments about middleware clients
<F2F JRA4 Meeting, CERN> <16 June 2004> - 11
Authorisation
Auditing
Grid Access
Service
Grid Middleware
Grid
Monitoring
Accounting
Data
Management
Site
Gatekeeper
Workload
Management
Monitoring
Authentication
Network Performance
Interfaces with Middleware Architecture
Bandwidth Allocation and
Reservation
<F2F JRA4 Meeting, CERN> <16 June 2004> - 12
Questions?
<F2F JRA4 Meeting, CERN> <16 June 2004> - 13