Transcript Application Management Service - progress

Facilitating the process of
enabling applications within grid
portals
Michal Kosiedowski
[email protected]
Poznan Supercomputing and Networking
Center
• PSNC was established in 1993
and is a R&D center in:
– New Generation Networks
• POZMAN and PIONIER networks
• 6-NET, SEQUIN, ATRIUM projects
– HPC and Grids
• GridLab, Crossgrid, SGIGrid,
PROGRESS, Clusterix projects
– Portals and Content Management
Tools
• Polish Educational Portal
"Interkl@sa",
Multimedia City Guide,
Digital Library Framework,
Interactive TV
PROGRESS
• Budget: ~4,0 MEuro
• Project Partners
–
–
–
–
Sun Microsystems Poland
PSNC IBCh Poznań
Cyfronet AMM, Krakow
Technical University Lodz
• Co-funded by the State Committee for Scientific
Research (KBN) and Sun Microsystems Poland
• Goal: construct an access environment to grid
resources and services
PROGRESS: architecture
Grid Service Provider
• Job Submission Service
– computing job building, submitting jobs to the grid for execution and
monitoring the execution
– grid job descriptions in the XRSL language are transferred to the
GRB for the execution of the job
– possibility to prepare workflows
• Application Management Service
– manages the PROGRESS application repository
– an application descriptor contains a reference to the application’s
executable and a set of its available, required or optional
arguments, required environment variables as well as input and
output files
– one executable may be referenced by many applications
– different application configurations are recognized as independent
applications
Data Management System
• A distributed system enabling the
management of grid data files
• Storing files in distributed storage modules of
various types: generic filesystems, archivers,
relational databases
• Uses metadata to describe files
• Allows access to data banks like a mirror of
Sequence Retrieval System
• Exposes its functionality within the Data
Broker Service
PROGRESS HPC Portal
Core portlets
• “My computing jobs”:
– creation, configuration and execution of grid jobs on the grid
– downloading and visualizing the results
– utilizes the functionality of the GSP Job Submission and Application
Management services and the DMS Data Broker.
• “Applications”:
– management of grid-enabled applications within the GSP
application repository
– co-operates with the GSP Application Management Service and the
DMS Data Broker.
• “My data”:
– management of the files stored within the DMS
– uploading and downloading files from the DMS
– communicates with the DMS Data Broker.
Enabling an application within
the PROGRESS HPC Portal
• Install the application on a computing server
or prepare an executable and upload it to the
Data Management System
• Use the “Applications” portlet to add the
application to the application repository
managed by the Application Management
Service
• Use the “My computing jobs” portlet to build a
grid job utilizing the newly added application
“My computing jobs” portlet
• Exposes the whole functionality of the Job
Submission Service
• Enables submission of any job utilizing any available
application
• Great for experienced users allowing them to fulfill
their requirements
• Too complicated for use by non-advanced users
• Solution: specialized application portlets utilizing the
same services as core portlets and familiar with the
corresponding grid-enabled application structure
“Gaussian” portlet (1)
“Gaussian” portlet (2)
“Gaussian” portlet (3)
“Gaussian” portlet (4)
Application portlets
• Provide users with interfaces that are easy to
use and require no grid knowledge
• Grid jobs may be configured very quickly with
an intuitive wizard
• Easy and quick access to results: application
portlets are familiar with corresponding
applications
• Problem: How to quickly build such
specialized portlets?
PROGRESS Portlet Framework
• The Framework allows preparing a high-level
interface to Web Services with special beans
to store the data obtained from WS
responses and classes used to translate
HTTP requests into the invocation of proper
WS methods.
• This allows to quickly build new portlets on
top of those classes to utilize the already
existing functionality that they provide.
HTTP
Provider
Provider
Provider
Provider
Provider
...
Portlets
Content
Generator
Request
Handler
Web Service
Proxy
Content
Generator
Request
Handler
Web Service
Proxy
Content
Generator
Request
Handler
Web Service
Proxy
Content
Generator
Request
Handler
Content
Generator
...
Request
Handler
Web Service
Proxy
...
...
Request
Handler
Web Service
Proxy
Web Services Invocation Framework
SOAP (to GSP, DMS)
Communication with Web Services (1)
• Web Services Invocation Framework is used
for communication with Web Services
Object executeMethod(
String methodName,
ArgumentTable methodArguments)
• AXIS toolkit is used to generate the binding
classes for complex class types mapping
• Each Web Service is described in an XML
configuration file
Communication with Web Services (2)
<?xml version="1.0" encoding="UTF-8"?>
<web-services>
<web-service id="appmgmt">
<name>Application Management</name>
<wsdl-location>ApplicationManagementWS.wsdl</wsdl-location>
<port-type-name>
http://viola.man.poznan.pl:5021/axis/services/ApplicationManagement
</port-type-name>
<port-type-namespace>ApplicationCtrl</port-type-namespace>
<description>GSP Application Management Service</description>
</web-service>
<web-service id="jobs">
<name>Job Submission</name>
<wsdl-location>JobSubmission.wsdl</wsdl-location>
<port-type-name>
http://viola.man.poznan.pl:5021/axis/services/JobSubmission
</port-type-name>
<port-type-namespace>JobSubmissionCtrl</port-type-namespace>
<description>GSP Job Submission Service</description>
</web-service>
<web-services>
Content generation (1)
•
•
•
•
•
The content is generated based on two HTTP request parameters
(page and action).
1. page e.g. ApplicationList, JobList
2. action e.g. removeApplication, submitJob
Actions are defined in actions.xml, pages in pages.xml (the
content of those files is read by appropriate classes,
ContentGenerator reads Action and Page objects from those
classes). Actions are executed before page content is generated;
pages usually contain default actions to generate their content.
Actions are usually page independent, they can also replace the
default page action
An action is associated with one or more RequestHandler
methods,or with methods of many RequestHandlers
The Java reflection mechanism is used to call RequestHandler
methods
Content generation (2)
• Page content is delivered by
RequestHandlers in form of PageBean
classes
• Each page can contain one or more classes
implementing the PageBean interface
(PageBean classes know how to deliver their
content in form of a DOM)
• The page generates its content by converting
its PageBeans into a DOM and then
transforming it using an XSLT template
Content Generator
Page
DOM
DOM1
PageBean1
RequestHandler1
HTML
DOM2
PageBean2
DOM3
PageBean3
DOM4
PageBean4
RequestHandler2
RequestHandler3
Content generation (3)
• Pages are defined in an XML config file:
<pages>
<page name="ApplicationList">
<display-name>Application List</display-name>
<description>List of PROGRESS applications</description>
<xsl-file>application-list.xsl</xsl-file>
<action name="ApplicationList">
<request-handlers>
<request-handler class="ApplicationGetRequestHandler">
<method name="getApplicationList"/>
</request-handler>
</request-handlers>
</action>
</page>
...
</pages>
Content generation (4)
• Invocation of many RequestHandlers
<page name="EditApplicationExecutable">
<display-name>Edit Application Executable</display-name>
<description>
Edit application executable file - choose/change the file.
The executable can also be uploaded to SZD.
</description>
<xsl-file>edit-application-executable.xsl</xsl-file>
<action name="EditApplicationExecutable">
<request-handlers>
<request-handler class="ApplicationGetRequestHandler">
<method name="getApplication"/>
</request-handler>
<request-handler class="SzdGetRequestHandler">
<method name="getUserDirectoryListing"/>
</request-handler>
</request-handlers>
</action>
</page>
Content generation (5)
• RequestHandler “workflow”
<page name="JobList">
<display-name>Job List</display-name>
<description>List of jobs for a given user</description>
<xsl-file>job-list.xsl</xsl-file>
<scripts>
<script language="JavaScript" src="jscript/job-submission/job_list.js" />
</scripts>
<action name="JobList">
<request-handlers>
<request-handler class="JobGetRequestHandler">
<method name="getJobsOfUser" output-name="userJobs"/>
<method name="filterJobs"
input-name="userJobs" output-name="filteredJobs"/>
<method name="sortJobs" input-name="filteredJobs"/>
</request-handler>
</request-handlers>
</action>
</page>
Content generation (6)
• Page error redirection – defined in an
error-redirections.xml file:
<?xml version="1.0" encoding="UTF-8"?>
<redirections>
<error-redirection-group id="EditJob:saveJob">
<error-redirection type="validation" redirect-to-page="EditJob" />
</error-redirection-group>
<error-redirection-group id="EditTaskInfo:saveTaskInfo">
<error-redirection type="validation" redirect-to-page="EditTaskInfo" />
</error-redirection-group>
</redirections>
Framework advantages
• It’s easy to reuse the portlets in different portal
frameworks and on various portals and websites
– The top-level Provider class can be easily adjusted to
implement any portal framework API
– The content generated by a portlet can be easily changed by
playing with configurations files
• The same RequestHandlers can be used by
mutliple portlets thus enabling easy creation of new
portlets communicating with the same WS services
– specialized application portlets, e.g.: “Gaussian” portlet,
“DNA Assembly” portlet
Conclusions
• PROGRESS grid access environment features the Application
Management Service which enables to easily manage
applications available in the grid, the “Applications” portlet
which allows to utilize the functionality of this service and the
“My computing jobs” portlet which can be used to create a
computing job based on any application available in the
repository.
• Our experience shows it’s important to provide users with
interfaces to grid-enabled applications that are easy to use and
do not require deep knowledge about the grid and/or the
application.
• PROGRESS Portlet Framework facilitate the construction of
specialized application portlets that deliver the functionality of
the corresponding grid-enabled applications within a grid portal.
PROGRESS
http://progress.psnc.pl/
http://progress.psnc.pl/portal/
[email protected]