Transcript OGCE Overview: Services, Gadgets, and Tags

OGCE Overview: Services,
Gadgets, and Tags
Marlon Pierce, Suresh Marru, Gregor von Laszewski, Mary
Thomas, Maytal Dahan, Gopi Kandaswamy, Rion Dooley, and
Wenjun Wu
Links for More Information
 Website: http://www.collab-ogce.org
 News/RSS/Blog: http://collab-ogce.blogspot.com
 Email: [email protected]
 SVN Code Repository:
http://ogce.svn.sourceforge.net/viewvc/ogce/
 And get tar.gz’s
 Source Forge: http://sourceforge.net/projects/ogce
 See us at SC08: IU booth 1:00-3:00 on Tuesday, November
18.
Project
Description
Status
Grid Portal
Packaged, one step installation of
Grid portlets into
Tomcat+Gridsphere
Stable, V2.4
for SC08
Axis Services
One step install of Resource
Discovery and Prediction Services.
More services to come.
Dev. Release
for SC08.
Workflow Suite
Support for scientific workflows,
based on LEAD tools. Suresh will
describe next time.
Dev. Release
for E-Science
GTLAB
JSF Tag Libraries for building Grid
portlets and Gadgets
Dev. Release
for SC08
JavaScript COG
(Cyberaide)
JavaScript libraries for building Grid
gateways.
Dev. Preview
for SC08
Gadget Container
Open Social Inspired container for I
Google gadget
Dev. Preview
for SC08
Incubator
projects
Pylons portals, metascheduling
services, bio-gadgets, etc.
SVN if you dare
Services, Workflow, GTLAB, Cyberaide, and Gadget container are part of
an emerging stack
OGCE and Gateways
 We develop and package software for use by Gateways.
 A lot of this comes from active Gateways.
 GPIR: TeraGrid User Portal
 Workflow tools: LEAD
 Resource Discovery Service, File Browser Applet: GridChem
Gateways
OGCE
Software
GPIR for Facebook, iGoogle
 GPIR: GridPort Information
Repository, stores dynamic and static
resource information for VOs
 Facebook



Mini TeraGrid system monitor and full
size system monitor
Mini:
http://apps.facebook.com/mini_tgsys
mon/
Full Size:
http://apps.facebook.com/tgsysmon/
 Google



Mini System monitor for iGoogle page
To add visit:
http://www.google.com/ig/adde?mod
uleurl=http://iola.tacc.utexas.edu:22
080/gpirapp/minigpir.xml&nocache=1&source=
imag
GPIR Facebook App
OLSGW Bio-gadgets
ClustalW gadget
Blast gadget
Job Submission History gadget
Some OGCE Services
New things available for download
Resource Discovery Service
 Extracted from the GridChem project/gateway
 Contributed by Rion Dooley
 Axis2 Web service
 Integrates information from INCA, GPIR services.
 What machines are in my Grid^H^H^H^H Cloud?
 What are the service names?
 Which ones are actually running?
 Works with TeraGrid deployed services, but you can use
it with your own information service installations.
Resource Prediction Service (RPS)
• Predicts an optimal set of resources for running
scientific applications
– Predicts a set of resources on which the sum of data
transfer time, queue wait time and compute time for the
application will be minimum
– Data transfer time: Uses input data size and NWS info
– Queue wait time: Uses QBETS info
– Compute time: Uses application performance info
• Axis2 web service with a MySQL backend
• RPS is a subset of the Fault Tolerance and Recovery
service used in LEAD and VGrADS
Packaging Services
 Everything builds with one command
 mvn clean install
 We use Apache Maven 2 for builds.
 Structured, modular builds and consistent layouts
 Build a single module:
 mvn clean install –f somemod/pom.xml
 We include Apache Tomcat in the download, so all you need
is Java and MySQL.
 Suitable also for the NMI build and test system.
 Want the Javadoc? mvn javadoc:javadoc
GTLAB
Tag Libraries for building gadgets and portlets
out of reusable parts.
User’s Browser
JavaScript
OGCE Gadget
Container
iGoogle, Facebook
GTLAB Gadgets
RPS, RDS, GPIR
Services
TeraGrid Services
Other Services
GTLAB’s Relationship to Other
Components
MyProxy Example
<h:form id="form">
<h:outputText value="Username:"/>
<h:inputText value="#{resource.username}" required="true"/>
<h:outputText value="Password: "/>
<h:inputSecret value="#{resource.password}" required="true"/>
<h:outputText value="MyProxy Server: "/>
<h:inputText value="#{resource.myproxyHostname}" required="true"/>
<o:submit id="submit" value="Submit" binding="#{builder.body}"
action="#{builder.junkAction}" >
<o:multitask id="multi" persistent="true" taskname="#{resource.taskname}">
<o:myproxy id="mypr" hostname="#{resource.myproxyHostname}"
lifetime="2" password="#{resource.password}" port="7512"
username="#{resource.username}"/>
</o:multitask>
</o:submit>
</h:form>
Executing Multiple Task
<o:submit id="submit" value="Submit" binding="#{builder.body}"
action="#{builder.junkAction}" >
<o:multitask id="multi" persistent="true"
taskname="#{resource.taskname}">
<o:jobsubmit id="js" myfaces="true"
arguments="#{resource.arguments}"
executable="#{resource.executable}"
hostname="#{resource.hostname}"
provider="#{resource.provider}"
stdout="#{resource.stdout}"/>
<o:filetransfer id="ft" myfaces="true"
from="#{resource.from}" to="#{resource.to}" />
<o:dependency id="dep" task="ft" dependsOn="js"/>
</o:multitask>
</o:submit>
GTLAB Features
 Extends Java Server Faces.
o Tag components wrap major COG Abstraction Layer features
o And Web Service clients, GridShib, etc.
o Supports tag inter-dependencies.
 Allows you to do standalone development.
 Use JSF portlet bridge to convert into portlets
 No new coding, just add/modify XML config files and jars.
 Or forget about portlets and
o Convert into Google Gadgets
o Develop as a Facebook application
o Etc.
Installing GTLAB
 Download the release code from www.collab-ogce.org or
check out from SVN:
o svn checkout
https://ogce.svn.sourceforge.net/svnroot/ogce/GTLAB
o SourceForge also supports tar.gz downloads
o Tagged release in
http://ogce.svn.sourceforge.net/viewvc/ogce/tags/
 Run “mvn clean install” in GTLAB directory.
 Download includes a web server.
 Start with “./startup.sh”, go to http://localhost:8080/GTLAB
What’s New with GTLAB?
 Completely revised dependency graph processing.
 Removed dependence on COG graph processing (still use the COG)
 We can now support graphs of any tag you care to write.
 Only support pipelines currently, not full DAGs
 Redesigned to simplify writing new tags.
 Extensive use of factories.
 You no longer have to change core code to add a tag
 Simplified process of creating new tags
 4 Java files and 3 XML config files
 Now can be done in one (mvn) step
 Many new tags
Levels of Abstraction in GTLAB code
You can use prepackaged gadgets/portlets.
You can develop new applications using
existing tag libraries.
You can make new tags for your services
You can hack or steal the code
 Note JSF is compatible with JSP, so you can mix and
match.
 And you can embed JavaScript
 YUI, Scriptaculous user interface goodies.
 Google and Facebook APIs.
Tag
MyProxy
JobSubmit
FileOperation
FileTransfer
XSubmissionOutput
ResourceDiscovery
ResourcePrediction
MyCommunityProxy
Community Log
Description
Gets a proxy credential (COG)
Runs remote commands (COG)
Create, delete, list remote files and
dirs (COG)
GridFTP downloads, 3rd party
transfers (COG)
Format outputs of job and file
operations
Clients for the RDS
Clients for the RPS
Get and decorate TG Community
credentials with Grid Shib
Log use of TG Community credentials
GTLAB Example
Description
MyProxyExample
Get a credential and store in OGCE
global proxy store.
Various examples showing how to
submit a grid job and get output.
JobSubmitExamples
Dashboard Examples
DirectoryBrowser
MultipleTaskExample
Runs showq on Big Red, Cobalt, and
Mercury and formats output.
Displays and downloads files on a
remote host via GridFTP
Shows how to couple file transfers,
operations, and
These will work as gadgets, portlets,
or standalone applications
GTLAB gadgets
can also run as
portlets with no
code changes by
using the JSF
portlet bridge.
iGoogle
Client
Aggregator
Tomcat Web Server
+
Portlet Container
Tomcat
Web
Server
Compile and deploy into server
GTLAB Gadgets
Code
Base
Decorate with bridge
and container jars, xml
config files.
GTLAB Portlets
Code
Base
GTLAB Applications as Google Gadgets:
MOAB dashboard, remote directory
browser, and proxy management.
What’s next?
 OpenID and RSS client support
 Support for more complicated graphs
 JavaScript/AJAX actions as well as HTML FORM
actions.
 But really, we’re looking for interested
developers and users.
Gadget containers
aggregate content
from multiple
providers. Content is
aggregated on the
client by the user.
Nearly any web
application can be a
simple gadget (as
Iframes)
Other Gadgets
Providers
Tomcat + GTLAB
Gadgets
Other Gadgets
Providers
RSS Feed, Cloud, etc
Services
Grid and Web
Services
(TeraGrid, OSG, etc)
Social Network
Services (Orkut,
LinkedIn,etc)
Common science
gateway architecture.
Aggregation is in the
portlet container.
Users have limited
selections of
components.
HTML/HTTP
Tomcat
+
Portlets and
Container
SOAP/HTTP
Grid and Web
Services
(TeraGrid, OSG, etc)
Grid and Web
Services
(TeraGrid, OSG, etc)
Grid and Web
Services
(TeraGrid, OSG, etc)
Making GTLAB Google Gadgets
 A Gadget is a standalone application that is integrated
into your personal iGoogle display.
 They can run on your Web server.
 You can use SSL/HTTPS
 Google Gadgets have two versions
 http: these use Google’s JavaScript APIs
 url: these wrap external applications with IFrames.
 Limitations:
 Not much real estate (but location, location, location)
 Cookies manage state on gadgets from the same server
but we need a way to integrate multiple independent
servers.
Making GTLAB Gadgets into Portlets
 Do all development in jsf_standalone.
 When you are ready to convert:
 cd GTLAB/transition
 mvn clean process-resources
 cd GTLAB/portlets
 mvn clean install
 You will need to manually edit portlet.xml, group.xml,
and layout.xml in
GTLAB/portlet/src/main/webapp/WEB-INF/
 That’s it.
Click “Add stuff” and
then “Add feed or
gadget”. Type the
URL of your gadget.
Various GTLAB
applications deployed
as portlets:
Remote directory
browsing, proxy
management, and
LoadLeveler queues.
Example Gadget Config File
<?xml version="1.0" encoding="UTF-8" ?>
<Module>
<ModulePrefs title="MyProxy Gadget Example"
scrolling="true"
height="500"/>
<Content type="url"
href="http://…/:8080/GTLAB/examples/MyProxyExample.jsf">
</Content>
</Module>
Save this as MyProxy.xml and place
in a Web accessible place.
Under the Hood: Building Components
 Portlets can be built with our Apache Velocity bridge.
 “Template” or “steal this code” approach.
 OGCE portlets use the Java COG Abstraction Layer to
access Grid services.
 Provides a buffer over different Grid toolkits and
versions.
 Provides ability to compose graph-based workflows.
 Tutorial in itself
 We have also developed Grid Tag Libraries and Beans
(GTLAB) to simplify this process.