Transcript TeraGrid Progress and Status

Use of Web Services
Stephen Mock
San Diego Supercomputer Center
TeraGrid 2007
Madison, WI
Outline
• What Are Web Services?
• Services Offered by and for TeraGrid
• Example
• Available Documentation
June 2007
TeraGrid 2007
2
Example Job Gateway WS Use
1. How long will this job
take on TG-Resource X?
UC
Santa Barbara
TeraGrid
MDS
Batch Queue
Prediction
Service
MDS4
Info
Service
What is the endpoint of
WS-Gram and RFT
On TG-Resource X?
Web Services
TG-Resource X
Gateway
Submit/Monitor Job
WS-GRAM
Job Submission
Stage and unstage files
RFT
File Transfer
June 2007
TeraGrid 2007
3
What Are Web Services?
• Interoperable machine-to-machine application
accessed over a network
– No Graphical User Interface (GUI)
– XML used to define the datatypes and operations
• Language and Operating System Independent
• XML documents passed back and forth
• Services described in standardized XML document
called WSDL
• Stateless
June 2007
TeraGrid 2007
4
Why Use Web Services
• If you’re building software that makes use of
remote data or operations
• Want to use a programmatic API within your
software
• Don’t care about Platform/Language of the web
service
June 2007
TeraGrid 2007
5
Services Offered By TeraGrid
• Globus Toolkit 4 Services
– MDS4 : Information Discovery
– WS-Gram : Job Submission, control, and monitoring
– RFT : File movement
– Delegation Service : Credential Delegation
– GRAM4-Audit : Job Cost Information
• NWS Batch Queue Prediction Service
June 2007
TeraGrid 2007
6
MDS4
• GT4 Service
• Each TG Resource has an IndexService
– Contains resource-specific information
– Service locations, software versions, #’s of CPUs, free CPUs, queue
names
– Reports info to TG-wide IndexService
• MDS.TERAGRID.ORG
– Aggregating point for TG-wide information
• Authenticated MDS
– Requires credentials for access
– Sensitive info like queue listings
• WebMDS website listing MDS info
– http://mds.teragrid.org:8080/webmds/webmds?info=indexinfo&xsl=
sgservicetablexsl
June 2007
TeraGrid 2007
7
WS-Gram
• GT4 Service
• Web Service version of GRAM from GT 2.x
• Job Submission
– Either ‘fork’ or to a scheduler like pbs, lsf, loadleveler
• Job Defined in XML
• Simple Job Example
<?xml version="1.0" encoding="UTF-8"?>
<job>
<executable>/bin/echo</executable>
<directory>${GLOBUS_USER_HOME}</directory>
<argument>Welcome to the Teragrid.</argument>
<stdout>${GLOBUS_USER_HOME}/echo-stdout.txt</stdout>
<stderr>${GLOBUS_USER_HOME}/echo-stderr.txt</stderr>
</job>
June 2007
TeraGrid 2007
8
Reliable File Transfer (RFT)
• GT4 Service
• Transfer files to and from TG-Resources
• Relies on GridFTP’s 3rd party transfer
• Adds reliability, retry, and recovery over globus-urlcopy
Data Source
myFile
TG-Resource
1
1. Please move myFile
To TG-Resource 2
GridFTP 3rd PartyTransfer
Third Party Transfer
June 2007
TeraGrid 2007
Data
Destination
TG-Resource
2
9
NWS Batch Queue Prediction Service
• Predict queue wait time for specific job size
• Predict probability specific job will run given a
deadline
• No credentials needed
• Developed by and running at UCSB
• http://nws.cs.ucsb.edu/ewiki/nws.php?id=Batch+Queue+Prediction
June 2007
TeraGrid 2007
10
NWS Code Example
package edu.ucsb.cs.nws;
public class NwsBatchqClient {
public static void main(String[] args) throws Exception {
NwsBatchqServiceLocator loc =
new NwsBatchqServiceLocator();
NwsBatchqSoapBindingStub stub =
(NwsBatchqSoapBindingStub) loc.getNwsBatchq();
System.out.println(stub.getMachines()); //prints list
System.out.println(stub.predict(0, //timestamp
"datastar", //machine
"normal", //queuename
4, //# of nodes
34l, //wallTime (s)
0, 0));
}
}
June 2007
TeraGrid 2007
11
Example Job Gateway WS Use
1. How long will this job
take on TG-Resource X?
UC
Santa Barbara
TeraGrid
MDS
Batch Queue
Prediction
Service
MDS4
Info
Service
What is the endpoint of
WS-Gram and RFT
On TG-Resource X?
Web Services
TG-Resource X
Gateway
Submit/Monitor Job
WS-GRAM
Job Submission
Stage and unstage files
RFT
File Transfer
June 2007
TeraGrid 2007
12
Documentation
• Documentation on TG Wiki
• http://www.teragridforum.org/mediawiki/index.php?title=TeraGrid_Science_Gateways_Primer
• http://www.teragridforum.org/mediawiki/index.php?title=Teragrid_Web_Services_Documentation
June 2007
TeraGrid 2007
13