Transcript OGCE Workflow Suite

OGCE Workflow Suite
Gopi Kandaswamy
Suresh Marru
Srinath Perera
Chathura Herath
Marlon Pierce
TeraGrid 2008
Workflow Suite
• Generic Service Toolkit
–
–
–
–
Tool to wrap command-line applications as web services
Handles file staging & job submissions
Extensible runtime for security, resource brokering & urgent computing
Generic Factory service for on-demand creation of application services
• XRegistry
– Information repository for the OGCE workflow suite
– Register, search, retrieve & share XML documents
– User & hierarchical group based authorization
• XBaya
– GUI based tool to compose & monitor workflows
– Extensible support for compiler plug-ins like BPEL & Jython
OGCE TeraGrid08 Tutorial
2
Features
• Security
–
–
–
–
Authentication and authosrization
Secure invocations between services
Support for gateway community accounts
Support for multiple user accounts
• Reliability
– Retry job submissions and file staging
– Fault Tolerance and Recovery service
• Over-provisioning and migration
• Compatibility
– Taverna, Kepler and Trianna
OGCE TeraGrid08 Tutorial
3
Applications
• LEAD
– Lower entry barrier to using weather analysis tools
– Improve detection, analysis & prediction of mesoscale weather
• Motif-Network
– Transformation of sequenced genomes to “domain-space”
• Cyber-Infrastructure Evaluation
– Performance evaluation of future supercomputer architectures
• ADAM
– Algorithms for feature extraction, data normalization,
classification and normalization
OGCE TeraGrid08 Tutorial
4
LEAD: A Weather Forecasting Workflow (1/2)
Terrain data files
1
3D Model
Data
Interpolator
Terrain
Preprocessor
Surface data,
upper air mesonet data and
wind profiler data
(Initial Boundary
Conditions)
2
WRF Static
Preprocessor
4
88D Radar
Re-mapper
11
15
ARPS
Ensemble
Generator
5
NIDS Radar
Re-mapper
WRF to ARPS Data
Interpolator
ADAM
Visualization on
users request
Satellite Data
Re-mapper
Repeat
periodically
for new data
13
8
6
Real time data Initialization
IDV
WRF
WRF
WRF
12
WRF
10
Radar data
(Level II)
Radar data
(Level III)
Static data
(lateral Boundary
Conditions)
7
ADAS
Satellite
data
3D Model Data
Interpolator
ARPS to WRF
Data
Interpolator
Surface, terrestrial
data files
Run once per
forecast region
9
NAM, RUC, GFS data
3
14
ARPS Plotting
Program
Data mining:
look for storm
signature
Analysis
Data Mining
Forecast Visualization
Triggered if a storm
5
is detected
LEAD: A Weather Forecasting Workflow (2/2)
WRF-Static running
on Tungsten
OGCE TeraGrid08 Tutorial
6
Motif-Network: Whole Genome workflow
• Domain webs of large genomes
– Input list of amino acid sequences
– Identify all known domains
– Construct webs
Ensemble-type processing
(minimal network reqs)
Capacity-type computing
Parallel processing
Capability-type computing
Jeff Tilson, RENCI
OGCE TeraGrid08 Tutorial
7
CI: Execute Sub-Workflow
• Input a campaign step filename
• Execute GAMESS per step
specification
Jeff Tilson, RENCI
OGCE TeraGrid08 Tutorial
8
More Information
Tuesday
1:30 pm
Engaging with the LEAD Science Gateway Project:
Lessons Learned in Successfully Deploying Complex
System Solutions on TeraGrid
Tuesday
3:30 pm
Reliable and Scalable Job Submission: LEAD Science
Gateway's Testing and Experiences with WS GRAM on
TeraGrid Resources
Tuesday
3:30 pm
The LEAD-WxChallenge Pilot Project: The Potential of
Grid-Enabled Learning
Wednesday 10:30 am Workflow Infrastructure for Multi-scale Science
Gateways
Wednesday 4:00 pm
On-Demand Severe Weather Forecasts Using TeraGrid
via the LEAD Portal
OGCE TeraGrid08 Tutorial
9
Questions?
OGCE TeraGrid08 Tutorial
10