Transcript Slide 1

GEODISE: Grid-enabled toolkits
for the Engineer
Andrew Price
UK e-Science Programme, Booth 2140
SC2004, Pittsburgh, USA
10th November 2004
© Geodise Project, University of Southampton, 2004.
http://www.geodise.org/
Grid Enabled Optimisation and Design
Search for Engineering (GEODISE)
Simon Cox-
Technical Director
Southampton e-Science Centre. Grid/ W3C
Technologies and High Performance
Computing
Andy Keane-
Director of Rolls Royce/
BAE Systems University Technology
Partnership in Design Search and
Optimisation
Mike Giles-
Director of Rolls Royce
University Technology Centre for
Computational Fluid Dynamics
Carole Goble-
BAE Systems- Engineering
Rolls-Royce- Engineering
Fluent- Computational Fluid Dynamics
Microsoft- Software/ Web Services
Intel- Hardware
Compusys- Systems Integration
Epistemics- Knowledge Technologies
Condor- Grid Middleware
Ontologies and DARPA
Agent Markup Language (DAML) /
Ontology Inference Language (OIL)
Nigel Shadbolt-
Director of Advanced
Knowledge Technologies (AKT) IRC
© Geodise Project, University of Southampton, 2004.
http://www.geodise.org/
The GEODISE Team
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
Richard Boardman
Sergio Campobasso
Liming Chen
Mike Chrystall
Trevor Cooper-Chadwick
Simon Cox
Mihai Duta
Clive Emberey
Hakki Eres
Matt Fairman
Mike Giles
Carole Goble
Ian Hartney
Tracey Hunt
Zhuoan Jiao
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
Andy Keane
Marc Molinari
Graeme Pound
Colin Puleston
Nicola Reader
Angus Roberts
Mark Scott
Nigel Shadbolt
Wenbin Song
Paul Smart
Barry Tao
Lisa Telfer-Brunton
Ivan Voutchkov
Jasmin Wason
Fenglian Xu
Gang Xue
© Geodise Project, University of Southampton, 2004.
http://www.geodise.org/
Outline
•
•
•
•
•
Description of Design Search and Optimisation
GEODISE Solution
Compute Toolbox
Database Toolbox
Applications
© Geodise Project, University of Southampton, 2004.
http://www.geodise.org/
Design Optimisation
•
What is Engineering Design Search & Optimisation (EDSO)?
 Improving designs by exploiting engineering modelling and analysis.
 Systematically modifying variables in a design to increase, or reduce a
quality measure (called the objective function), within certain
constraints.
 Involves computational and data intensive processes.
© Geodise Project, University of Southampton, 2004.
http://www.geodise.org/
Geodise Solution
•
•
Bring Grid Technology to Engineers
 Develop tools that are easy to learn and to use.
 Integration into environments familiar to engineers (e.g. Matlab, Jython).
Geodise Toolboxes
Compute Toolbox – enables applications to run on the Grid.
Database Toolbox – manages data produced by the grid applications.
XML Toolbox – converts Matlab/Jython data structures to/from XML.
Knowledge Toolbox – uses knowledge from domain experts to provide
advice; tools to support function and workflow annotations.
Workflow Tool – helps engineers to construct, execute, monitor and
manage workflows.
© Geodise Project, University of Southampton, 2004.
http://www.geodise.org/
Compute Toolbox Architecture
Matlab
Environment
script.m
gd_jobsubmit.m
gd_getfile.m
Java CoG
Globus GT2
Resource
GRAM
job submission
GridFTP
file transfer
GRAM
Service
GridFTP
Service
© Geodise Project, University of Southampton, 2004.
http://www.geodise.org/
Certificate Management Functions
gd_certinfo
Returns information about the user's certificate
gd_createproxy
Creates a Globus proxy certificate
gd_proxyinfo
Returns information about the user's proxy certificate
gd_proxyquery
Queries whether a valid proxy certificate exists
gd_destroyproxy
Destroys the local copy of the user's Globus proxy
certificate
© Geodise Project, University of Southampton, 2004.
http://www.geodise.org/
Certificate Management
C:\GEODISE
>> gd_createproxy
© Geodise Project, University of Southampton, 2004.
http://www.geodise.org/
Job Submission Functions
gd_jobstatus
Gets the status of a Globus GRAM job
gd_jobsubmit
Submits a compute job to a Globus GRAM job manager
gd_jobpoll
Queries the status of a Globus GRAM job until complete
gd_jobkill
Kills a Globus GRAM specified by a job handle
gd_chmod
Changes file permissions of a file on a Globus resource
gd_condorsubmit
Submits a job to a Condor pool via a Globus resource
gd_submitunique
Submits a GRAM job to a unique working directory
gd_listjobs
Returns all the job handles belonging to the user
© Geodise Project, University of Southampton, 2004.
http://www.geodise.org/
Job Submission
C:\GEODISE
>> RSL = '&(executable="/bin/date")(stdout="date.out")';
>> host = 'grid-compute.oesc.ox.ac.uk';
>> jobhandle = gd_jobsubmit(RSL, host)
jobhandle =
https://escience-dept2.sesnet.soton.ac.uk:30001/30705/1098694366/
>> isdone = gd_jobpoll(jobhandle)
isdone =
1
>>
© Geodise Project, University of Southampton, 2004.
http://www.geodise.org/
GridFTP File Transfer
gd_getfile
Retrieves a remote file using GridFTP
gd_putfile
Puts a remote file using GridFTP
gd_transferfile
Performs a third-party file transfer using GridFTP
gd_makedir
Creates a remote directory using GridFTP
gd_listdir
Lists the contents of a directory on a GridFTP resource
gd_fileexists
Tests the existence of files on a Globus resource
gd_rmdir
Deletes a remote directory using GridFTP
gd_rmfile
Deletes a remote file using GridFTP
gd_rmuniquedir
Deletes a remote directory and its contents
© Geodise Project, University of Southampton, 2004.
http://www.geodise.org/
File Transfer
C:\GEODISE
>> exists = gd_fileexists(host, 'date.out')
exists =
1
>> gd_getfile(host,'date.out','localfile.txt');
>> type('localfile.txt')
Mon Oct 25 09:52:46 BST 2004
>> gd_rmfile(host, 'date.out');
>> exists = gd_fileexists(host, 'date.out')
exists =
0
>>
© Geodise Project, University of Southampton, 2004.
http://www.geodise.org/
Data Management Challenges
1 0 0
0 2 0
0 0 3
• Data may be generated from distributed applications running on the
Grid.
• Traditionally data stored on file systems with little descriptive
information – hard to find and share.
• Engineers may want to access the data from distributed locations.
© Geodise Project, University of Southampton, 2004.
http://www.geodise.org/
Need Metadata
Database
Archive
Metadata
Metadata
airfoil
cad
obj=2
Locate
Files
Globus
Server
Archive
1 0 0
x.y.z = 43 0 2 0
0 0 3
Files
© Geodise Project, University of Southampton, 2004.
http://www.geodise.org/
Database Toolbox Overview
•
•
•
•
•
Store data with descriptive information
 Standard and application specific metadata.
 Query over metadata to easily locate required data.
 Data retrieval based on ID rather than location.
Familiar interface for engineers
 Wrap toolbox as Matlab/Jython functions – easy integration.
 Can be used in Matlab/Jython scripts – popular among engineers.
Support data aggregation (data groups) concept.
Central and local databases (shared vs. personal).
Secure Web service access to central database over SSL.
 Certificate-based authentication and authorisation.
© Geodise Project, University of Southampton, 2004.
http://www.geodise.org/
Architecture
Client
Grid
Geodise Database
Toolbox
Globus Server
Geodise
Matlab
Functions
GridFTP
Jython
Geodise
Jython
Functions
Java
Client
Code
Location
Service
CoG
Apache
Axis
WCE
Geodise Database
Web Services
SOAP
Authorisation
Service
Metadata
Archive & Query
Services
© Geodise Project, University of Southampton, 2004.
http://www.geodise.org/
Metadata
Database
Applications - Geodise
CFD 3D engine nacelle optimisation
0
1
2
Conventional Inlet
•
•
Negative Scarf Inlet
Goal - reduce ground noise
generated by fan when plane
takes off.
Optimise aerodynamic
performance when scarf angle
is varied.
© Geodise Project, University of Southampton, 2004.
http://www.geodise.org/
Applications - GEM
Electromagnetic optimisation
Transmission of light properties through a photonic
crystal.
Parameters are radius of holes and light wavelength.
•
•
Large number of
designs, parameters
and solutions.
Query for a particular
data range to postprocess.
© Geodise Project, University of Southampton, 2004.
http://www.geodise.org/
Applications – GENIE
Tuning Earth system model components
•
•
•
GENIE supports environmental scientists modelling long term climate change.
Optimising a parameterised model over a multi-dimensional state space.
Database used for monitoring, sharing, post-processing and fault tolerance.
Default
Tuned
Target
© Geodise Project, University of Southampton, 2004.
http://www.geodise.org/
Summary
•
•
Engineer
Design Search and Optimisation
Computation/ Middleware
 Compute Toolkit
• Globus & Condor
• Using UK Level 2 Grid
•
•
 Workflow in Matlab
Database
 XML Toolkit
 Database Toolkit
GEODISE
PORTAL
Knowledge
repository
Ontology for
Engineering,
Computation, &
Optimisation and
Design Search
Visualization
Session
database
Traceability
OPTIMISATION
OPTIONS
System
APPLICATION
SERVICE
PROVIDER
Intelligent
Application
Manager
Reliability
Security
QoS
CAD System
CADDS
IDEAS
ProE
CATIA, ICAD
• Archive for Files
• Archive for Metadata
• Query & Retrieve
Geodise toolkits exploited by a
variety of applications
Globus, Condor, OGSA
Optimisation
archive
COMPUTATION
Licenses
and code
Analysis
CFD
FEM
CEM
Parallel machines
Clusters
Internet Resource Providers
Pay-per-use
Design
archive
“Build
complex
things from
lots of simple
things”
© Geodise Project, University of Southampton, 2004.
http://www.geodise.org/
Intelligent
Resource
Provider