Transcript MOCCA – A Distributed CCA Framework based on H2O

MOCCA - H2O-based CCA component
framework for programming grids and
metacomputing systems
Maciej Malawski, Marian Bubak,
Michał Placek, Daniel Harężlak,
Dawid Kurzyniec, Vaidy Sunderam
Institute of Computer Science AGH, Kraków, Poland
Academic Computer Centre CYFRONET-AGH, Kraków, Poland
Distributed Computing Laboratory in the Dept. of Math and Computer
Science, Emory University, Atlanta
Institute of Computer Science
AGH
Outline
• Motivation – programming grids
• Component approach and CCA as a
component standard
• H2O resource sharing platform
• Design and implementation of MOCCA
• Applications and initial results
• Future research – GCM interoperability
2
Institute of Computer Science
AGH
Problem – how to program the
Grid
• Many programming models:
– MPI
– Service Oriented Architectures, Web Services
– Tuple spaces, HLA
– Distributed Objects
– Custom protocols
– Components
– ...
3
Institute of Computer Science
AGH
Common Component
Architecture
• Component standard for High Performance
Computing
• Uses and provides ports described in SIDL
• Support for scientific data types (complex
numbers, data arrays)
• Existing tightly coupled (CCAFFEINE) and loosely
coupled, distributed (XCAT) frameworks
4
Institute of Computer Science
AGH
Existing CCA Frameworks
• CCAFFEINE
– Tightly coupled
– Support for Babel
– MPI support
• XCAT
– Loosely coupled
– Globus-compatible
– Java-based
• DCA
– MPI based
– MxN problems
• SCIRun2
– Metacomponent model
• LegionCCA
– Based on Legion Metacomputing system
5
Institute of Computer Science
AGH
Requirements for a
Metacomputing-oriented CCA
Framework
• Facilitated deployment - provide easy mechanisms for
creation of components on distributed shared resources;
• Efficient communication - both for distributed and local
components;
• Flexible - allow flexible configuration of components and
various application scenarios;
• Support native components, i.e. components written in
non-Java programming languages and compiled for specific
architecture.
• Interoperable with Grid standards (Web services)
Solution: H2O
6
Institute of Computer Science
AGH
H2O Resource sharing
platform
• Providers own resources
• They independently share
them over the network
– access control policies
• Clients discover, locate,
and utilize resources
• Resources configurable via
plugins
• Aggregation and reselling:
cascading pairwise
relationships
7
Institute of Computer Science
AGH
H2O Component Model
•
Nomenclature
Clients
– container = kernel
– component = pluglet
•
Pluglet = remotely accessible object
– implements Pluglet interface, used by
kernel to signal/trigger pluglet state
changes
Functional
interfaces
Interface Pluglet {
void init(RuntimeContext cxt);
void start();
void stop();
void destroy();
}
•
(e.g. Hello)
Pluglet
Pluglet
Remote access: based on the RMI model
Kernel
– Pluglets export functional remote interfaces
tutorial/step1/srv/Hello.java
public interface Hello extends Remote {
String hello() throws RemoteException;
}
8
Institute of Computer Science
AGH
Example scenarios of H2O
Registration and Discovery
Publish
UDDI
JNDI
LDAP
DNS
GIS
e-mail,
phone, ...
...
Find
Deploy
A
A
Deploy
Client
Provider
Provider
Provider
native
code
B
A
B
A
Client
Client
B
Provider
Deploy
Legacy
App
Repositor
y
A
Reseller
B
C
1. Provider = deployer

e.g. resource = legacy
application
Developer
A
B
C
3. Client = deployer
2. Reseller:=
developer = deployer

9
Repositor
y
e.g. computational service
offered within a grid system

e.g. client runs custom
distributed application on
shared resources
Institute of Computer Science
AGH
RMIX Communication
Substrate
• Extensible framework
• Remote Method Invocations
paradigm
Service
• Pluggable protocol providers
• Multiple protocols supported
RMIX
– JRMPX, ONC-RPC, SOAP
RMIX
RMIX
JRMPX XSOAP
RMIX
RPCX
RMIX
JXTA
• Request-Response and
Asynchronous calls
• Combines simplicity, flexibility,
and performance
10
Java
Web Services ONC-RPC
JXTA
SOAP clients
Institute of Computer Science
AGH
RMIX: multiple protocols
• Protocol switching
• Protocol negotiation
H2O Kernel
• Various protocol stacks for
different situations
security
– SOAP: interoperability
– SSL: security
– ARPC, custom (Myrinet,
Quadrics): efficiency
H2O Kernel
Internet
firewall
H2O Kernel
efficiency
H2O Kernel
Harness
Kernel
efficiency
H2O Kernel
11
Institute of Computer Science
AGH
RMIX over JXTA
• Fully operational RMI implementation running over
JXTA P2P network
• Methods can be
invoked on remote
objects located
behind firewalls or
NATs
• Our implementation
of JXTA socket
factories manages
all the JXTA
connectivity
transparently from
user’s point of view
12
Institute of Computer Science
AGH
MOCCA Implementation in H2O
• Each component running in separate pluglet
– Facilitated deployment and security
• Thanks to H2O kernel security mechanisms, multiple components
may run without interfering
• Using RMIX for communication – efficiency, multiprotocol
interoperability
• Flexibility and multiple scenarios – as in H2O
• MOCCA_Light: pure Java implementation - need for supporting
multilanguage components
13
Institute of Computer Science
AGH
Remote Port Call
4. RMIX call
Component Pluglet
Component Pluglet
MOCCA
Services
5. call
1. getPort
CCA
Component
User Side
2. create
3. call
Dynamic
Port
Proxy
CCA
Component
Provider Side
14
Institute of Computer Science
AGH
How to use MOCCA
(step by step)
• Implement component code extending CCA interfaces
(cca.Port, cca.Component)
• Compile component classes into JAR file
• Publish application JARs on HTTP server
• Use the Java client API or write a Jython script to assemble
application from components
– Specify components and their connections
– Specify locations of H2O kernels where to instantiate
components
• Running the script automatically deploys necessary pluglets
into H2O kernels and spawns application
15
Institute of Computer Science
AGH
Example script
builder = MoccaMainBuilder()
uriKernel1 = URI.create("http://emily.mathcs.emory.edu:7800/")
uriKernel2 = URI.create("http://zeus10.cyf-kr.edu.pl:7800/")
userBuilderID = builder.addNewBuilder(uriKernel1, "MyBuilderPlugletA")
providerBuilderID = builder.addNewBuilder(uriKernel2, "MyBuilderPlugletB")
properties = MoccaTypeMap()
properties.putString("mocca.plugletclasspath",
"http://emily.mathcs.emory.edu/mocca/mocca-samples.jar")
properties.putString("mocca.builderID", userBuilderID.getSerialization())
userID = builder.createInstance("My StarterComponent",
"mocca.samples.pingpong.impl.MoccaStarterComponent”,
properties)
properties.putString("mocca.plugletclasspath",
"http://emily.mathcs.emory.edu/mocca/mocca-samples.jar")
properties.putString("mocca.builderID", providerBuilderID.getSerialization())
providerID = builder.createInstance("MyPingComponent",
"mocca.samples.pingpong.impl.PingPongComponent",
properties)
connectionID = builder.connect(userID,
"PingPongUsesPort",
providerID,
"PingPongProvidesPort")
MoccaBuilderClient.invokeGo(userID)
16
Institute of Computer Science
AGH
Automatic Flow Composer
Example
• Compose application graph
from initial data (e.g. initial
ports) or incomplete graph
• First implemented for XCAT
framework
Flow
Composer
Lookup
Component
Registry
Compose
Flow
Optimizer
Evaluate
Link
Evaluator
• Easy migration to MOCCA
• Modification of code
required (xcat.Port)
Evaluate
Site
Evaluator
• Similar performance for
XCAT and MOCCA (exchange
of text documents)
17
Institute of Computer Science
AGH
Communication Intensive
Application Benchmark
• Simplified scenario:
– 2 components
– Provides port: receive and send-back array of double
(ping-pong)
• Tested on local Gigabit Ethernet and on transatlantic Internet
between Atlanta and Krakow
• 2.4 GHz Linux machines
• Comparison with XCAT
18
Institute of Computer Science
AGH
Small Data Packets
Factors:
• SOAP header overhead in XCAT
• Connection pools in RMIX
19
Institute of Computer Science
AGH
Large Data Packets
• Encoding (binary vs. base64)
• CPU saturation on Gigabit LAN (serialization)
• Variance caused by Java garbage collection
20
Institute of Computer Science
AGH
Example: modeling gold clusters
•
Clusters of atoms
•
Modeling of clusters
– Very interesting forms
between isolated atoms or
molecules and solid state
– Important for the technology
of constructing nanoscale
devices.
– Several energy minimization
methods such as
MDSA or L-BFGS,
– Choosing an empirical
potential
– Highly compute-intensive
– The optimal result depends on
the number of possible
iterations and initial
configurations for each
simulation run.
21
Institute of Computer Science
AGH
Example – deployment
22
Institute of Computer Science
AGH
Integration with existing Grid
middleware
User's
virtual
resource
pool
NS
lookup()
bind()
H2O H2O
H2O
H2O
H2O
H2O
Resource
Broker
SSH
Grid node
Standalone
machine
Cluster
PBS
LCG
• A pool of computing resources may be created by
submitting a number of H2O kernels on many Grid sites
• Application components may be deployed on the kernels
belonging to the pool
23
Institute of Computer Science
AGH
Multilanguage support: motivation
• Grids are heterogeneous
• Multiple programming languages – in single application
–
–
–
–
Java for middleware
C for system programming
FORTRAN for computing
Python for scripting
• Multiple protocols – in single application
–
–
–
–
High speed local networks (Myrinet)
TCP/SSL/TLS in WAN
SOAP for loosely coupled message exchange
Overlay P2P networks for traversing private network
boundaries (NATs)
• Context: MOCCA component framework
24
Institute of Computer Science
AGH
Multilanguage Solution - Babel
•
•
•
SIDL – Scientific Interface Definition Language
–
Standard for CCA Components
–
Supports arrays and complex types
–
Focus on interfaces
SIDL parser
–
Code generator
–
Runtime library
Intermediate Object
Representation (IOR)
–
Core of Babel object
–
Array of function
pointers
–
Generated code in C
f90
Python
C++
package example version 1.2 {
class Hello {
string hello( in string hello);
}
}
/**
* Method: hello[]
*/
char*
example_Hello_hello(
/*in*/ example_Hello self,
/*in*/ const char* hello);
25
C
Babel
Babel:
–
f77
Java
// user defined non-static methods:
/**
* Method: hello[]
*/
public java.lang.String hello_Impl (
/*in*/ java.lang.String hello )
{
// DO-NOT-DELETE splicer.begin(example.Hello.hello)
// Insert-Code-Here {example.Hello.hello} (hello)
return ”Server says: ” + hello;
// DO-NOT-DELETE splicer.end(example.Hello.hello)
}
Institute of Computer Science
AGH
Currently: Babel for Local
Applications
• All Babel objects in
one process
• Implemented in
CCAFFEINE
framework
• Existing
multilanguage CCA
components – see
CCA tutorial
26
Fortran
native
library
SIDL
Java
application
Babel IOR
Babel IOR
SIDL
C++
native
library
Institute of Computer Science
AGH
Our Solution
• Babel + RMIX
• Implementation of
Babel RMI
extensions
– generic mechanism
of method invocation
(reflection)
– Dynamic loading of
communication
library
– No need for code
generation and
compilation
27
Network
RMIX
library
RMIX
library
SIDL
SIDL
Babel IOR
Java
application
Babel IOR
SIDL
SIDL
C++
native
library
Fortran
native
library
Institute of Computer Science
AGH
Beyond CCA ?
• Supporting multiple component standards
– Goal: to enable loading of components written for different
standards (e.g. Corba CCM, other)
– Examples of similar solutions: CCAFFEINE supports „classic”
and „Babel” components; SCIRun2 implementing metacomponent model
• Using MOCCA as promising platform for feasibility studies in
various aspects of Grid components
– For experiments with advanced features
• Scheduling and load-balancing
• Fault-tolerance
• Semantic description and composition
– As a platform for higher-level grid services and tools
28
Institute of Computer Science
AGH
Fractal/GCM – CCA
Interoperability?
• CCA as Fractal
– Adapter calls setServices() on a
component
– Component registers ports on the
adapter
– Component is ready for introspection
and connection
C
C
C
BC
Adapter
Services
CCA
Component
• Fractal as CCA
– CCA framework creates component,
adapter and invokes setServices()
– Adapter introspects the component
and registers interfaces to the
framework
– Adapter obtains references to
external interfaces (getPort()) and
binds them to the component
29
Services
Adapter
C
BC
Fractal
Component
Institute of Computer Science
AGH
References
• Maciej Malawski, Dawid Kurzyniec, and Vaidy Sunderam. MOCCA
– towards a distributed CCA framework for metacomputing,
Accepted for: 10th International Workshop on High-Level
Parallel Programming Models and Supportive Environments
(HIPS2005) at IPDPS’2005
http://mathcs.emory.edu/dcl/h2o/papers/h2o_hips05.pdf
• H2O Project homepage: http://www.mathcs.emory.edu/dcl/h2o/
• CCA Forum: http://www.cca-forum.org
– CCA Specification
– Tutorial
• MOCCA homepage: http://www.icsr.agh.edu.pl/mambo/mocca
– Download binary and source distribution
– README
30
Institute of Computer Science
AGH