Transcript Objectives
Optimized Java computing as
an application for Desktop Grid
Olejnik Richard1, Bernard Toursel1, Marek Tudruj2,
Eryk Laskowski2
1
Université des Sciences et Technologies de Lille
Laboratoire d’Informatique Fondamentale de Lille (LIFL UMR CNRS 8022)
{olejnik, toursel}@lifl.fr
http://www.lifl.fr/PALOMA
2 Institute of Computer Science Polish Academy of Sciences Warsaw, Poland
{tudruj, laskowsk}@ipipan.waw.pl
Components for Desktop Grid
2
Heterogeneous Applications
• Each application is composed of one or
more tasks.
• Different tasks may have different
computational needs.
• There may be inter-task communication.
• Each task can be assigned to a computer
with any architecture.
3
Grid'5000
Grid'5000 is a research effort developing a large
scale infrastructure for Grid research. 10
laboratories are involved, in the objective of
providing the community of Grid researchers a
testbed allowing experiments in all the software
layers between the network protocols up to the
applications.
The current plans are to assemble a physical platform
featuring 8 clusters, connected by the Renater
Education and Research Network at 1Gb/s (10
Gb/s is expected in future).
4
Objectives
• Execution efficiency of parallel and
distributed Java applications on Grid.
• Transparent and optimized object
placement with dynamic load balancing
strategies (execution aspects).
• Transparent control of parallelism and
easy collecting of results (programming
aspects).
5
What we propose
•
Single System Image (SSI) of clusters.
•
Special mechanisms at the middleware level:
•
–
Dynamic and automatic adaptation to variations of
computation methods and execution platforms.
–
Special mechanisms at the programming
environment level, which facilitate expression of
parallelism and distribution.
Using Components in Grid environment
6
Issues
• Issues in building distributed applications
– Collaboration between different applications
– Cross programming languages and platforms
– Controlling parallel applications transparently
– Managing software complexity and evolution
– Reuse and sharing of existing scientific code
– Encapsulation and modular construction
7
CCADAJ Component
Architectures
• Control component library
– Provides parallel/distributed control
– Connects components in a parallel way
– Data exchange between components:
- Demand driven & Data driven
- Event notification
• Multiple level composition
• CCADAJ features
– CCA Compliant (Common Component Architecture
http://www.cca-forum.org/)
– Services
- Instantiation, Connection
- ConnectionEvent, ComponentEvent
- Composition
9
CCA Features
• Components and Ports
– Components interact through interfaces (ports)
– Components can provide ports:
- implementation of port interfaces
- the service a component offers
– Components can use ports
- Call of the method in the port
- Capability the component needs to use
– Connecting components through ”provides-uses” ports
10
CCA: Interactions between
components
GoPort
MultiplierPort
StarterComponent
•
AdderPort
MultiplierComponent
AdderPort
AdderComponent
GoPort
–
–
–
•
MultiplierPort
Special port to “execute” a component
Implements go() method, which starts
execution of the component
Framework search for go ports and uses
them
•
–
•
–
•
”No uses” / ”uses neither provides”
/”provides”
Ports connected by types
–
–
UsesPort (multiplierPort ,AdderPort)
• Call getPort to obtain port by services
Call method on port
• Ex: x=multiplierPort.getProduct(y,z);
Connection uses/provides ports
Port types must match
Port names are unique in a component
Framework puts info about provider
into user component’s service object
connect StarterComponent MultiplierPort MultiplierComponent MultiplierPort
connect MultiplierComponent AdderPort AdderComponent AdderPort
11
Composition (super-component)
ProvidesPort
UsesPort
• Super component
– Component
- Incorporates ”provides”,
”uses”, and/or ”go” ports
InnerProvides
GoPort
InnerUses
InnerComponent
InUses2
InnerComp2
Some
Additional
Services
SuperComponent
– Additional services
(framework)
- Instantiation of inner
components
- Mechanism for exposing
inner component’s ports
to the outside
- Inner components
activations
- Inner components
connection
12
The static optimization heuristics
Before a Java program is executed on a GRID,
an introductory optimization algorithm is
performed, which will determine an initial
distribution of program elements (objects) on
Java Virtual Machines.
The algorithm starts with creating a method
dependence graph of a Java program. In a
MDG, methods are shown as nodes, their
mutual calls are shown as edges.
13
The optimization algorithm
• Execute programs for some representative data
• Carry out the measurements of the number of
mutual method calls and the number of new
threads spawnings
• Store measurement results in a trace file
• Create a method call graph with the use of
method dependency graph and trace file
• Perform clustering and mapping phases:
- in the clustering phase, the algorithm merges pairs of
nodes from MCG if it leads to a reduction of the program
execution time,
- the mapping phase assigns clusters to the real physical
JVMs with load balancing.
14
class Test
class Producer
class CubbyHole
class Consumer
1
c1
h1
p1
10
Method
Call Graph
(MCG)
run
(te8)
10
get
(me39)
run
(te25)
1
put
(me47)
p2
h2
c2
1
main
(me57)
10
run
(te8)
10
get
(me39)
run
(te25)
put
(me47)
1
1
c5
h5
p5
10
run
(te8)
thread spawn
method call
10
get
(me39)
put
(me47)
run
(te25)
Dynamic Objects Redistribution
• Workload is computed as a function of the
intensity of method invocations
• Two-phase algorithm is performed concurrently
with application execution:
– to detect an application distribution imbalance
– to correct the imbalance in a distributed manner
• Objects from overloaded machines are
transferred to the underloaded ones
16
Observation mechanism of relations
between objects
JVM
global
object
+
local
object
JVM
global
object
global
object
inputInvocation (II)
outputLocalInvocation (OLI)
local
object
+
local
object
outputGlobalInvocation (OGI)
local
object
Legend
•
•
invocation
sum
JVM
17
Inter-Object Dynamic Relation
Graph
18
Conclusions
Some optimization algorithms for distributed Java
programs based on analysis of graph representations.
The optimization explores both static and dynamic
dependencies between objects and their methods.
The proposed static analysis is used as a preliminary
stage that is followed by a dynamic load balancing,
which, in most cases, exploits a little of static
information coming from Java source code.
19
A dynamic load balancing mechanism is used,
supported by three kinds of information:
information about the computer load and
performance, information about dynamic relation
between objects and information deduced from
code analysis.
Compile-time optimizations do not introduce any
penalty in execution time of the program, and
thus, they can use more sophisticated heuristics,
which give better results.
20
Future Works
• CCADAJ development
– Add new functionality
– Emphasis on parallel/distributed control
components
– Component deployment
– Collaboration with other CCA-Compliant
frameworks
• Real problem solving
21