Transcript Enhancing Real-time CORBA via Real

Enhancing Real-time CORBA
via Real-time Java features
Arvind S. Krishna
Douglas C. Schmidt
Raymond Klefstad
Elec & Comp. Eng. Dept
Elec & Comp. Eng. Dept
Vanderbilt University
University of California, Irvine
[email protected]
{arvindk, schmidt}@dre.vanderbilt.edu
International Conference on Distributed Computer Systems
Tokyo, Japan
Friday, March 19, 2004
Talk Outline
Tech transitions in the DRE domain  Real-time
middleware & Real-time Java
− RT-Java + real-time middleware  ZEN project
− ZEN R&D process
−
−
−
−
−
Design & Architecture
Applying Real-time Java features
Empirical Results
Concluding remarks and References
DRE Domain: Characteristics
Types
• Wide applicability
Total Ship Computing Environments
Total Ship C&C Center
• Range from Total ship-board computing
systems to Industrial process control
systems
• Common Requirement
• Right answer delivered too late becomes
wrong answer
Characteristics and Requirements
• Distributed Systems require
•
•
capabilities to manage connections and
message transfer between separate
machines
Real-time Systems  require
predictable and efficient control over end-toend system resources
Embedded Systems  have weight,
cost, and power constraints that limit their
computing and memory resources
Industrial Process Control
Increasingly DRE applications are combined to form “systems of systems”
Current Real-time Middleware Trends
• Current Real-time CORBA ORBs are
developed in C and C++
– ACE+TAO from ISIS Vanderbilt
– ORBexpress from OIC
– eORB from Prism Technologies
• Real-time CORBA has not been
universally adopted by DRE application
developers
– Complexity of the CORBA C++
mapping
– Steep learning curve caused by
feature rich and complex C++
language
• Increasingly hard to find “good” C++
application developers and retain them
Java Programming Language has emerged
having less complexity than C++
– Safety
– Simplicity
– Productivity
However, Java is not suitable for
developing DRE applications:
 The scheduling of Java threads is
purposely under-specified (so to allow
easy implementation of JVM on as many
platform as possible)
 The GC can preempt for unbounded
amount of time Java Threads
 Java provides coarse-grained control
over memory allocation, and it does not
provide access to raw memory
 Java does not provide high resolution
time, nor access to signals, e.g. POSIX
Signals
The Real-Time Specification for Java (RTSJ)
extends Java in the following areas:
• New memory management models that can
be used in lieu of garbage collection
• Stronger semantics on thread and their
scheduling
• Access to physical memory
• Asynchronous Event handling mechanism
• Timers and Higher time resolution
• Priority pre-emptive scheduler
RTSJ Thread & Memory Models
Scoped Memory
Region based memory based tied to number of
active threads in that region
Properties
Any thread may create a scoped region – using new
operator
However, only a real-time thread may allocated from
that region
Immortal Memory
• Same lifetime as the JVM
• Objects allocated never garbage
collected
RTSJ Thread Model
Real-time Threads – priority and
scheduling characteristics specified
NoHeapRealtimeThreads
• Do not “touch” the heap
• Use of NHRT threads can have
exec eligibility higher than that of
GC
Physical Memory
• Allows access to specific locations
based on addresses.
Scoped Memory in Action
Obj A
created in
Heap region
Heap
(new RealtimeThread(…))
.start()
new A(…)
time
Scoped Memory in Action
Scoped Memory
Note: ma1 is
reference is
in heap,
Ma1 is an “inner
scope”
can hold
references only
to “outer regions”
<LTMemory>
Heap
ma1
Reference count = 1
Reference count = 0
Properties
• Reference counted; no of active threads
in region
• When reference count of a region drops
to zero
– All Objects within that region
are considered unreachable
– Finalizers of all
objects run;
Logic instance of
Assignment Rules Runnable
• obj in region ma can
hold
ref to obj in
ma1
 Current
region mb if
Allocation Context
lifetime (ma) <= lifetime (heap)
logic1
(new RealtimeThread(…))
.start()
time
ma1.enter(logic1)
ma1 = new LTMemory(…)
Talk Outline
Tech transitions in the DRE domain  Real-time
middleware & Real-time Java
− RT-Java + real-time middleware  ZEN project
−
−
−
−
Design & Architecture
Applying Real-time Java features
Empirical Results
Concluding remarks and References
Motivation for ZEN Real-time ORB
Integrate best aspects of several key technologies
• Java: Simple, less error-prone, large user-base
• Real-time Java: Real-time support
• CORBA: Standards-based distributed
applications
• Real-time CORBA: CORBA with Real-time
QoS capabilities
ZEN project goals
• Make development of distributed, real-time, &
embedded (DRE) systems easier, faster, &
more portable
• Provide open-source Real-time CORBA ORB
written in Real-time Java to enhance
international middleware R&D efforts
Phase I – Applying Opt Strategies
• Foot-print Reduction Optimization
• Micro ORB Architecture  Virtual
Component Pattern
• Micro POA Architecture  Pluggable
components
• Request Demux/Dispatch
Optimizations
• Connection Management 
Acceptor-Connector pattern, Reactor
(java’s nio package)
• Buffer Management Strategies
• Request Demultiplexing  Active
Demultiplexing & Perfect Hashing
• POA Optimizations
• Object Key Processing Strategies 
Asynchronous completion token
pattern
• Servant lookup  Reverse lookup
map
• Concurrency Strategies  HalfSync/Half-Async
RT-CORBA
Thread Pool
Object Adapter
ORB CORE
Phase II – Applying RTSJ
Phase I  Optimization patterns and
principles
• ORB-Core Optimizations
•
•
•
Micro ORB Architecture  Virtual Component
Pattern
Connection Management  Acceptor-Connector
pattern, Reactor (java’s nio package)
Collocation and Buffer Management Strategies
• POA Optimizations
•
•
•
•
Request Demultiplexing  Active Demultiplexing &
Perfect Hashing
Object Key Processing Strategies  Asynchronous
completion token pattern
Servant lookup  Reverse lookup map
Concurrency Strategies  Half-Sync/Half-Async
Phase II  Enhance Predictability by applying
RTSJ features
• Associate Scoped Memory with Key ORB
Components
– I/O Layer : Acceptor-Connector, Transports
– ORB Layer: CDR Streams, Message Parsers
– POA Layer: Thread-Pools and Upcall Objects
• Using NoHeapRealtimeThreads
– Ultimately use NHRT Threads for
request/response processing
– Reduce priority inversions from Garbage
Collector
Phase III  Build a Real-Time CORBA
ORB that runs atop a mature RTSJ
Layer
Talk Outline
– Tech transitions in the DRE domain  Real-time
middleware & Real-time Java
− RT-Java + real-time middleware  ZEN project
−
−
−
−
−
Design & Architecture
Applying Real-time Java features
Empirical Results
Open challenges
Concluding remarks and References
Applying RTSJ features – Motivation
Original design of ZEN
• All components allocated in heap
• Request processing thread may be
preempted by GC (demand garbage
collection)
Goals
• Compliance with CORBA specification
• Interoperability with classic CORBA
• Reduce overhead for applications not using
real-time features
• End-user transparent
Design of ZEN for Real-Time CORBA
• Apply scoped memory along critical request
processing path
• Provide Policies at the POA level for RTSJ
aware users
• Allows NHRT threads used for request
processing
• Proper use of NHRT threads would minimize GC
execution during request processing
Analyzing Request Processing Steps
Server Side Connection Acceptance
4. An acceptor accepts the new incoming connection.
6. A new connection handler T1 is created to service
requests
7. The Transport's event loop waits for data events from
the client
Server Side Request Processing Steps
11. The request header on connection is read to
determine the size of the request.
12. A buffer of the corresponding size is obtained from the
buffer manager to hold request and read data.
13. The request is the demultiplexed to obtain the target
POA, servant, and skeleton servicing the request. The
upcall is dispatched to the servant after demarshaling
the request.
14. The reply is marshaled using the corresponding GIOP
message writer; Transport sends reply to the client.
Server ORB
13
Independent
Steps for two different
clients do not share context
Object Adapter
GIOP
Message
Parsers
1.0
1.1
11, 14
5
1.0
Connection
Cache
Buffer Manager
12
C1
C2
Acceptor
4
Transport T1
C3
6, 7,14
Repetitive & Ephemeral
Carried out for each
client request
Typically objects live for
one cycle
Thread Bound
Steps executed by
I/O threads ThreadPool threads
Application of Scoped Memory – ZEN
Ephemeral
• Two requests can be mapped to two
separate scope regions
• Temporary objects may be cleared after
request processing
O
UT
ER

ES
:I
NN
I/O
SPACE
EN
C
POA SPACE
ER
ORB SPACE
ES
OP
SC
ORB
SPACE
it
ex
2.Recursively enter each space from
I/O  POA scopes
3.Implicitly exit regions from POA 
I/O scope
ED
ST
NE
I/O scope  read request
ORB scope  process request
POA scope  perform upcall send
reply
I/O SPACE
RE
FE
R
1.Break Steps into three broad
regions based on request
processing steps
()
Applying Scoped Memory
• Encapsulate steps as “logic” class
associate this logic class with real-time
threads
• Threads enter the scoped region;
processes request; exits region enabling
objects to be finalized
te
r
Threadbound
en
Independent
Create all objects in a scoped region
POA
SPACE
Scope Memory in ZEN: Action
Associate a
start scope
with real-time
thread
POA Scope
1: (new RealtimeThread(default Scope))
I/O Scope
ORB Scope
I/O Scope is
now current
active
region
The three scopes are
created during ORB
initialization time
2: (I/O Scope).enter()
time
3: waitForData()
I/O thread
waits for
data
NETWORK
Following Slides are adapted from Angelo Corsaro
I/O Stage Processing
I/O Scope
• Participants The participants for this phase include,
acceptors, and transports
The message
is created
within the
scoped region
• RTSJ application
–Each of these components are thread-bound components
and are designed based on inner logic class
–
I/O Scope
Corresponds to the logic run by the thread
–Instead of creating the entire component in scoped memory,
we create the inner logic class in a scoped memory region,
mio
–This logic class is associated with the thread at creation time
3:Create new Message
new GIOPMessage ()
2:Read Data
time
NETWORK
1:Data
arrival
ORB Stage processing
ORB Scope
The thread
enters ORB
scope to parse
request
Nested inner
scope: all
refs from ORB
-> I/O are
valid
ORB Scope
Scope Stack • Participants Message parsers,
CDR Streams
I/O
Scope
• RTSJ application
ORB
Scope
I/O Scope
parseAndProcessRequest()
new GIOPMessage ()
time
NETWORK
–The appropriate message
parser associated to parse
request
–The message parser and
buffer created in a nested
memory region, morb.
–Using RTSJ memory rules,
references from the ORB to
the I/O space are valid
POA Stage processing
Enter POA scope
to process
request and
send response
Scope Stack
POA Scope
ORB Scope
Up-call
related
objects
created in
this scope
I/O
Scope
ORB
Scope
POA
Scope
I/O Scope
performUpCall()
parseAndProcessRequest()
new GIOPMessage ()
time
NETWORK
Steps
• Demux request to get target
POA, servant and skeleton
• Perform upcall on the servant
marshal reply back to client
RTSJ Application
• Message parser – parses the
request to find target servant
and skeleton
• Set up context for the upcall
• Upcall Object – holds info
necessary to perform upcall
• Output buffer – holds response
Talk Outline
– Tech transitions in the DRE domain  Real-time
middleware & Real-time Java
− RT-Java + real-time middleware  ZEN project
−
−
−
−
−
Design & Architecture
Applying Real-time Java features
Empirical Results
Open challenges
Concluding remarks and References
Predictability Enhancement
Overview
• POA Demultiplexing experiment
conducted to measure improvement
in predictability
Result Synopsis
• Average Measures:
• Scoped Memory does have
some overhead ~ 3 s
• Dispersion Measures:
• Considerable improvement in
predictability
• Dispersion improves by a ~
factor of 4
• Worst-Case Measures:
• Scoped memory bounds
worst case
• Heap shows marked
variability
Associating scoped memory
– Does not compromise performance
– Significantly enhances predictability
– Bounds worst case latency
Enter Exit Analysis
Overview
• Quantify overhead incurred by
using Scope Memory:
• enter () – entering scope region
• exit () – leave the scope region
Result Synopsis
•Average Measures:
• Constant enter () time across all
message sizes.
• exit () time increases with
message size
•Dispersion Measures:
• exit () methods incur
considerable variability when
compared to enter ()
•Worst-Case Measures:
• Similar behavior to both enter
and exit () time
On exit finalizers of objects run, hence larger
messages have higher average latency
– enter () time uses constant time O(1)
algorithm to validate illegal entry
Roundtrip Latency Analysis
Overview
• Influence of Scoped memory in the
Roundtrip latency measures
Result Synopsis
• Average Measures:
• For smaller clients, scoped
memory incurs greater overhead
• As requests increase, Scoped
memory outperforms heap
• Dispersion Measures:
• Considerable improvement in
predictability
• Dispersion improves as much as
50%
• Worst-Case Measures:
• Scoped memory bounds worst
case
• Though mean values are greater
99% and Worst case measures are
bounded
– As number of requests increase, GC activity
increases for Heap Memory.
– Scope memory kicks in to reduce GC activity
thereby improving processing time
Concluding Remarks & Future Work
Concluding Remarks
• We presented R&D efforts on integration of
RTSJ and RT-CORBA
• Our efforts focus towards effective use of
RTSJ and Real-time CORBA to improve
QoS for Java based real-time systems
Future Real-Time CORBA Research
• Policies at the POA level for RTSJ aware users
• Use NHRT threads for request/response
processing
• Threading Models for RTSJ
• Modeling RTSJ exceptions e.g.
ScopedCycleException
• Complete implementation of Real-time CORBA
specification
Downloading ZEN
• www.zen.uci.edu
References
• ZEN open-source download & web page:
• http://www.zen.uci.edu
• Real-time Java (JSR-1):
• http://java.sun.com/aboutJava/communityprocess/jsr/
jsr_001_real_time.html
• Dynamic scheduling RFP:
• http://www.omg.org/techprocess/meetings/schedule/
Dynamic_Scheduling_RFP.html
• Distributed Real-time Java (JSR-50):
• http://java.sun.com/aboutJava/communityprocess/jsr/
jsr_050_drt.html
• AspectJ web page:
• http://www.aspectJ.org
• JRate
• http://tao.doc.wustl.edu/~corsaro/jRate/