Transcript kay

CORBA Controls Workshop
MICO
Architecture & Applications
Kay Römer
ETH Zürich
Switzerland
Talk Overview
 Past, present, and future
– Development and use
– Book projects, Workshops
– CORBA Branding
 Architecture
Overview
– Micro-kernel based approach
 Application
– Zero-Copy CORBA for high bandwidth
applications in clusters of PCs
2
MICO Past, Present, Future
MICO Overview
 MICO = Mini CORBA (inspired by Minix)
 MICO = MICO Is CORBA (inspired by GNU)
 Open Source CORBA implementation
–
–
–
–
–
–
1996:
1997:
1998:
1999:
2000:
2001:
kick off as research/teaching project
first public version
binary distribution
branded by OpenGroup
text book
2nd MICO Workshop
 Currently working towards CORBA 3 (e.g.,
component model), security (MICOSec)
3
MICO Devel & Use
MICO Past, Present, Future
 Distributed development
– CVS repository, write access by 5 maintainers
– Mailing lists with ~1500 subscribers
– Major code contributions from 20 programmers
world wide
 Industrial applications
– E.g., The Weather Channel, Lufthansa Systems
 Teaching
– E.g., Lectures and Courses at ETH Zurich, U
Frankfurt, TU Darmstadt, U Stuttgart, San
Francisco State University
 Research
– E.g., Zero-Copy CORBA, QoS support for CORBA
4
MICO Past, Present, Future
MICO Books
 Foundation for lectures and CORBA
courses
 1998: binary distribution @ MKP
– Binaries for Windows and several Unix
platforms, tutorial, documentation
 2000: text book in German @ dpunkt
– Dual (user and system) view on CORBA
middleware
 2003: merger of text book and
distribution @ MKP (planned)
5
MICO Workshops
MICO Past, Present, Future
 Forum for MICO users and developers
– 1 day, invited talks
– 50-100 participants
 1998: 1st Intl. Workshop @ Darmstadt
 2001: 2nd Intl. Workshop @ Stanford
 2003: 3rd Intl. Workshop @ San
Francisco (planned)
6
 1999: MICO branded as CORBA 2.1
compliant by OpenGroup
– Test suite developed by ApTest (Ireland)
– Rather buggy, MICO as a test case
Appl
interoperability
MICO Past, Present, Future
CORBA Branding
Appl
portability
ORB
ORB
7
CORBA Branding cont.
 Tests
–
–
–
–
–
Local function syntax (interface)
Local function semantics (implementations)
IDL Compiler (c++ language mapping)
Interface Repository
IIOP
 Open issues
– Test services?
– Coverage?
– Any better than simple empirical test?
 Ongoing (?) work at OMG and GMD Fokus to
improve branding and develop Open Source
test suite
8
Talk Overview
 Past, present, and future
– Development and use
– Book projects, Workshops
– CORBA Branding
 Architecture
Overview
– Micro-kernel based approach
 Application
– Zero-Copy CORBA for high bandwidth
applications in clusters of PCs
9
Architecture
 Initial goal: provide middleware platform for
middleware research and teaching
MICO Architecture
– Similar to Minix (Mini Unix) in the operating
system domain
 Requirement: easy to understand, modify,
and extend
–
–
–
–
Clear, modular structure
Simple, well understood interfaces
Keep it simple
Do not rely on complex tools
10
Architecture cont.
MICO Architecture
 Turns out that requirements help keep
up with evolution of CORBA
 New CORBA application domains
– Embedded systems
– Nomadic systems
– ...
 Evolving CORBA standard
– New object adapters (e.g., POA)
– New interoperability protocols (e.g.,
SOAP)
11
Micro Kernel Approach
Micro Kernel Approach
 Minimize ORB
functionality
 Move functionality to
– Servers (separate
process)
– Runtime libraries
(same process)
 Extensibility: adding, removing,
exchanging servers and libraries
without touching ORB
12
ORB Functionality
 Local request processing
ORB Functionality
– Select appropriate object adapter
– Keep track of pending invocations
 Task scheduling
– Coordinate concurrent ORB tasks
13
Request Processing
ORB Request Processing
1. Invocation adapter interface
2. Active invocation table
3. Object adapter registry
4. Object adapter interface
14
Request Processing
Colocated Client & Server
 DII: initiates method invocation
 POA: executes method on object
implementation
15
Request Processing
Separated Client & Server
 IIOP Client: transforms method
invocation into network message
 IIOP Server: transforms message into
method invocation
16
Request Processing
Server Activation
 Mediator: looks up or runs appropriate
server and forwards method invocation
17
ORB Task Scheduling
Task Scheduling
 Tasks:
– Application can be caller and callee at the
same time (distributed callbacks, nested
method invocations)
– Process incoming and outgoing messages
on multiple communication channels
 Requirements:
– Multithreading optional (e.g., Palm OS)
18
Event Based Scheduler
 Split task into nonblocking segments
Task Scheduling
– Blocking operation at start of each segment
– E.g., wait for data from network, timeout,
or condition („events“)
 Register segments
with scheduler
– Executes segment
when blocking
operation completes
 Non-preemptive
19
Evaluation
 CORBA suggests a modular design:
Evaluation
–
–
–
–
Invocation interfaces
Skeleton interfaces
ORB core
Object adapters
 Most implementations melt this
modular design into a monolithic
implementation
– Changes in CORBA spec often require
major ORB changes
20
Evaluation cont.
 Often easy to adopt MICO to changes in
the CORBA specification
– Due to micro-kernel approach
 Advent of the POA
Evaluation
– BOA untouched
– BOA and POA coexist, even in one
application
 New interoperability protocols (SOAP)
– Easy to introduce new interop protocols,
even non-GIOP-based ones
21
Talk Overview
 Past, present, and future
– Development and use
– Book projects, Workshops
– CORBA Branding
 Architecture
Overview
– Micro-kernel based approach
 Application
– Zero-Copy CORBA for high bandwidth
applications in clusters of PCs
22
Zero-Copy CORBA
 Clusters of Personal Computers (CoPs)
equipped with Gigabit Ethernet
Zero-Copy CORBA
– Best performance, lowest price
 Many multimedia apps require fast bulk data
transfer
 Poor memory subsystem limits overall
communication bandwidth
– Data copies slow things down
 Aim: improve end-to-end performance of bulk
data transfers
 Christian Kurmann´s PhD project at ETH Zurich
23
Zero-Copy CORBA
Performance Limits
 Pentium II 400MHz, Intel 440BX chipset,
Gigabit Ethernet
 Network: 125 Mbyte/s
 33 MHz PCI: 126 Mbyte/s
 Memory copy: 92 Mbyte/s
– Memory bandwidth < network bandwidth
 Linux-2.2 measurement: 42 Mbyte/s
 Therefore: eliminate data copies at all
layers
24
ORB
Data Copies
foo(arg1, arg2)
foo(arg1, arg2)
H
TCP/IP
H GIOP
GIOP
H
H GIOP
H
H
DMA
DMA
25
NIC
Zero-Copy CORBA
GIOP
NIC Driver / Network
Zero-Copy CORBA
 Problem: limited Ethernet packet size (MTU)
of 1500 bytes
– Sender fragments data stream into sequence of
small packets
– Receiver has to reassemble stream
– Requires data copies since MTU
< memory page size (4k)
4k (page)
 Solution
– Driver pretends MTU of 4k
– (De)fragments page (3 packets) 1460
1480
using fast DMA to/from NIC
– Uses speculative techniques to defragment
without copies; makes the common case fast
26
1156
TCP/IP
 NIC driver handles whole memory pages
– Page remapping instead of copying
Zero-Copy CORBA
 TCP/IP headers are kept separate from
payload
– Scatter/gather lists (DMA descriptors)
4k
Header
Frag 1
H
H
27
Frag 2
Frag 3
CORBA ORB
 In heterogeneous CoPs no need to
marshal data structures
Zero-Copy CORBA
– Transmit memory image of large data
structures with zero copies
 Separate control from data transfer
– References instead of embedded data in
GIOP messages
GIOP
Data 1
Data 2
28
MPEG Transcoder
Zero-Copy CORBA
 Maximum bulk transfer rate: 75
Mbyte/s (instead of 42 Mbyte/s)
 Example app: parallel transcoding of
MPEG streams
Data
source
Distributor
Parallel
Encoder
29
Integrator
Summary
 Started off as research/teaching project
– Book projects, Workshops
 Developed into full-fledged CORBA
implementation
Summary
– With help from the Open Source community
– Goal: keep pace with CORBA standard
 Micro-kernel based architecture
 Application: high bandwidth applications
in clusters of PCs
 www.mico.org
30