Transcript Messaging Systems and Technology

Messaging Systems and
Technology
Introduction
• Synchronous middleware doesn’t suit all
applications
• Message Oriented Middleware (MOM)
provides features like:
• asynchronous communications between processes
• store-and-forward capabilities
• transactional messaging
Basic Asynchronous Operations
• Send (dest, message)
• Receive ( target, message)
queue
send
receive
MOM usage
• Applications often need:
• deferred processing of some ‘slow’ transaction (eg
printing an invoice)
• support efficient 1-to-many and many-to-many
communications
• to send messages whether the server is available or
not
• to provide event notification to a dynanic user
community
• loosely coupled client-server systems
Example - Transactions
//transaction 1
transaction {
UpdateDBMS(…..)
PutMessage (....)
} onCommit { .... }
onAbort {.....}
DBMS
//transaction 2
transaction {
GetMessage (....)
} onCommit { .... }
onAbort {.....}
Queue
MOM Technologies
• Broadly two (somewhat intersecting)
categories
• Message brokers (IBM, BEA, Microsoft, etc)
• Broadcast-based publish-subscribe (TIBCO, IONA)
• Both categories differ greatly in their
features and capabilities
Some Application Examples
•
•
•
•
Stock price update notification
General workflow systems
Application integration
System management
Some Example Products
• TIBCO/Rendezvous
• IONA’s OrbixTalk
• IBM’s MQ Series (Publish-Subscribe)
TIB/Rendezvous
• Pioneers of broadcast/multicast publishsubscribe technology
• Publishers and subscribers communicate
using subjects
Create/
Publish
Subject
Pub
Sub
Register/
Subscribe
Sub
Sub
Subjects
• Hierarchical names identify a subject of
interest
•
•
•
•
/CSIRO
/CSIRO/gossip
/CSIRO/gossip/ADSaT
/CSIRO/work/ADSaT
• Wildcards can be used
• /CSIRO/**
• /CSIRO/*/ADSaT
Multicast/Broadcast
• A published message is sent across the
network only once
• All subscribers receive the same message
• IP Multicast or broadcast is used on LANs
• Important scalability issue as
bandwidth/processor usage is low
LAN Architecture
Subscribe
Publisher
rvd
Broadcast
Network
rvd
Subscribe
Publisher
WAN Architecture
Subscribe
Local
Broadcast
Network
Publisher
rvd
Filtered,
Point-to-Point
rvrd
Local
Broadcast
Network
rvrd
Wide Area Network
rvd
Subscribe
Publisher
Quality-of-Service (QOS)
• Reliable
• publisher informed after time-out period if message
cannot be delivered to a subscriber
• Certified
• guaranteed delivery or both parties informed of
failure
• to survive process failure messages can be logged to
disk until specified time-out period expires
Programming Options
• C/C++/Java
• Self-describing message format
• TIB/ObjectBus layers a standard CORBAcompliant ORB upon the TIBCO protocols.
• TIBIOP provides broadcast messages in a
CORBA environment.
Security
• Security is a problem in publish-subscribe
multicast technology
• TIBCO Data Security (DS) product:
• publisher controls secure key distribution to
subscribers using digital signatures
• subscribers use key to encrypt messages on a subject
• choice of encryption algorithm available
OrbixTalk
• Uses IP multicast in CORBA environment
OrbixTalk
Application
CORBA
Event Service
OrbixTalk
Application
OrbixTalk API
OrbixTalk Multicast Transport Protocol
Topics
• Communication based on topics (read
subjects, wildcards, etc)
• 3 QOS levels:
– otmcp: UDP IP multicast. Limits packet size to 1280
bytes and no guarantee of receipt.
– otrmp: Augments IP multicast to ensure message
delivery to all subscribers until configurable time-out
expires
– otsfp: OrbixTalk MessageStore daemon uses a storeand-forward protocol to provide guaranteed message
delivery.
OrbixTalk Architecture
otmcp
Sender
Listener
otrmp
Sender
Multicast
Protocols
Listener
IP Address
Topic
OrbixTalk
Daemon
Sender
otsfp
Sender
Message
Store
Daemon
Topic
IP
Address
Listener
Listener
Architecture Issues
• Daemons can have hot backups to support
failure of primary daemon
• Several MessageStore daemons can be used
to facilitate load balancing:
• each OrbixTalk process only use one MessageStore
• Compaction utility must be scheduled to
remove old messages
Security/WANs
• No specific security or WAN support
• security can be programmed using Orbix
Transformers
• WANs traffic needs appropriately configured
routers, or an IIOP bridge between LANs to be
written
MQSeries PubSub
• MQSeries is probably most widely
deployed MOM product
• Basic technology provides a message queue
architecture:
• communication via shared queues managed by
Queue Managers
• persistent/non-persistent messages
• transactional queue access with XA
• broad platform support
MQSeries PubSub
1 PubSub Broker per QM
Publisher
2
Topic B
BROKER A
Topic B
Subscriber
1
LAN/WAN support
Topic A
Publisher
1
Topic B
BROKER B
Topic A
Topics interchanged by Brokers if needed
Subscriber
2
Architecture
• Brokers can be organized to communicate
hierarchically
HQ
USA
West
Asia
East
HK
Features
• Typical hierachical topic names, wildcards,
etc
• Streams can be used as higher level topic
partitioning scheme
• DEFAULT stream per broker
• optional additional streams for specified topics
• broker allocates a thread to handle each stream
Quality-of-Service
• Persistent
• messages written to log file
• survive broker/QM failure
• Non-persistent
• messages lost if broker fails
• fast
• Message priority supported in queues
Other features
• MQ can act as an XA transaction manager
• Basic security provided, integrated with
native operating system
• Security exits support integration with 3rd
party products
• Other products, eg MQ Integrator
Performance Test
• Publishers and subscribers run on different
machines
• Dual pentium 500Mhz NT 4.0 boxes,
0.5GB memory
• MQ QM/broker and Orbix daemons run on
same machine as publishers
• Reliable, non-presistent protocols used
256 Byte Message
Milliseconds
700
600
500
OrbixTalk
TIBCO
MQ
400
300
200
100
0
10
20
30
40
50
No. Of Subscribers
512 Byte Message
Milliseconds
1000
800
OrbixTalk
TIBCO
MQSeries
600
400
200
0
10
20
30
40
No. Of Subscribers
50
Interpretation
• Multicast technologies are fast!
• TIBCO rvd becomes bottleneck
• ‘Publisher too fast’ exceptions
• OrbixTalk generates large number of
interrupts
• one per message per listener
• MQ scales well, but queue/broker
architecture is inherently slower
Summary
• MOM provides excellent solution to
many business problems
• Range of technologies available, with
very different:
• architectures/technologies
• features
• performance
• Important to understand issues and select
right one for your business