Transcript Servlet Mediator

Information Searching and Retrieval
from Distributed Databases
using Mediators, CORBA and XML
Presented by:
Supervised by:
Dec 18, 2000
Vincent Cheung
Prof Michael Lyu,
Prof K. W. Ng
Research objectives

Address the Use of XML in enhancing Data
Representation and System Communication
 Address the Use of Mediator Architecture for
Searching in Distributed Environment
 Address the Use of XML and HTTP to simulate
CORBA IIOP calls
 We have developed a CORBA-based mediator
system which can overcome the limitation of
firewall problem to achieve a worldwide system
Presentation outline

What is Mediator?
 Using XML
 Implementation with CORBA
 CORBA vs. Firewalls
 Overcome the Firewall
 Evaluation
 Future Work
 Conclusion
 Demonstration
Searching in distributed sources
 To
integrate different components in the
systems, there are three approaches:
 CORBA
 Mediators
 Agents
 They
are not orthogonal, e.g.:
 Implement
mediators by CORBA
 Agents may use mediators
What is mediator?

A middle layer for forwarding clients queries
to appropriate sources, and integrate the data
before returning to users
Clients UI
Client UI
Client UI
query
mediator
result
Database
Engine
Database
Engine
Database
Engine
Using XML

We need a COMMON format of information in
order to let distributed heterogeneous
components for data exchange.
 XML provides a good solution because:



XML is semi-structured and highly flexible in
data representation. Highly structured
traditional relational and object-oriented data
schema can be mapped to XML schema
without losing information.
A common XML schema for heterogeneous
system can be achieved
Hence, we use XML data schema in our
system.
A piece of XML data
<news>
<source>South China Morning Post</source>
<date>April 15, 2000</date>
<title>Press warning appropriate, says Beijing
</title>
<reporter>Kong Lai-fan</reporter>
<reporter>Greg Torode</reporter>
<content>Beijing yesterday defended remarks made
by senior SAR-based official Wang Fengchao that
local media should avoid reporting separatist
views. </content>
</news>
XML-QL

To query semi-structured data, traditional
query languages may not be adequate
 We use XMLQL, which is designed for semistructured XML data
 An example:
where <news> $B <news> in ‘’news_db.xml’’,
<date>
<year>2000</year>
<month> 4 </month>
<day> 15 </day>
</date> in $B
construct <result> $B </result>
Why use CORBA?
 Common
Object Request Broker
Architecture
 Designed for application development
within distributed heterogeneous
environment
 We use CORBA to build our mediator
system.
Architecture of our system
3rd tier
1st tier
2nd tier
Data Source
Web-base UI
Servlet
Interface
Web-base UI
Web-base UI
UI Queries and
Results
Mediator
(forwarding
queries and
integrating
results)
Queries
Data Source
…
Mediator
Results
Nth tier
Handling some special cases
 Some
special cases: Infinite loops,
broken connection and too many layer
of traversal.
 Infinite loops:
mediator
qid = 123
qid = 123
qid = 123
mediator
qid = 123
mediator
Solution: To give each query a unique ID, and
mediators will keep track all query IDs which have no
replied answers yet. Duplicated IDs would indicate
an infinite looping has occurred.
Special cases

Broken connection
Solution: Use a timeout parameter to specify
the maximum amount of time that we are
willing to wait.
 Too may layers of traversal
Solution: use a maximum layer parameter to
specify the maximum number of layers that
we want to go.
Timeout = 15000
Max_layer = 3
Timeout = 10000
Max_layer = 2
mediator
mediator
Timeout = 5000
Max_layer = 1
mediator
IDL design of our system

IDL (Interface Definition Language) defines
export interface of CORBA objects
 Our IDL design:

The parameter type for special cases handling
Struct SysPara
{
long qid;
long timeout;
short maxlayer;
}
IDL design of our system

Mediator may make queries to Databases or
Mediators
 Hence, we want Databases and Mediators
can be very similar objects
 Now, both of them are implementing
QueryEngine Interface
QueryEngine
Interface QueryEngine {
String query(in SysPara para,
Implemented by
in string xmlquery);
QueryMed
}
QueryDB
QueryDB Object
 Directly
connects to the data source
 Caller calls query()
 It takes the query statement parameter
and make query to related data source
 Returns answer in XML string stream
format
QueryMed Object

Same invoking method, query()
 Besides QueryEngine, it implements another
interface, QueryMediator
public interface QueryMediator {
public QueryEngine[] qelist();
public void qelist(QueryEngine[] arg);
public void append_result(String res);
}

qelist holds a list of QueryEngine objects, i.e.
QueryMed or QueryDB objects, which will be
called by that mediator.
 It starts a thread for each target QueryEngine
object, and the thread will call append_result()
to integrate results from various sources
Problems with CORBA firewall

We cannot achieve a worldwide query system
because of the firewall.
 CORBA uses Internet InterORB Protocol
(IIOP) for communication.
 Message body of IIOP is encoded in
Common Data Representation (CDR), which
translates IDL data types into a byte-ordering
independent octet string.
 Firewalls cannot decode the message body
of IIOP in application level.
 But we have application level gateways for
telnet, FTP, and HTTP
CORBA firewalls
 We
do have some firewalls which are
dedicated for CORBA communications
 E.g. IONA Orbix WonderWall and
Visibroker Gatekeeper
 They have limitations:
 Vendor
dependent: Orbix firewalls must use
Orbix developed client and server objects
 Not all features of CORBA can be used:
Callbacks may not be used
 Not commonly used
 Extra purchase, etc…
CORBA and firewalls

IIOP cannot pass firewall, but HTTP can
HTTP
Servlet
XML
CORBA
enclave
CORBA
enclave
firewall

IIOP
A real worldwide CORBA system can be
achieved
Solution to firewall problem

HTTPGateway is also implementing
QueryEngine.
QueryEngine
Implemented by
QueryDB

QueryMed
HTTPGateway
HTTPGateway is a virtual query engine to
forward the query to the target systems
Simulated Call
CORBA Enclave
Mediator M
firewall
CORBA Enclave
DB Object
HTTP Gateway H
Servlet
Mediator
SM
Sample XML message in
parameter passing
<request object="QueryMed" method="query" return="string">
<parameter order="1" name="para">}
<SysPara>
<qid>398498241824033984092</qid>
<maxlayer>4</maxlayer>
<timeout>2000</timeout>
</SysPara>
</Para>
<parameter order="2" name="QueryStatement">
<string>
<news> \$B </news> in "database.xml"
<keyword>satellite</keyword> in \$B
construct <result> \$B </result>
</string>
</Para>
</request>
Its DTD
<!DOCTYPE parapassing [
<!ELEMENT request (parameter*)>
<!ATTLIST request object (#CDATA)>
<!ATTLIST request method (#CDATA)>
<!ATTLIST request return (#CDATA)>
<!ELEMENT parameter (SysPara | string)>
<!ATTLIST parameter order (#CDATA)>
<!ELEMENT SysPara (qid,maxlayer,timeout)>
<!ELEMENT qid (#CDATA)>
<!ELEMENT maxlayer (#CDATA)>
<!ELEMENT timeout (#CDATA)>
<!ELEMENT string (#CDATA)>
]>
Evaluation
ADVANTAGES:
 It can overcome the CORBA IIOP vs. firewalls
problem by using HTTP calls,XML, and
servlet.
 Security issue can still be maintained.
External CORBA objects can call only the
objects combined with Servlet
 Both primitive data types, like String, or
complex classes, SysPara, can be well
represented by XML in parameter passing
Evaluation

It is not limited for the clients and servers
must be CORBA implementation.
 Internal CORBA objects would not notice the
difference of calling an external objects when
comparing to callings of internal objects
 With sharing the same XML and IDL
standards, we can easily achieve a worldwide
query system.
Evaluation
WEAKNESS:
 With using HTTP calls to simulate IIOP
is relative slower than using firewalls
dedicated for CORBA, as we need to do
many extra works to initialize the servlet,
to convert parameters to XML format,
invoke the servlets to work, etc.
Future Work
 Generalize
the IIOP simulation
 For
all kinds of objects, parameter types,
return types, etc
 Direct code generation from IDL
 Design
the mechanism that supports
CALLBACKS
 When callback can be simulated, we
can enhance the features of our
mediator system
Conclusion
 Using
Mediators in querying distributed
sources
 How to use CORBA and XML to
integrate the system
 Cooperation between XML and CORBA
in simulation of IIOP calls
Demonstration

Aim to show the mediator system can:


make queries to varies sources in parallel
go beyond firewalls
User Interface
(SHB 1027)
Servlet Mediator
H111, Kuomao Hall
Data Source 1
HTTP Gateway
H111, Kuomao Hall
H111, Kuomao Hall
FI
R
E
W
A
L
L
Servlet Mediator
(SHB 913, pc90003)
Data Source 2
(SHB 913, pc90003)
Q & A Session
Welcome to Give
Questions and Comments
<appreciation>
Thank You</appreciation>