Transcript Heterogenous XML-based Data Integration

Julie Basu
Senior Development Manager
Nirav Chanchani
Senior Product Manager
Oracle Corporation
Session id: 40258
Heterogeneous XML-based
Data Integration
Using XQuery, J2EETM, and Database
Technologies
Presentation Agenda







Business Problem
XML-based Data Integration
Querying with XQuery
Building Applications using the J2EETM Platform
Leveraging the XML Database
Demonstration
Next Steps
Business Problem
Synthesized
Information
 Multiple domain-specific
applications
– Manufacturing,
Inventory, Supply
Chain, Financial, …
 Information is trapped within
these applications
CRM Application
Financial
Application
Shipping and
Distribution Application
Order Management
Application
Contract Management
Application
How does an organization get a consolidated view of its
information – in real time ?
Technical Challenges
 Domain-specific information
–
Replication does not make sense
 Independent operation of applications
–
Access to information has to be in real-time
 Different access method for each application
–
Each application has its own protocol and access method
What architecture can best accommodate
my present and future needs ?
–
–
–
Complexity: Avoid the n by m matrix
Flexibility: Add new sources easily
Time to market: Within days, not months
Presentation Agenda







Business Problem
XML-based Data Integration
Querying with XQuery
Building Applications using the J2EETM Platform
Leveraging the XML Database
Demonstration
Next Steps
What is XML-based Data
Integration?
XML-based Data Integration or
Enterprise Information Integration (EII):
 Create aggregated views using XQuery
 Get information from diverse sources in XML
 Consume synthesized information
XML Data Integration Example
XQuery Engine
Synthesized
Information
Order Tracking
J2EETM CA
EIS
JDBC
Parts Inventory
Database
HTTP
Shipment Tracking
Web Service
Technologies Involved
 Why XML?
–
Different data formats
 Why XQuery ?
–
Declarative way to query
XML documents
 Why J2EETM ?
–
Standards-based
infrastructure platform
 Why XML Database ?
–
–
–
Native XML storage
XML data management
Performance optimizations
XQuery Engine
XML
Database
J2EETM Platform
XML-based Data Integration
Comparison with Existing
Technologies
Difference
Application
Integration
Data
Warehousing
Traditional
Report
Generation
Similarity
More about data pumping
and synchronization
among systems
Involves data
adapters, data
translation and
transformation
Explicit ETL steps
required; large data
volume; batch loading, not
real-time
Optional cache
pre-population step
similar to the
“loading” step
SQL & relation-based vs.
XQuery & XML-based
Query-based
Focus is on report
formatting and delivery
Presentation Agenda







Business Problem
XML-based Data Integration
Querying with XQuery
Building Applications using the J2EETM Platform
Leveraging the XML Database
Demonstration
Next Steps
What is XQuery?
 Query XML Documents
 FLWOR expression:
–
–
–
–
–
For: similar to SQL “from” clause
Let: declaring a variable
Where: similar to SQL “where” clause
Order by: similar to SQL for sorting
Return: similar to SQL “select” clause
 Operations
–
–
–
Filtering
Transformation
Joining
XQuery Sample Code
<descriptive-catalog>
{
for $i in document("catalog.xml")//item,
$p in document("parts.xml")//part[partno =
$i/partno],
$s in document("suppliers.xml")//supplier[
suppno = $i/suppno]
order by $p/description, $s/suppname
where $i/price < 1000
return
<item>
{
$p/description,
$s/suppname,
$i/price
}
</item>
}
</descriptive-catalog>
Advantages of XQuery
 Declarative view definition
–
Less programming
 Query optimization
–
Just like SQL, XQuery engine optimizes queries
 Lower cost of code maintenance
–
Less code when compared to DOM or SAX
What’s New in XQuery?
 Different from XSLT, not re-inventing wheel
–
–
Document ordering
Rules matching vs. nested SQL-like queries
 XQuery 1.0 goals
–
–
Read-only, no transaction semantics, no security
Update capability not there yet
 XQuery runtime
–
Could be in middle tier, database, or a hybrid
Presentation Agenda







Business Problem
XML-based Data Integration
Querying with XQuery
Building Applications using the J2EETM Platform
Leveraging the XML Database
Demonstration
Next Steps
Oracle Application Server 10g The J2EETM Platform
Building applications using J2EETM services in
Oracle Application Server 10g




Data Source Access
Client Access
Caching
Security
J2EETM Data Sources
 Different types of data sources with
varying access protocols and APIs
–
–
–
Data sources : Applications, RDBMS,
Documents, Message Queues, Proprietary
Access mechanisms: JDBC, J2EETM CA,
Web Services, JMS, File I/O
Translation mechanisms: Oracle’s XML-SQL
utility for JDBC result set, other
J2EETM Client Access
Programmatic client access
 XQuery Java API is being standardized in
Java Community Process
–
JSR 225: Oracle and IBM leads
 Higher-level APIs: EJB, Web Services,
J2EETM CA, JSP tag library
XML Data Caching
Cache data from sources and results
 Choices: In memory, file, or database
 Benefits: Better Performance
–
–
–
Decrease data fetching cost
Reduce latency of unreliable or remote sources
Eliminate re-execution of queries
 Costs: Need to keep cached data fresh
–
–
Set expiration time for cached info
Invalidate or refresh upon update, e.g., DB triggers
Data Source & Cache Security
 Delegate security checking to sources
–
–
For SQL / JDBC data source use JAAS
For J2EETM CA leverage the Principal Mapping
and Custom Login module
 Security of cached data - poses additional risk!
–
–
Require login / authentication before cache access
Use JAAS to enforce cache access policies
Presentation Agenda







Business Problem
XML-based Data Integration
Querying with XQuery
Building Applications using the J2EETM platform
Leveraging the XML Database
Demonstration
Next Steps
Oracle XML Database (XML DB)
 Native XML storage
–
Available since Oracle Database Release 9.2
 Inherits RDBMS features: Security, Transaction,…
 XML-specific features
–
XML indexing, XPath & XSLT support,
XML schema validation, XML partial update
 Supports SQL/XML
–
Allows blending relational and XML data operations
Leveraging Oracle XML DB
 XML DB can be an XQuery source
–
–
Can define XML views of relational data
XQuery engine can rewrite query into SQL/XML
 XML DB could also be used for caching
–
Efficient storage & indexing for large data sets
 Can leverage security framework of XML DB
–
For both source and cache
What about Standards?
 Most components based on existing standards:
XML-based, J2EETM-based
 Oracle is helping to move standards forward:
–
Query APIs
 XQuery Java API: JSR 225 Oracle and IBM leads
 XML Extension to JDBC: JSR 114
–
XML metadata API to enhance usability
 Candidate technology: CCI in J2EETM CA
D E M O N S T R A T I O N
Summary
 What?
–
–
–
–
Get information from diverse sources in XML
Join, filter and transform data by XQuery
Apply Caching and Security appropriately
Consume synthesized information
 Why?
–
–
Real-time information access and synthesis
Productivity: Quick development cycle,
low code maintenance cost
Next Steps ….
 See the solution in action at Oracle
DEMOGrounds!
–
At the Oracle Application Sever 10g booth - Simplify
Information Integration
 Relevant web sites
–
OTN: http://otn.oracle.com
 Contact information
–
Product Manager: [email protected]
QUESTIONS
ANSWERS
Reminder –
Please complete the
OracleWorld online session
survey
Thank you.