Transcript Slides (powerpoint)

CSE 590DB: Database Seminar
Autumn 2002: Meta Data
Management
Phil Bernstein
Microsoft Research
Course Administration

Course credit



I’m giving today’s lecture


So I get 2 credits
Next week, we have a visitor


1 credit for reading the papers and showing up
2 credits for leading a discussion
Wang-Chiew Tan
Send me mail to sign up for later weeks
What’s Meta Data?

DB defns, form defns, documents, interface defns,
source code, help text, executables, icons, makefiles, ...
Spec
Forms
ER Diagram
Customer
Order
Product
Scheduled
Delivery
Salesperson
Business Rules
Emp.Sal <
Emp.Mgr.Sal
Business
Process
Architecture
Update
Marketing
Authorize
Credit
Order
Entry
Bill
Customer
Table Defns
Schedule
Delivery
Inventory
View Defns
Interface Defns
The Schemas are the Data

Plus relationships between the schemas


This isn’t like data-processing data that you
store in relational databases



Depends-on, generated-from, authored-by, …
It’s more like the content of a SQL catalog
It’s inherently heterogeneous & distributed
SQL doesn’t help a lot to manipulate it
What’s Meta Data Management?


Generic mechanisms to help store,
search, and manipulate meta data.
Main components


A database engine
Information Models (i.e. meta models)


Schemas whose instances are meta data (i.e. models)
Tools
Generic – for any meta data application
 For Vertical Applications – usually design-time,
but can be run-time

Meta Data Manager Architecture
Customer
Update
Marketing
cust
emp
dept
dno
dna
Order
Authorize
Credit
Product
Scheduled
Delivery
Order
Entry
Bill
Customer
Salesperson
Schedule
Delivery
select all
Information Model
• Predefined types
Inventory
Forms
ERD
Spec
Customer
Order
Product
Scheduled
Delivery
Salesperson
Architecture
Tables
Views
Code
Repository Mgr
or OO DBMS
• Objects, properties
• Rich relationships
• Extensibility
• Versioning
• Configurations
Database System
Database
Generic Tools
• Browser
• Scripting language
• Data translators
(import/export)
• Model editor
• Model merge
• Component mgr
Typical Usage Pattern
Meta Data Source
Database catalog, language introspection,
modeling tool, prog. environment, …
Scanner (importer)
Object-Oriented
Structure
Meta Data
Application
Database design, impact analysis,
data translation, data integration,
view integration, message
mapping, data whse loading,
data migration, ….
Generator (exporter)
Code
SQL DDL, interface defns, XSLT, Java, …
Meta Terminology
meta-meta-meta data Definition of “object”
= meta-meta-model
meta-meta data
= meta-model
Definition of “Table”
meta data
= model
Definition of (schema for)
the Employee Table
data
Employee Table
Vertical Applications I

Database design



Map ER model to SQL schema
Reverse engineer SQL schema to ER model
DB Application development



Map SQL schema to default form
Map business rule to SQL constraints and
form validation code
Manage dependencies between code and
schemas and forms
Vertical Applications II
Data translation



Import source and target schemas
Build a mapping between them
Data translator interprets the mapping
Schema integration


Merge data sources into a global schema
View integration



Define use-case scenario
Identify views for each use-case
Integrate views into a conceptual schema
Vertical Applications III

Message Mapping


Map messages from one format to another
Data Warehousing





Import schemas of data sources
Identify overlapping attributes, etc.
Build data cleaning scripts
Build data transformation scripts
Enable data lineage tracing
Vertical Applications IV

Data Migration




Import a schema and its modified version
Build a mapping between them
Data migration tool inteprets the mapping
Scientific data management



Merge schemas from related experiments
Manage transformations of experimental data
Track evolution of schemas and transformations
Vertical Applications V

Information Resource Management




Inventory control of schemas and apps for
large enterprises
Import & browse schemas and interfaces
Impact analysis
Semantic query processing


Pose queries against conceptual model
Automatically map the query to DB schemas
Vertical Applications VI

Integrated CASE







Model the application in UML
Translate the UML to interfaces, schemas, etc.
Tools for consistency checking, test generation,
impact analysis, ….
Integrating customized applications
Workflow design and management
Document management
Application configuration management
Course Topics

Meta data applications



Schema management



Data translation [Popa et al]
Data integration [Spaccapietra & Parent]
Meta meta models [Noy et al]
Properties of mappings [Hull]
Meta data mechanisms




Lineage tracing [Buneman et al.] (Wang-Chiew Tan)
Mapping generation [Atzeni & Torlone]
Schema merging [Buneman, Davidson, Kosky]
Model Management [Bernstein]
Model Management


Model – a complex information structure, e.g., XML
schema, SQL schema, .NET type def, UML model.
Mapping – a transformation from one model into another





Map between two XML schemas
Map SQL schema to XML schema
Map data source to data warehouse
Map classes to data source defns
Model Management Algebra



Match (M1, M2, map)
Merge (M1, M2, map, M3)
Compose (map1, map2, map3)



Copy, Delete, Update
Diff, Apply, ModelGen
Select, Enumerate
Implementation Vision
Generic Tools
Customer
Update
Marketing
Order
Authorize
Credit
Product
Order
Entry
Model-Driven
UI Generator
cust
emp
dept
dno
dna
Scheduled
Delivery
Bill
Customer
Salesperson
Schedule
Delivery
Inventory
select all
• Browser
• Editors
• Import/export • Catalogs
• Scripting
Model Match Compose Operation
SpecialiCopy
Manager Merge
zations
Apply …
Object-Oriented
Repository
Inferencing
Engine


SQL
DBMS

Web pointers

//www.research.microsoft.com/~philbe

//research.microsoft.com/db/ModelMgt/