Transcript TIPHON SPAN NGN Workshop

Ch 4 : Multimedia Database
Science and Technology Faculty
Informatics
Arini, ST, MT
arinizul@gmail. Com
[email protected]
I. Multimedia Data
• Multimedia data :
– From the presentation view point, multimedia data is huge
& involves time dependent /independent characteristics.
• Presentation & subsequent interactions
– Temporal : video, audio, animation
– Spatial : image
– Complex structure demands deriving semantics from
contents
» Need for Content-Based Access : color, texture, shape
– Collaborative Support Environment
– Consisting of
• Alphanumeric : text, hypertext
• Image : bitmap graphic (2D) and vector graphic (3D)
• Video : analog (videocassette, laserdisc) and digital (Quick time,
real player, win player )
• Animation : 2D, 3D (virtual reality : modeling object)
• Audio : digital audio
II. Multimedia Data Management
• Multimedia data management system is
to allow efficient storage, manipulation
using of multimedia data in all its varied
forms.
– Storage, retrieval, integration and
presentation requirements of multimedia
data differ significantly from those for the
traditional data.
• Implemented to : Multimedia Database
– Content Multimedia : MM-DBMS
III. Multimedia Database Management System
(MM-DBMS)
• A framework that manages different types of data
potentially represented in a wide diversity of
formats on a wide array of media sources
– To provide a suitable environment for using and
managing multimedia database information
• MM-DBMS must include :
– The traditional DBMS functions (e.g., database definition
and creation, data retrieval, data access and organization,
data independence, privacy, integration, Integrity control,
version control and concurrency support but applied to
various multimedia data types.
– The functional requirements imposed on a MMDBMS can
be grouped into two categories :
• Data representation requirements
• Data manipulation requirements.
4.1. MM-DBMS Functional Requirements
• Data representation requirements
– Support for Generalization/Specialization Hierarchy (Book;
Newspaper)
– Attribute Specification (Title, author, number)
– Ordering of Documents
– The presentation of the paragraphs, images, and drawings of a
multimedia document could depend on the users accessing the
document.
– Data independence
• Separate the database and the management from the
application programs
• Data manipulation requirements.
– Integration
• Data items do not need to be duplicated for different
programs
– Concurrency control
• allows concurrent transactions
5
– Data manipulation requirements.
– Persistence
• Data objects can be saved and re-used by different
transactions and program invocations
– Privacy
• Access and authorization control
– Integrity control
• Ensures database consistency between transactions
– Recovery
• Failures of transactions should not affect the persistent
data storage
– Query support
• Allows easy querying of multimedia data
– Flexible Acquisition
– Efficient Storage
– Efficient Retrieval
MM Database
6
4.2. Data Structure
Multimedia data
Raw
Data
Registering
Data
(Uncompressed
Image)
Media
Raw
(Size & coding details of
raw data)
Descriptive
Data
Registering
(Textual numerical
annotations)
Descriptive
Text
Characters
Coding scheme (ASCII), length
/ end symbol
Key words, information for
structuring
Images
Pixels
Height/ Width of picture, Mode Pic.Date = 21/04/07
of Compression, if JPEG, tables Pic.Reason = Birthday
for quantization purpose
Etc
Motion
Video
Sequence
Pixels
Frames/second, coding details,
frame types…
Scene description
Audio
Sample
sequence
Audio coding (PCM,…) ,
resolution of samples
Content of audio passages in
short form
4.3. Data Model
• Traditional :
– Relational Data Model
• Store everything as First Normal Form tables
• Access, Oracle, MySQL
• Semantic :
– Object Oriented Data Model
• Store everything as classes of objects
• ODL, OQL
– Object-relational Data Model
• Fundamentally relations but are not First
Normal Form table form
• POSTGRES
4.4. MM Database Architectures
Based on Principle of Autonomy
•
Each media type is organized in a media-specific manner suitable for
that media type
•
Need to compute joins across different data structures
•
Relatively fast query
processing due to
specialized structures
•
The only choice for
legacy data banks
MM Database
9
MM Database Architectures (cont.)
Based on Principle of Uniformity
•
A single abstract structure to index all media types
•
Abstract out the common part of different media types
(difficult!) - metadata
•
One structure - easy implementation
•
Annotations for different media types
MM Database
10
MM Database Architectures (cont.)
Based on Principle of Hybrid Organization
•
A hybrid of the first two. Certain media types use their own
indexes, while others use the "unified" index
•
An attempt to capture the advantages of the first two
•
Joins across multiple data sources using their native indexes
MM Database
11
Organizing Multimedia Data Based on the
Principle of Uniformity
• Consider the following statements about media
data and they may be made by a human or may
be produced by the output of an image/video/text
content retrieval engine.
– The image photol.gif shows Jane Shady, “Big Spender”
and an unidentified third person, in Sheung Shui. The
picture was taken on January 5, 1997.
– The video-clip videol.mpg shows Jane Shady giving
“Big Spender” a briefcase (in frames 50-100). The video
was obtained from surveillance set up at Big Spender’s
house in Kowloon Tong, in October, 1996.
– The document bigspender.txt contains background
information on Big Spender, a police’s file.
MM Database
12
Querying SMDSs (Uniform
Representation)
Querying SMDS based on top of SQL. Basic
functions include:
• FindType(Obj): This function takes a media object Obj
as input, and returns the output type of the object. For
example,
FindType(iml.gif) = gif.
FindType(moviel.mpg) = mpg.
• FindObjWithFeature(f): This function takes a feature f
as input and returns as output, the set of all media objects
that contain that feature. For example,
FindObjWithFeature(john)=
{iml.gif,im2.gif,im3.gif,videol.mpg:[1,5]}.
FindObjWithFeature(mary)=
{videol.mpg:[1,5],videol.mpg:[15,50]}.
MM Database
13
4.5. Data Retrieval : 4 Types
1. Conventional database system
– This is the widely-used approach to manage and
search for structured data.
– All data in a database system must conform to
some predefined structures and constraints
(i.e., schema’s).
– To formulate a database query the user must
specify which data objects are to be retrieved,
the database tables from which they are to be
extracted and predicate on which the retrieval is
based.
– A query language for the database will generally
be of the artificial kind, one with restricted
syntax and vocabulary, such as SQL.
4.5. Data Retrieval : 4 Types
2. Information retrieval (IR) system :
– IR system is mainly used to search large text
collections, in which the content of the (text)
data is described by an indexer using keywords
or a textual abstract, and keywords or natural
language is used to express query demands.
• Use Keyword Querying
– For example for an image or video we have to
describe it in words or in a way need to store lot
of metadata (textual form).
4.5. Data Retrieval : 4 Types
3. Content based retrieval (CBR) system
– This approach is used to retrieve desired multimedia
objects from a large collection on the basis of features
(such as color, texture and shape, etc.) that can be
automatically extracted from the objects themselves.
• Use Visual querying and semantic querying
– Although keyword can be treated as a ”feature” for text
data, traditional information retrieval has much more higher
performance than content-based retrieval because keyword
has the proven ability to represent semantics, while no
features have shown convincing semantic describing
ability.
– But major drawback of this method is that it lacks
precision.
4.5. Data Retrieval : 4 Types
4. Graph or tree pattern matching :
– This approach aims to retrieve object subgraphs from an object graph according to
some denoted patterns.
V. 2 Search Types
• Metric
• Combinatorial : image, audio (music), video
VI. MM-DBMS Structure
• Data Analysis
– Data structure (4.2 above)
• Data Modeling
– Data Model (4.3 above)
• Data Storage
– DBMS Architecture (4.4 above)
• Data Retrieval :
– 4 Types data retrieval (depend on query and
searching method) (V. above)
• Multimedia Communication
– Multimedia Networking