Transcript Multimedia Database

Multimedia Database
Management System
Wei Tsang Ooi
CS731
MMDBMS :
Querying Interface,
Indexing
and
Buffer Management
Why MMDBMS ?
Concurrency Control
Recovery
Privacy
Query Support
Version Control
Example of MMDBMS
Digital Library
News-On-Demand
Video-On-Demand
Music Database
Telemedicine
Geographic Information System
No Intergration
id
size
fps
title
filename
000001 530M
30
soam
l6.mpg
000002 450M
30
tibor
l7.mpg
000003 600M
30
parag
l5.mpg
000004 510M
30
wei
l4.mpg
Semi-intergrated
BLOB
000001b70ae9902...
Fully Intergrated





index



buffers
storage
Nature of Multimedia Data
Large amount of data
Time sensitive
Vague matching
Database Components

Query
Interface
storage
manager
buffer
manager
Query
Processing
index
Query Interface
and
Processing
Problems
Needs support for :
•
•
•
temporal and spatial relations
“natural” interface
fuzzy query
SQL is not suitable.
13 Temporal Relations
Allen 83
before
starts
meets
equals
overlaps
during
finished by
finishes
contains
started by
overlapped by
met by
after
Spatio Relations
Papadias, Theodoridis 96
•
•
Topological Relations
Directional Relations
Topological Relations
disjoint
inside
meet
equal
overlap
covers
covered by
contains
Directional Relations
North
NW
NE
West
East
SW
SE
South
Spatio-temporal Relations
overlap-after
meet-during
...
Querying
Image
Audio
•
•
•
Music
Sound
Speech
Video
Querying Image
Common approach
•
•
•
allow query by sketches (color, shape,
texture) or examples.
perform matching by Feature Vectors
F = (v1, v2, ... vn)
e.g. Color Histogram
Querying Image
Exisiting Systems :
•
•
•
•
•
QBIC
VisualSEEK
PhotoBook
Virage
FourEyes
Querying Music
Hawley 93
•
•
•
Input by MIDI Keyboard
Measure relative pitch (U, D, S)
Perform exact match with existing
database.
Querying Music
Ghias, Logan, Chamberlin & Smith
95
•
•
•
Input by humming
Extract relative pitch
Perform approximate matching
Querying Music
Chou, Chen, & Liu 96
•
•
Query by chord
Represents musics by chord
• C Am Em F C Am Em F ...
•
Perform fuzzy match
Querying Music
Chen & Chen 98
•
•
Query by “rhythm” (tempo ?)
Represents musics by rhythm
• | ¶¶— | ¶·¶·| ¶¶¶¶ | ...
•
Perform fuzzy match
Querying Sound
Wold, Blum, Keisar & Wheaton 96
•
Analyze audio to extract features
• loudness, pitch, brightness, bandwidth
and harmonicity
•
•
Segment the audio to pieces
Feature Vector Matchings
Querying Speech
Hauptmann & Witbrock 97
Informedia
•
•
Use speech recorgnition to convert
audio to text
segment audio using silence
detection
Query by speaking keywords
Querying Video
Query by speech
•
Informedia
Visual approach
•
VideoQ
VideoQ
Chang et. al 97
•
•
User can sketch objects
Specifies
•
•
•
•
•
•
color
texture
shape
motion
duration
camera zoom and pan.
VisualQ Example
Titanic Sinking
Someone Skiing
Indexing
Indexing
Requirements
•
•
support spatio-temporal operations
support fuzzy matches
Indexing Images
N-dimentional indices for feature
vector
Well studied in DB/CG community
Two examples :
•
•
VP-tree
R-tree
VP-tree
Q
R
P
R
T
S
U
W
V
PQS
VWTU
VP-tree
Q
R
P
R
T
S
U
W
V
P
S
U
Q
T
VW
R-tree
Indexing Audio
Audio are modeled as strings
Inexact match is needed
Common indices for string search
can be used
Example
•
PAT-tree
PAT-tree
ab
abc
ababc
c
abc
b
c
abc
c
babc
bc
c
Indexing Video
Treat time as third dimension
We can use any multidimension
indexing structures
Buffer Management
Buffer Management
Minimizing response time
Ensuring continuity and
synchronization
Prefetching & Replacement
Glossaries
Presentation
Media Stream
Media Object
Buffer Management
Relevance Based
Distance Based
Buffer Consumption
Relevance Based
Moser, Kraiß & Klas 95 : L/MRP
Definition:
•
•
State = (curr obj, skip)
Relevance : State  Real
Relevance Based
Prefetch :
•
future objects with highest relevance
Replace :
•
old objects with lowest relevance
Example (5,+2)
Relevance
History
Referenced
Skip
Time Scale
Distance Based
Özden, Rastogi & Silberschatz 96
Definition
•
distance of a client C is the offset
between C and the immediate
following client.
Distance Based
Replace :
•
blocks consumed by clients with
largest distance
Example
C1
C2
C3
C4
Example
C1
C2
C3
C4
Example
C1
C2
C3
C4
Buffer Consumption
Wu & Yu 97
Definition :
•
Buffer consumption = amount of
buffer used x total time
Buffer Consumption
Result :
•
•
Minimizing buffer consumption
increase system throughput
Increase retrieval rate of current
stream is better then prefetching
next waiting stream
Open Problems
Querying
Query video by action ?
Query music by
•
relative pitch + chord + rhythm ?
Continuing quest to improve
accuracy
Indexing
Well studied problem
Indexing for approximate string
search ?
Buffer Management
Should take into consideration:
•
object dependencies (MPEG)