Transcript elvington

Multimedia Databases
(MMDB)
A Content-Based
Image Retrieval Perspective
(CBIR)
Types of Media Files

Static media: images and handwriting

Dynamic media: video and sound bytes

Dimensional media: 3D games or CAD)
MMDB Motivation Factors

Acquisition: email, phone, web sites like FLIKR

Generation: camera phones, digital cameras,

Storage: databases design

Processing: power and techniques more sophisticated

Huge increase in multimedia data on computers and
their transmission over networks.
Database Support

Databases provide consistency,
concurrency, integrity, security and
availability of data for the large amount
of multimedia data available.

From a user perspective, databases
provide functionalities for manipulation
and querying the huge collections of
stored data.
Media Data Stored

Media data: actual media data representing images, audio, and video
that are captured, digitized, processed, compressed and stored.

Media format data: consists of media data format stored after the
acquisition, processing, and encoding phases. Examples are sampling
rate, resolution, frame rate, encoding scheme etc.

Media keyword data: For example, for a video this might include the
date, time, and place of recording , the person who recorded, the scene
that is recorded. Also called content descriptive data.

Media feature data: This contains the features derived from the media
data. A feature characterizes the media contents. For example, this could
contain information about the distribution of colors, the kinds of textures
and the different shapes present in an image. This is also referred to as
content dependent data.
Image Retrieval (IR)

Image Retrieval is the process of
searching and retrieving desired images
from a large database.

IR provides resourceful use of prolific
image data

The efficiency of implementations have
increased over the past two decades
IR Methodology

A simple image retrieval implementation uses
individually entered keywords or descriptions of
inserted images so that retrieval is performed over the
annotations in normal textual forms.

If an image is poorly or incorrectly annotated, or a poor
choice of arbitrary query values are given by the user
then the desired output is not received even if it exists
in the database.

Therefore, a lot of research has gone into automatic
annotation of image description.
Image Features Stored

Color: Red, Blue, Green, etc

Texture: Similarity in grouping of pixels

Shape: Edge detection

Spatial: Spacing of Features

Semantic: Correlated description of image data. E.G:
Color = blue, Shape = Large, Texture = smooth,
Spatial = Top of image: Sky
Feature Extraction

Image Segmentation: open ended topic

Segment Classification: based off
characteristics

Filtering Techniques: Extract image
features such as texture by passing
images through a filter
Feature Application In DB

Users could supply a range for color,
texture, or shape for queries

Features can be generated on a typical
semantic set for automatic annotation of
new pictures
Content based image Retreival
(CBIR)

Avoids the necessary use of textual
descriptions

Organizes digital archives by visual content

Retrieves images based on visual similarity to
a user-supplied query image or image
features.
Query Types

Keyword: common text searching techniques

Feature: Ex. Draw area for location and size.
Select color regions. Select shape. B+-tree is
traversed based off given index value.

Semantic: Provide words to describe feature
sets that are used to query a database

Composite: Index involves combination of
above
Query examples from CIBR at the
end of the early years
Content-Based

Straight-forward implementation is each
feature is used as an index. Not very efficient
for querying

Create an index as described earlier as a
combination of region classification, spatial
location, shape, and color.
EX. 20-bit index key: 3bits location, 8 bits
color, 4 bits size, 5 bits shape. B+-tree
indexing method is used.
Relevance Feedback

A query modification technique attempts
to improve retrieval performance through
iterative feedback and query refinement.

Used in ALIPR
Data Flow From CBIR at the end of
early years
IR Implementation Examples

Yahoo or Google Image Searches:
based mainly on annotated description
and filename

Automatic Linguistic Indexing of Pictures
(ALIPR): learning algorithm that
annotates with feedback
Future Work

The open ended nature of image segmentation
restricts the accuracy of object recognition. As
segmenters improve so will the databases capability.

The integration of image retrieval can be implemented
in computer vision applications.

Many researchers believer that image retrieval has
grown out of its infancy and now focus will be on
applications and proliferating algorithms into indivduals
lives.
Bibliography

Datta, R., Joshi, D., Li, J., and Wang, J. Z. 2008. Image retrieval: Ideas,
influences, and trends of the new age. ACM Comput. Surv. 40, 2 (Apr. 2008), 160. DOI= http://doi.acm.org/10.1145/1348246.1348248

Stanchev P., Using Image Mining for Image Retrieval, IASTED International
Conference “Computer Science and Technology”, May 19-21, 2003, Cancun,
Mexico.

"Multimedia Database." Information Technology Portal (IT Portal) - India. Web. 04
Dec. 2009. <http://www.peterindia.net/MultimediaDatabase.html>.

"Image retrieval -." Wikipedia, the free encyclopedia. Web. 04 Dec. 2009.
<http://en.wikipedia.org/wiki/Image_retrieval>.

Smeulders, “Content-based image retrieval at the end of the early years”
A.W.M.Journal:IEEE transactions on pattern analysis and machine intelligence,
2000, Vol:22, 12, 1349