Transcript 按一下以新增標題
Content-Based Image
Retrieval Using Fuzzy
Cognition Concepts
Presented by
Tienwei Tsai
Department of Computer Science and Engineering
Tatung University
2005/9/30
Outline
1. Introduction
2. Problem Formulation
3. Proposed Image Retrieval System
4. Experimental Results
5. Conclusions
1. Introduction
• Two approaches for image retrieval:
– query-by-text (QBT): annotation-based image
retrieval (ABIR)
– query-by-example (QBE): content-based
image retrieval (CBIR)
• Standard CBIR techniques can find the
images exactly matching the user query
only.
• In QBE, the retrieval of images basically has
been done via the similarity between the query
image and all candidates on the image database.
– Euclidean distance
• Transform type feature extraction techniques
– Wavelet, Walsh, Fourier, 2-D moment, DCT, and
Karhunen-Loeve.
• In our approach, the DCT is used to extract lowlevel texture features.
– the energy compacting property of DCT
2. Problem Formulation
• Let I be the image database with I := {Xn | n = 1, . . ., N}
where Xn is an image represented by a set of features:
Xn := {xn m | m = 1, . . ., M}.
– N and M are the number of images in the image database and
the number of features, respectively.
• To query the database, the dissimilarity (or distance)
measure D(Q, Xn) is calculated for each n as
M
D(Q, X n ) wm .d m (qm , xnm ), for n 1, ..., N .
m1
– dm is the distance function or dissimilarity measure for the mth
feature and wm R is the weight of the mth feature.
– Query image Q := {qm | m = 1, …, M}.
– For each n, holds. By adjusting the weights wm it is possible to
emphasize properties of different features.
3. The Proposed Image
Retrieval System
Figure 1. The proposed system architecture.
Feature Extraction
• Features are functions of the
measurements performed on a class of
objects (or patterns) that enable that class
to be distinguished from other classes in
the same general category.
• Color Space Transformation
RGB (Red, Green, and Blue) ->
YUV (Luminance and Chroma channels)
YUV color space
• YUV is based on the CIE Y primary, and also
chrominance.
– The Y primary was specifically designed to follow the
luminous efficiency function of human eyes.
– Chrominance is the difference between a color and a
reference white at the same luminance.
• The following equations are used to convert from
RGB to YUV spaces:
– Y(x, y) = 0.299 R(x, y) + 0.587 G(x, y) + 0.114 B(x, y),
– U(x, y) = 0.492 (B(x, y) - Y(x, y)), and
– V(x, y) = 0.877 (R(x, y) - Y(x, y)).
Discrete Cosine Transform
2 Feature Extraction via DCT
• The DCT coefficients F(u, v) of an N×N image
represented by f(i, j) can be defined as
N 1 N 1
(2i 1)u
(2 j 1)v
2
) cos(
),
F (u, v) (u) (v) f (i, j ) cos(
2N
2N
N
i 0 j 0
where
1
( w) 2
1
for w 0,
otherwise.
Characteristics of DCT
• the DC coefficient (i.e. F(0, 0)) represents
the average energy of the image;
• all the remaining coefficients contain
frequency information which produces a
different pattern of image variation; and
• the coefficients of some regions represent
some directional information.
Similarity Measurement
• Distance measure
– the sum of absolute differences (SAD): avoid
multiplications.
– the sum of squared differences (SSD): exploit
the energy preservation property of DCT
• The distance between qm and xnm under
the low frequency block of size k×k :
) F
k 1 k 1
d m (qm , xnm
u 0 v 0
qm
(u, v) Fxnm
(u, v)
2
Fuzzy Cognition Query
• To benefit from the user-machine
interaction, we develop a GUI for fuzzy
cognition, allowing users to adjust the
weight of each feature more easily
according to their preferences.
• Each image is represented by M features.
• Three features (i.e., luminance Y,
chrominance U, and chrominance V) are
considered for each image.
4. Experimental Results
• 1000 images downloaded from the WBIIS
database are used to demonstrate the
effectiveness of our system.
• The user can query by an external image
or an image from the database.
• In our experiments, we found that the low
frequency DCT coefficients of size 5×5 are
enough to make a fair quality of retrieval.
Figure 2. Retrieved results using a butterfly as the query image and
its luminance as the main feature.
Figure 3. Retrieved results using a butterfly as the query image and
emphasizing the weight of its V component.
(a)
(b)
Figure 5. Retrieved results using a mountain scene as the
query image and Its Y component as the main feature:
(a) the query image; (b) the retrieved images.
(a)
(b)
Figure 4. Retrieved results using a mountain scene as the
query image and Its U component as the main feature:
(a) the query image; (b) the retrieved images.
5. Conclusions
• In this paper, a content-based image retrieval method
that exploits fuzzy cognition concepts is proposed.
• To achieve QBE, the system compares the most
significant DCT coefficients of the Y, U, and V
components of the query image and those of the images
in the database and find out good matches by the help of
users’ cognition ability.
• Since several features are used simultaneously, it is
necessary to integrate similarity scores resulting from the
matching processes.
• An important part of our system is the implementation of
a set of flexible weighting factors for this reason.
Future Works
• For each type of feature we will continue investigating
and improving its ability of describing the image and its
performance of similarity measuring.
• A long-term aim is combining the semantic annotations
and low-level features to improve the retrieval
performance.
• For the analysis of complex scenes, the concept that
provide a high amount of content understanding enable
highly differentiated queries on abstract information level.
The concept is worthy of further study to fulfill the
demands of integrating semantics into CBIR.
Thank You !!!