Transcript Relevance Feedback and User Interaction for CBIR

Relevance Feedback and User
Interaction for CBIR
Hai Le
Supervisor:
Dr. Sid Ray
Outline







Introduction to CBIR
CBIR query structure
Feature weighting
Objectives
Plans and approaches
Current progress
Conclusion
Introduction to CBIR




CBIR systems are required to meet users needs for
image retrieval
Retrieval by query, such as an example image
Application areas include, weather forecasting, medical
research, fabric design, WWW search just to name a
few..
CBIR systems which are commercially available
include IBM’s QBIC, Blobworld, VisualSeek, Virage etc.
Current Approaches to CBIR





Current approaches follow a similar routine
Extract lower level features of the images
Measure the degree of similarity between
image features
Apply weighting to the most predominate
features
Index the images based on the combined
similarity of features
Image Features

Colour
–
–

Texture
–
–
–

Global colour histogram
Colour correllogram
Coarseness
Contrast
Directionality
Shape
–
–
Area
Shape moments
Indexing and Similarity
Measurements



Queries are translated into a point in a multi
dimensional feature space
The most similar images are those which are closest to
the query point, numerical values can be obtained
Common distance measures include:
–
–

Euclidean
City block
Some indexing structures include:
–
Bayesian networks
Feature Weighting and Relevance
Feedback





Many features may be used to determine similarity
These features need to be combined
Some features may be deemed more important than
others
Feature weighting provides a means to distinguish the
importance of various features
Questions which arise include, how do we determine
the importance of particular features?
Summary of CBIR query structure
Input query image and
image database
Extract Image features
Calculate the weighting
of the various features
Index images by sorting the images
according to a distance measurement
Present retrieved images to user
User Feedback?
Summary of CBIR query structure
Input query image and
image database
Extract Image features
Calculate the weighting
of the various features
Index images by sorting the images
according to a distance measurement
Present retrieved images to user
User Feedback?
Feature Weight Calculations


Assign higher weights to more important
features and lower weights to less importance
features
Deciding which features are more important:
–
–
Analyse the contents of the image database and
determine weights based on the variation of
features
Use user perception and relevance feedback to
determine which features are more important
Methods of weight calculation



Analyse the feature content of the images in
the database in respect to the features of the
query image
Features with high amounts of variation are
deemed more important than those with small
amounts of variation
These features can be given a higher
weighting during distance calculations
Methods of Weight calculation

Hore and Ray methods involved
–
–

Using the standard deviation (σ) of the distances
between the query image and the database images
for a particular feature. A high standard deviation
would mean a large amount of variation for that
particular feature
Using the standard deviation of the entire image
database divided by the standard deviation of the
top N images for a particular feature.
Second method provided the best results
Methods of Weight calculation



Use of user perception to determine the importance of
various features
User selects positive and negative images from the
retrieved image set
Different systems handle the positive and negative
images in different ways some of these include:
–
–
Determining the spread of the positive and negative images
Comparing the positive and negative images to the original
query image
Methods of Weight calculation




Naster et. al, proposed that the probabilities of the
images in the database are updated depending on the
location of positive and negative images
MUSE developed by Marques et. al uses a unique
system in which images which are similar for a certain
features are clustered together
The images which belong to certain clusters are
updated depending on the positive and negative
images selected by the user
Feature weighting is calculated depending on the users
selection habits
Objectives of the project



Devise a feature weighting scheme based on
the approaches previously mentioned
Devise an analysis for feature weighting
Develop a CBIR testbed
Plans and approaches



Develop a simple CBIR testbed using colour
and shape features
Build on the weighting scheme devised by
Hore and Ray by including user interaction
Use rank correlation coefficient to distinguish
between computational and perceptual results
Current progress

Image database of 40 images including:
–
–



6 groups of 5 images containing similar images based on
colour and shape
1 group of 10 images containing random images
Simple CBIR testbed using histogram matching based
on a quantised RGB colour space
Used R, G, B channels as separate features, for testing
purposes
Weight calculation based Hore and Ray method using
standard deviation
Conclusion




CBIR systems have numerous applications, and
incorporate many areas of computing
There is a discrepancy between computational results
and perceptual results due to the use of low level
image features
These problems can be overcome by using feature
weighting and by involving the “user in the loop”
The project will build on previous feature weighting
methods
More about the project
http://csse.monash.edu.au/~hle
[email protected]