Transcript ppt

Finding textures by textual
descriptions, visual examples,
and relevance feedbacks
Author: Hsin-Chih Lin, Chih-Yi Chiu, ShiNine Yang
Source: Pattern Recognition Letters 24
(2003) 2255–2267
Reporter:簡程輝
1
/25
Introduction(1/6)

This study propose a fuzzy logic CBIR system, named as
LinStar Texture, for finding textures through textual
descriptions, visual examples, and relevance feedbacks.

Semantic gap
:
The gaps that are between low-level
features (e.g.,numerical vectors) and high-level concepts
(e.g.,textual descriptions)

How to overcome the drawback : Use a fuzzy logic CBIR
system, map low-level features to high-level concepts
2
Introduction(2/6)

Texture: Tamura features including coarseness, contrast,
regularity,directionality, line-likeness, and roughness

Fuzzy clustering: Five linguistic terms characterize highlevel textual concepts of textures.

Similarity definition: A query is expressed as a logic
composition of linguistic terms or feature values.
3
/25
Introduction(3/6)
4
Introduction(4/6)
5
/25
Introduction(5/6)
6
Introduction(6/6)
Database
creation
Query
comparison
System architecture: (a) database creation (b) query comparison
7
/25
Tamura features(1/7)

Coarseness




Texture granularity.
The size and number of texture primitives.
n×n: image size
k is obtainedas the value which maximizes the
differences of the moving averages
taken over a 2k×2k neighborhood
Fine
,
Coarse
8
/25
Tamura features
Tamura features(2/7)

Contrast


The difference in intensity among neighboring
pixels.
μ4 is fourth moment of the image.
High contrast
Low contrast
9
/25
Tamura features (3/7)

Directionality

The shape of texture primitives and their
placement rule.
Directional



Non-directional
HD is the local direction histogram
np is the number of peaks of HD
wp is the range of pth peak between valleys
10
/25
Tamura features (4/7)

Directionality
A texture image
11
/25
Tamura features (5/7)

Line-likeness


The shape of texture primitives. It is often
simultaneously directional.
PDd is the n×n local direction cooccurrence matrix
of points at a distance d.
Line-like
Blob-like
12
/25
Tamura features (6/7)

Regularity

Variations of the texture-primitive placement.
Regular
Irregular
Divided an image into blocks??
regular texture : composed of identical or similar primitives
irregular texture : composed of various primitives,
13
/25
Tamura features (7/7)

Roughness



Tactile variations of physical surface
A rough texture contains angular primitives.
A smooth texture contains rounded blurred primitives.
Rough
Smooth
14
/25
Linguistic terms for the six Tamura features
Membership functions
of a Tamura feature.
15
/25
Linguistic terms for the six Tamura features
16
Fuzzy Clustering (1/3)n
xi
5
17
/25
Fuzzy Clustering (2/3)
18
Fuzzy Clustering (3/3)
An Image
Coarseness
Contrast
Directionality
Line-likeness
Regularity
Roughness
19
/25
Querying by textual descriptions

Query can be expressed as a logic composition of
linguistic terms.
20
/25
Querying by textual descriptions
A textual description:
(fine  regular)  very high
21
/25
Querying by visual examples


User submits a visual
example with a specified
constrain
The query can be
expressed as a logic
composition of feature
values.
22
/25
Querying by visual examples

A visual example with a specified constrain:
similar on (coarsenes s  regularity )  contrast
23
/25
Querying by relevance feedbacks


After initial search, the user can give relevant
and/or irrelevant examples to refine the query
and improve the retrieval efficiency.
Assume a user gives two relevant examples.


Similar to both images
Similar to either of the two images
24
/25
Experimental results
25
/25
Experimental results
26
/25
Experimental results

Effectiveness:



recall
precision
T is the number of retrieved images.
n is the number of relevant images.
N is the total number of relevant images in the
database (N=9 in our experiments).
27
/25
Experimental results

Average retrieval time (45 queries)


A textural description query: < 0.01s
A visual example query: 0.11s
28
/25
Conclusions and Comments




在此用 fuzzy clustering 作分群 ,因語意描
述材質上還是模糊地帶蠻重的表達方式,必須
利用fuzzy 處理,以找出規則,趨近真意
運用隸屬函數方式找出其隸屬值,取用隸屬值
來作比較,將材質的特性量化,才能以其量化的
值做比對
運用這些的方式其實還是對語意內容還是會有
落差,仍須人以視覺去修正以達到所要的需求
可以運用這種fuzzy觀念及相關性回饋的觀念
找尋地磚、布、紙質…等
29
/25
30
Conclusions




Mapping from low-level Tamura features to
high-level linguistic terms
Textual => concepts, linguistic terms
Relevance feedbacks
Highly intuitive and effective
31
/25