Transcript ppt1
Advanced Seminar – University of Trento – June 2006 Michela Lecca - TeV What is COMPASS COMPASS is a distributed application for content-based image retrieval using remoted databases. It has been developed in 2000 by Roberto Brunelli and Ornella Mich by the group of Technology of Vision, ITC – irst, Trento, Italy Michela Lecca - TeV The problem The considerable amount of multimedia database requires sophisticated indices for its effective use. Manual indexing is the most effective method to do this, but it is also the lowest and the most expensive. Automated methods have to be developed. Michela Lecca - TeV Multimedia Data Management The traditional methods based on the textual description and searches are no more practicable for two reasons: 1. associating textual description to multimedia data can be very expensive; 2. textual description may be not adequate for multimedia retrieval. Michela Lecca - TeV COMPASS COMPASS can be used for two main activities: 1. to search database images similar to a query image (query by example); 2. to browse image databases . Michela Lecca - TeV Query by example COMPASS is a client-server architecture in which a client application submits a user query to multiple image servers. The answers are then merged and proposed to the user as a single results. Michela Lecca - TeV Query by example Image Databases One or more query images Description of the database images by low-level features Description and Comparison with the Described Image Databases Described Image Databases Ranked List of Responses Michela Lecca - TeV Image Description The images in the database, such as the user query, are described by low-level features. Michela Lecca - TeV Image Description (2) Simple General Descriptors: to describe the general visual appearance of the image or of a region of interest; Texture Descriptors: to quantify and qualify properties such as smoothness, coarseness and regularity (pattern replication); Shape Descriptors: to describe the shape of an interest image region. Michela Lecca - TeV Simple General Descriptors (1) The color properties are described by 1. Hue 2. Saturation 3. Luminance Michela Lecca - TeV Simple General Descriptors (2) Color Space HSI The color properties are described by using the space HSI. Michela Lecca - TeV Simple General Descriptors (3) Hue Color perceived as ranging from red through yellow, green and blue, as determined by the dominant wavelenght of the light. In HSL space it is measured in degrees. green (120o) cyan (180o) blue (270o) yellow (60o) red (90o) violet (300o) Michela Lecca - TeV Simple General Descriptors (4) green yellow Saturation Hue It defines how gray the color is and it ranges in [0, 1]: 0 is cyan gray, 1 is a pure color. red blue violet Michela Lecca - TeV Simple General Descriptors (5) Intensity It defines the lightness of the colors. Variability range: [0,1]. intensity = 0.3 intensity = 0.4 intensity = 0.5 Michela Lecca - TeV Simple General Descriptors (6) Edges Distribution The edgeness is defined as dx I + dy I where I is the intensity of the image. So it is related to the image gradient. Michela Lecca - TeV Simple General Descriptors (7) The distribution of the simple general descriptors are represented by 16 bins histograms. [R. Brunelli – O.Mich, On the Use of Histograms for Image Retrieval, Proc. of IEEE ICCM 1999, Florence, Italy] Michela Lecca - TeV Texture Descriptors (1) Texture means properties such as smoothness, coarseness, and regularity (pattern replication). Michela Lecca - TeV Texture Descriptors (2) Intensity and Hue Co-occurrence Histograms: 2-D histograms of distributions of hue and intensity, of two pixels related each to other by a positional operator T. In COMPASS, indicated by (x, y) and (x', y') the position in the image of the pixels p and p' resp., T is defined as T(x, y) = (x + 2, y + 2). p and p' are T-related iff (x', y') = T(x, y). Michela Lecca - TeV Texture Descriptors (3) Intensity and Hue Co-occurrence Histograms: Intensity co-occurrence : to describe textural information of grey level image; Hue co-occurrence: to describe the textural properties of color image. Michela Lecca - TeV Texture Descriptors (4) Wavelets: Texture is strictly related with the scale factor at which an image (or a portion of it) is observed. Wavelets are a powerful approach to multiresolution image processing. Michela Lecca - TeV Texture Descriptors (5) Wavelets: The wavelet transform is a decomposition of the signal into different frequency components by means of a family of real orthonormal bases fa, b(x) obtained through translation and dilation of a kernel function f(x) called mother wavelet: fa,b(x) = a-1/2 f((x-b)/a) Michela Lecca - TeV Texture Descriptors (6) Wavelets: The wavelet transform of a continuous signal f(x) is Wa, b f = ʃ f(x) f*a, b(x) dx where f* indicates the analyzed wavelet. [S. Mallat, A Theory for Multiresolution Signal Decomposition: The Wavelet Representation, Proc. IEEE PAMI, Vol. 11, N. 7, 1989] Michela Lecca - TeV Shape Descriptors (1) For region shape descriptors: 52 Li Moments: to describe the shape of a region; these moments are computed by using the FourierMellin transformation; Fourier Coefficients: to describe the external contour of a region; they represent the shape in the frequency domain (low frequency -> general features, high frequency -> details). Michela Lecca - TeV Shape Descriptors (2) 52 Li Moments: Y. Li, Reforming the theory of invariant moments for pattern recognition, Pattern Recognition, 25(7), 1992 Fourier Coefficients: A lot of Books of Mathematical Analysis ... Michela Lecca - TeV Descriptor Invariance and Coding All the used features are normalized to be invariant by translation, rotation, rescaling and combination by thereof. The low-level features are then encoded in a vector (96-dimensional vector in original version, 208 in the extended version), whose entries vary in the range [0, 255]. Michela Lecca - TeV Image Comparison (1) The similarity between the query image and each item of the remoted databases is defined as L1-distance between the corresponding feature vectors v = [v1, ..., vn], w = [w1, ..., wn]: d(v, w) = Si=1, ..., n | vi - wi| [R. Brunelli – O.Mich, On the Use of Histograms for Image Retrieval, Proc. of IEEE ICCM 1999, Florence, Italy] Michela Lecca - TeV Image Comparison (2) In the case of queries with multiple images, COMPASS uses warpable metrics to emphasize (or minimize) the impact of the most (or less) relevant features of the image query set. [See references for details.] Michela Lecca - TeV Performances The search process is very efficient: retrieving the closest item in a database of 1 million elements takes less than a second on a standard Pentium4 with 2GHz CPU !!! http://compass.itc.it/compareIt.html Michela Lecca - TeV Browsing Databases Database images are clustered in groups of images similar to each other according to their L1- distance. Each cluster is then represented by a key image, the closest to the center, and acts as hyperlink to the elements of the cluster. Michela Lecca - TeV COMPASS Interface [http://compass.itc.it/demos.html] Michela Lecca - TeV References ● ● ● R. Brunelli, O. Mich, COMPASS: an Image Retrieval System for Distributed Databases, Proc. of ICME 2000, New York, USA R. Brunelli, O. Mich, Image Retrieval by Example, IEEE Transaction on Multimedia, Vol. 2, N. 3, 2000 C. Andreatta, CBIR Techniques for Object Recognition, Tech. Rep. ITC-irst 04-12-01, Dec. 2004 http://compass.itc.it/papers.html Michela Lecca - TeV