Transcript Meta-Search

IST 497E Information Retrieval
and Organization
Meta Search
IST 497 E Meta-Search, Pradeep Teregowda
Overview
What is a Meta Search Engine
Features
Differences
Architecture
Some Meta Search Engines
Conclusions and Future ?
Overview
What is a Meta Search Engine ?
What is a Meta Search Engine ?
Traveling Further
Why Meta Search ?
What Is a Meta Search Engine
?
Dictionary meaning for Meta:
“more comprehensive : transcending.”webster.com
Simple Explanation
“A Meta Search engine allows you to search
multiple search engines at once,
returning more comprehensive and
relevant results, fast.” –MetaCrawler
{modified}
Traveling Further
A little bit of History:

Started with Harvest (1995) –[Ref:
Information Discovery and Access
System – C. Mic Bowman et al].


Was developed for gathering information
from repositories, building topic-specific
content indexes, web-caching,flexible
searching.
In many ways current Meta Search
engines have similar aims.
Why Are Meta Search Engines
Useful ?
Meta Search improves the Search
Quality in many ways:
Comprehensive,
 Efficient,
 One query queries all {one-click
paradigm},

Why Meta Search ?
Individual Search engines don’t cover all
the web by themselves,
Individual Search Engines are prone to
spamming {people trying to raise their
ranking profile. In a non-legitimate manner
or to promote commerce},
Difficulty in deciding and obtaining results
with combined searches on different search
engines,
Why Meta Search ?
Data Fusion {multiple formats
supported},
In Case of niche search engines
provides the ‘big picture’,
Takes less effort.
Overview
Features
Features
Features.
Unifies the Search Interface and provides a
consistent user interface,
Standardizes the query structure,
May make use of an independent ranking
method for the results {rank-merging},
May have an independent ranking system for
each search engine/database it searches,
Meta Search is not a search for Meta Data.
Overview
Differences
Differences {Search vs. Meta Search}.
Differences {Search Vs. Metasearch}
Doesn’t generally have a Database by
itself,
Does not search{crawl} the web.

A Meta-Search Engine in terms of search
engine.

Essentially is a hub of search engines/databases
accessible by a common interface providing the user
with results which may/may not be ranked
independently of the original search engine/source
ranking.
Overview
Architecture/Internals
A block representation,
What do those blocks do ?,
Queries,
Ranking.
Architecture
Feedback
Query
User Interface
User
SE1
Knowledge
Personalize
Dispatcher
SE2
SE3
Web
Display
What Do Those Blocks Do ?
User Interface

Normally resemble search engine interfaces
with options for


Types of search [Media]
Search Engines to Use
Dispatcher


Generates actual queries to the search engines
by using the user query
May involve choosing/expanding search engines
to use
What Do Those Blocks Do ?
Display.

Generates Results page from the replies
received, May involve
ranking,parsing,clustering of the search results
or just plain stitching.
Personalization/Knowledge.

May contain either or both. Personalization may
involve weighting of search results/query/engine
for each user.
Queries
STARTS { protocol }.




A simple protocol that text search engines should
follow to facilitate searching and indexing multiple
collections of text documents.
Choosing best source for a query,
Evaluating a query at those sources,
Merging the query results from them.
Inquirus: Expand Queries.
Ex: (What does Satellite stand for = Satellite
stands for).
Independent Ranking
Stitch together the results [Dogpile],
Selection of a particular Search Engine
based on a query – a meta index
[SavySearch-Resource Balancing],
Rank merging based on Search Engine
rating [MetaCrawler],
Context analysis for search results with
respect to the query [Inquirus].
Fusion of Other Media
Inquirus.

Motivation for a meta-search {Images}:


How ?


Queries from the user can be modified, since
individual search engines are good at different types of
queries the results are very good.
See Reference {Skipped because of topic overlap}.
Can it work for others ?

Example: FTP,Music – Probably yes [P2P searches].
Other Work
Other work {Media Fusion}.

MetaSEEk.
Visual Search Engine [for images],
 Makes use of query by example.


Ixquick.


MP3,Images,News.[all come from different
search interfaces –may not exactly be fusion].
Dogpile.

Multimedia,Images,News,Files.
Overview
Some Meta Search Engines
On the web,
Why are they not so popular ?,
Some Reviews.
Meta search Engines on the
Web
MetaCrawler,
Ixquick,
Inquirus,
SavySearch {now cnet search ?},
Dogpile,
Sherlockhound,
Vivisimo.
Why Are They Not So Popular ?
Ads !

Some Meta Search engines pick up ads as part
of search results from the participating search
engines.


Example: Dogpile.
Similar reasons as general search engines (ads
clutter search results).

Example: MetaCrawler,Ixquick.
Paid Placement Search Engines included,
Relation with the search engines they
depend on. {load,interaction}….
Why Are They Not So Popular ?
Results as good as the worst search
engine in the group.
Combining results using meta-index and
rankings can lead to incorrectly ranked
results from a search engine reducing the
relevance of correct results.
 If your highly ranked search engine
returns a badly ranked result, then your
results are also badly ranked.

Some Reviews
Search Engine Watch [May 2001]
Meta Search Engine % Results From A
Major Search
Engine
Vivisimo
100 % [No Ads]
Ixquick
70 %
MetaCrawler
48 %
Dogpile
14%
Overview
Conclusions and Future
Conclusions and Future
Conclusions and Future
Vivisimo reported 43 % increase in
traffic –[wired.com Aug. 14, 2001],
Apple ‘Sherlock’ has been popular,
Multiple Media Search,
P2P Searches ?,
Growth or Niche Search Engines.
References
The MetaCrawler Architecture for Resource
Aggregation on the web(1997) – Erik
Selberg & Oren Etzioni.
Context and Page Analysis for Improved
Web Search(1998) –Steve Lawrence & C.
Lee Giles.
Experiences with Selecting Search Engines
Using Metasearch(1997) – Daniel
Dreilinger & Adele E. Howe.
Time to Wake Up…
Web References
Search Engine Watch Article [Meta
Search Or Meta Ads?],
Wired Article [Searching For Google’s
Successor],
MetaSEEk [Writeup],
Text and Image Meta Search on the
Web [Citeseer Paper Reference].