Transcript Whither Search Engine - Some Practical Recommendations

Internet Resources Discovery (IRD)
Whither Search Engine (SE)?!
Some Practical
Recommendations
1
A.Frank
Contents
•
•
•
•
•
2
When not to use SEs?
Use patterns of SEs
Rules for choosing SEs
Practical recommendations
General/Specialty subject SEs
A.Frank
Googlism 
3
A.Frank
When not to use SEs?
• You know it all.
• You prefer asking friends (or paid experts ).
• You know the Web site for it (and didn’t forget the
exact URL or have auto-completion or bookmark or
can access through another known site).
• You already found a specific/relevant digital library
or database (maybe in Invisible Web).
• Tired of paid inclusions, SE spamming, and
sponsored commercial results.
• Tired of chasing down useless URLs.
4
A.Frank
Use patterns of SEs (2004)
iProspect SE User Attitudes Survey Results, March 2004
(www.iprospect.com/) –
 57% of Web users use the same SE when they are looking for
information.
 Most searchers (92%, up from 71% in June 2002) are loyal to
their favorite SE, and stick with it (by modifying their query)
even if they don't initially find what they're looking for (in the
first 3 pages).
 Just 30% of Web users have a few specific SEs they use
regularly.
 Only 13% use a different SE depending on what they are
looking for at that time.
 Google has a "loyalty rate" of 66%, Yahoo! is next at 55%,
followed by MSN at 54% and AOL at 49%.
5
A.Frank
Web
WEB
Meta-Search Engine
Search Engine
Index
Directory
General Specialty
General
Specialty
Which kind to use? All 
6
A.Frank
When to use an Index?
• Need to search for a narrow piece of
information.
• Have a specific objective/site in mind.
• Want to find/rank many related Web sites.
• Want to factor quantity in (index has crawler
based results).
• Need to check/fix spelling (based on Web
statistics).
7
A.Frank
When to use a Directory?
• Clear about the exact topic of your query.
• Need general information on a rather broad
topic/category.
• Want to amass knowledge on a fairly wide subject.
• Would like to browse (and then search) a certain area.
• Want to factor quality in (directory has humanpowered results), not quantity.
• Need information that is usually carefully evaluated
and even annotated.
8
A.Frank
When to use a Meta-SE?
• When single Basic-SE fails to provide good results.
• One-stop shopping - prefer to search multiple
SEs/sites at once to get blended ranked results (so as
to save effort/time).
• When the query is simple (complex fields/options
don't usually work).
• Searching for multi-faceted topics.
• Want to get clustered results to focus search on the
relevant keywords.
• Looking for current events/news.
9
A.Frank
When to use a Specialty-SE?
• When general-SE fails to provide good results.
• When your target is very topic/technology
specific.
• Want to find more than just Web pages/sites.
• Need more results from the Invisible Web.
• Want your search terms to more likely have
the meanings you intended them to have.
10
A.Frank
So how to choose your SE(s)?
11
A.Frank
General rules for choosing SEs
• Use "major" SEs that are both well-known
and well-used (and that hopefully won’t be
downgraded or disappear soon ).
• Prefer SEs that employ both a huge index and a
comprehensive directory (gives better results;
can also switch between).
• Stick to SEs of established companies that treat
search as their main business/expertise.
12
A.Frank
Technical rules for choosing SEs
• Insist on a thin (user-friendly) interface but also
support an advanced search page.
• Fully index a broad range of file types.
• Provide multimedia (images, audio/MP3, video)
search tags (also news, products).
• Suggest spelling correction based on Web statistics.
• Have a featured toolbar (easier to invoke from the
desktop).
• Enable “Cursor Search” of word(s) on a Web page by
right-clicking the mouse.
13
A.Frank
Theory vs. Practice?!
14
A.Frank
Popularity of Google vs. Yahoo! 
 Search for “google” in
– Google: 48,700,000 results.
– Yahoo!: 57,500,000 results.
 Search for “yahoo!” in
– Google: 152,000,000 results.
– Yahoo!: 110,000,000 results.
15
 Maybe “Know thy enemy”?!
A.Frank
‫‪Common sense rules for choosing SEs‬‬
‫• "לעולם אל תשים את כל הביצים בסל אחד"‪:‬‬
‫‪ ‬בחר ביותר מאחד!‬
‫• "טובים השניים מהאחד"‪:‬‬
‫‪ ‬בחר בשניים עיקריים!‬
‫• "‪ ...‬עד שיבוא (הכתוב) השלישי ויכריע ביניהם"‪:‬‬
‫‪‬כדאי אבל שיהיה גם שלישי (מסוג שונה) כגיבוי!‬
‫• "בתי הקברות מלאים בכאלו שחשבו שאין להם תחליף"‪:‬‬
‫‪ ‬אז כדאי שיהיו גם תחליפים לכל אחד ואחד!‬
‫‪A.Frank‬‬
‫‪16‬‬
Top 5 Search Engines/Sites (Jan 2004)
17
A.Frank
Share of Searches Sites (Dec 2004)
18
A.Frank
Share of Search Providers (Dec 2004)
19
A.Frank
Practical recommendations
• Two major SEs (usually use both):
1. Google (GG)
–
Alternative: Personalized Google 
(labs.google.com/personalized)
2. Yahoo! search (YH)
–
Alternative : Teoma/AskJeeves (AJ)
• One Meta-SE (as a backup):
3. DogPile
–
20
Alternative: Vivisimo\Clusty
 Note: Choices are not Hebrew oriented.
A.Frank
‫המלצות פרקטיות (לעברית)‬
‫•‬
‫שני מנועי חיפוש עיקריים (לרוב שימוש בשניהם)‪:‬‬
‫‪ .1‬גוגל ישראל‬
‫–‬
‫חלופה‪ :‬מורפיקס (לצורך המורפולוגיה)‬
‫‪ .2‬וואלה! חיפוש‬
‫–‬
‫•‬
‫חלופה‪ :‬נענע‪/‬תפוז (אבל כולם פורטלים בעיקר)‬
‫מנוע חיפוש‪-‬על אחד (כגיבוי)‪:‬‬
‫‪ .3‬סטארט‬
‫–‬
‫חלופה‪( 2Find :‬מנוע חיפוש משלנו ‪)‬‬
‫‪ ‬הערה‪ :‬התמיכה שלהם בעברית עדיין לא מושלמת‪.‬‬
‫‪A.Frank‬‬
‫‪21‬‬
So what was the message ?
22
A.Frank
So how does it look?
23
A.Frank
Why use Google? (1)
• Biggest, most comprehensive coverage:
 ~8 billion Web pages (but ~1 billion of it isn’t full-text
searchable!)
 ~10 billion documents, if you count images and newsgroup
postings.
• Fastest around.
• Most relevant results (voted 3 times most outstanding SE by
Search Engine Watch readers).
• Provides good directory results (PageRanks results
of DMOZ Open Directory).
24
A.Frank
Why use Google? (2)
• Has thinnest interface around.
• But provides rich set of advanced search
features/tools(/hacks).
• Finds similar/related pages.
• Supports Web pages translation.
• Cached (HTML) copy of pages (great for quick
view of DOCs/PDFs and for 404s ).
• Google alert – use of push technology.
25
A.Frank
Why use Yahoo! search? (1)
• Has brand new Yahoo! search – gives highly
relevant Web results (at Google level ).
• Still supports an expert’s humanly-compiled
directory (dir.yahoo.com).
• Has (also) a thin interface (search.yahoo.com)
while providing a rich set of advanced search
features/shortcuts.
26
A.Frank
Why use Yahoo! search? (2)
• For legacy reasons (oldest of all directories).
• Puts particular emphasis on personalization
and customization (my.yahoo.com).
• Had enough of Googlism (www.googlism.com ).
• It devoured/uses (know-how from) Overture
(Inktomi, AltaVista and AllTheWeb, etc)
27
A.Frank
Yahoo acquisitions!?
Yahoo
google
28
A.Frank
4 Things Yahoo can and Google can’t!
• Find websites linking to a page
– linkdomain:
• Mix syntax
– Link:amdocs.com site:gov
• Long queries (>32 terms)
– Especially important when using OR
• Search for XML/RSS
29
A.Frank
New alternative - Why use MSN?
• Fresh from the Oven -- Launched Nov 11,
2004.
• Vast index of information: ~5 billion
documents.
• The most up-to-date information – MSNbot is
active all the time.
• Direct answers -- from Microsoft Encarta®,
encyclopedia.
• Direct actions -- to MSN channels.
30
A.Frank
Why use Teoma?
• It’s an expert/guide (term in Gallic ).
• Provides “subject specific” ranking of pages.
• One search – three responses:
1. Results: Lists relevant Web pages
2. Refine: Suggestions to narrow your search.
3. Resources: Recommends link collections
from experts and enthusiasts.
31
A.Frank
Why use AskJeeves (that uses Teoma)?
• Provides a Natural Language interface (uses
NLP).
• Suggests related searches.
• Hides technical details (of Teoma).
• Just purchased Interactive Search Holdings
(MyWay, MySearch, My Web Search, iWon,
and Excite).
32
A.Frank
Why use DogPile?
• Customizable list of indices, directories and SSEs.
• Indices searched include Google, Yahoo, and
AskJeeves/Teoma. Directories searched include About,
LookSmart and Open Directory.
• View results by their relevance or by search engine.
• Designed to identify the (non-)commercial intent of a
user's search - proposes to refine your results.
• Winner of “Best Meta Search Engine” award from
Search Engine Watch for 2003.
33
A.Frank
Why use Vivisimo\Clusty?
• Indices searched include MSN, GigaBlast, Lycos, and
WiseNut. Directories searched include LookSmart and
Open Directory.
• Provides automatic clustering in hierarchical folders.
• The results are grouped into successively narrower
subcategories, allowing to drill down through a topic
without additional searching.
• Won second place for “Best Meta Search Engine” in
the 2003 Search Engine Watch awards and winner in
2002.
34
A.Frank
Choice of SEs is a delicate balance
35
A.Frank
General Subject SEs
lii.org
• Librarians’ carefully selected index to the internet
www.ipl.org/
• Evaluated and annotated subject directory
infomine.ucr.edu
• Selected scholarly internet resource collections
www.about.com/
36
• Academic collection of "sites" on many subjects
 www.finderseeker.com/
• Search Engine for Specialty Search Engines
A.Frank
Specialty Subject SEs
 www.search.com/sitemap
• Directory for Meta Specialty Searches
 www.leidenuniv.nl/ub/biv/specials.htm
• A collection of special SEs
 www.academicinfo.net/
• Educational subject directory
 www.searchability.com/
• Multi-subject guides to specialized SEs
 lib.nmsu.edu/instruction/specialtysearch.htm
• Lists of Specialty Search Engines
37
A.Frank
Ready Reference Desks
38
www.refdesk.com/
• Comprehensive reference desk
www.ipl.org/
• Ready reference collection
lii.org/search/file/reference
• Ready reference and quick facts
academicinfo.net/reffind.html
• Educational reference desk
www.faganfinder.com/
• Help people find what they are looking for
A.Frank
If more time ... we could SEEk more 
39
A.Frank
Bibliography/Credits
 http://searchenginewatch.com/reports/article.php/2156431
 searchengineshowdown.com/
 www.noodletools.com/debbie/literacies/information/5l
ocate/adviceengine.html
 infopeople.org/search
 www.lib.berkeley.edu/TeachingLib/Guides/Internet/
 www.monash.com/spidap.html
 www.searchlore.org
 www.li-net.net/il/search (Hebrew)
40
A.Frank
Bottom line
Seek/Search the best way for you!
41
A.Frank