Transcript WHIM_prsentation - Columbia University

Introduction to
semantic search engine
Tiwei Chen
Spring 2009
31 March 2009
1
Keyword Search Engine
• We are able to combine the
information from different web
resources, even if they use
different terminologies and
languages
31 March 2009
2
Keyword Search Engine
• Iphone example:
I want to find the information of cheapest
iphone. I can start form Google or Ebay to
search whether there is someone selling
the iphone. Then, I can collect all the
useful information appearing on the
searching result by myself.
31 March 2009
3
Keyword Search Engine
• Restaurant example:
I want to find a restaurant with good
quality and close to my home. Then my
first step might be searching on the food
website and finding the evaluation of some
restaurants. After collecting some useful
information, I can decide which restaurant
is more suitable for me to have my dinner.
31 March 2009
4
Semantic Search Engine
different result?
• Restaurant example:
We can type in what we need with our
natural language. After computer receives
our natural language request, it might
further ask whether we can accept the
price above 300 dollars or do we mind to
have Italian food.
• Allow us to interact with computer and
type in more opinion with our natural
language
31 March 2009
5
Keyword Search Engine
• Problem: synonyms
• Example:
When a user uses apple as a key word to
search, search engine might feed back the
results in fruit domain or Apple computer
domain
• If we search virus on the internet, we can
find virus related to computer science
field. And we can also find different
definition of virus in biology and medicine
domain.
31 March 2009
6
Semantic Web
• Usually put things in order according to
the meaning of word
• Advantage:
It helps to search in domain knowledge.
• Construing a tree structure can let the
root has some relation to their children.
One subclass can be an attributes or an
object.
31 March 2009
7
Semantic Search Engine
• Extend the range of searching by
Semantic Web
• Use hierarchical and vertical
structure to search data. This
method related to specific domain
knowledge
31 March 2009
8
Semantic Search Engine
• Hakia, Evri, Poweset, Cognition:
We can use a natural language to
describe our need when we do the
search
• Allow users use a word, a phrase or a
sentence to search web pages
31 March 2009
9
Semantic Search Engine
• Example:
• When I type in “Where is Columbia
University?”
• There must be a parser used to parse
a sentence and analyze the structure
of the sentence.
• Does the word contains any
knowledge?
31 March 2009
10
Probabilistic latent
semantic analysis
• Example:
• If we type in “the weather in New York”
• The words “the” and “in” might appear
many times in our corpus. “The” and “in”
contains no knowledge in it although it
appears usually.
• The corresponding result should be more
concentrate on “New York” and “weather”
when we do the search
31 March 2009
11
Semantic Search Engine
• Difficulties:
Users might type in a sentence with
strange grammar structure or including
some complicated grammar structure.
Search engine can hardly understand this
kind of strange sentences which it has
never “learned” before.
31 March 2009
12
Evri
• Similar to a database system:
tree diagram in Evri is similar to E/R diagram in a
relational database system
• It allows users connect to another website
according to the meaning of the word and gives
hyperlink an attribute according to its meaning
31 March 2009
Kaiser: COMS E6125
13
How to construct semantic
search engine?
• People disclose more personal preference
to the search engine ->
actually they are going to create their own
personal semantic web and give a meaning
for the material
• Many users create their personalized
semantic webs->
search engine companies can aggregate all
the semantic webs and construct a bigger
semantic web
31 March 2009
14
Advantages
• The content of knowledge also can be
updated from user contribution
• Searching engines can analyze users’
behavior through this approach
• Enables every user share its own semantic
web and take advantages of other people’s
semantic web
31 March 2009
15
Challenge in implementation
semantic search engine
• How to parse the sentence?
• Parser!
• How to differentiate the synonyms/
How to organize data?
• Semantic Web!
31 March 2009
16
Conclusion
• Search less, understand more
31 March 2009
17
• Thank you
31 March 2009
18