Transcript General presentation template

A Sentiment-al Journey:
Text and Sentiment Analysis of Online
Discussion Responses
Susan Corbelli and Anya Suneson
December 2014
Powering forward. Together.
Home: Sacramento Municipal Utility District
Electric Utility
Located in Northern California
Have about 610,000 Customers
– 540,000 Residential
– 70,000 Commercial
Covers territory:
• Sacramento County
• Part of Placer County
Why an electric utility does customer
research?
• Monitor existing programs
• Develop new programs
• Improve customer experience
Traditional methods:
• Telephone, mail, online surveys
• Focus groups
Now venturing out into new
methods…
Travel Bucket List
“Bucket list” of Innovative Research Methods
 Ethnography
 Usability testing
 Online Research Community
 Discussion boards
 Text analytics
The “Vacation Home” – Online Research
Communities
• Spent lots of time researching and planning
• Big investment
• Use regularly
• Requires regular maintenance
• Lots of people want to use
SMUD’s Online Research Community
The “Time Share” – Discussion Boards
• Use occasionally
• Short term use
• No long term commitment
• Great way to try something different!
Case Study:
Analysis of Online Discussion Board Responses
Research Objectives and Methodology
• Objective: Determine customer energy-related needs
and ways SMUD can address them
• Data Collection Method: Online Discussion Boards
• Invitation sent out to 386 SMUD Plugged In participants
• Online discussion was open for 2 weeks
• Facilitated by SMUD research specialist
• About 100 respondents participated in discussion
Online Discussion Board Example
Positive Customer Feedback
I appreciate the opportunity to be able to
share my opinions and ideas with SMUD.
Since I consider SMUD
to be a highly ethical
company, I would like
to be a participant in
helping it flourish, in
any way I can.
Tons of love for SMUD. That is all =)
I really love that my opinion
matters to you.
I appreciate the opportunity to give feedback and voice my concerns,
as well as suggestions for future development of my energy provider.
Text Analysis Tools Utilized
•
•
•
•
Traditional Verbatim Coding
Word clouds
Semantic score
Sentiment Analysis
Text Analysis: Traditional Coding
• Pros: Human coder gets nuances machine can’t
• Cons: Labor- and time-intensive
SMUD meets all needs already
42%
Keep energy affordable, keep bill low
41%
Provide safe, reliable power
25%
Use more solar, renewable energy
23%
Offer discounts to disadvantaged groups
13%
Household energy efficiency, more ways…
Provide unbiased energy advise
10%
7%
0%
10%
20%
30%
40%
50%
The “Date Night” – Word Clouds
• Inexpensive
• Don’t require much planning or preparation
• Quick look
• Fun
Text Analysis: Word Clouds
• Pros: Quick, easy, free, visually-appealing
• Cons: Not sophisticated, word-level and not phrase-level
The “Day in Port” – Sentiment Analysis
• Heard “buzz” about SA
• Cost ranges from free to very expensive
• Quick look
• But soon realized need more time in this “destination”
What is Sentiment Analysis?
• Also called Opinion Mining
• Is a systematic analysis of online expressions. Focuses on
evaluating attitudes and opinions on a topic of interest using
machine learning techniques (Rombocas)
• Classifies words, text, documents according to the opinion,
emotion or sentiment that they express
• Determines:
– Subjective vs. Objective polarity
– Positive vs. Negative polarity
– Strength of the opinion
Benefits of Sentiment Analysis
• Can analyze large sets of data quickly
• Can use on many different sources of unstructured data
• Measures emotions which are important for marketing
• Data less influenced by researcher
• Less subjective than human coding
• Allows real-time analysis
Text Analysis: Sentiment Score
• Tool: Python NLTK
• Pros: Easy to use, fast, free
• Cons: Harder to explain, less actionable, further analysis
requires Python programming knowledge
Text Analysis: Semantic Analysis
• Tool: Semantria
• Pros: Sophisticated, can churn big volumes, combines sentiment score
and phrase-level categorization
• Cons: Costly, takes time to learn
Semantic Analysis
•
•
Facets are the most important ideas with their accompanying attributes. Facets
rely on Subject Verb Object (SVO) parsing to find trends even when there are
weak or no noun phrases in your text.
Affordable energy, reliable energy, fixed income, low income and solar power
were some of the facets identified in our text
What did we learn on our journey?
• Discussion boards were a great way to have more of a
conversation with our customers
• Text analysis methods can help save time and offer
new insights
• Sentiment analysis appears to be useful for finding
themes and important ideas in verbatims – especially
for certain types of questions
• Need more time to better understand and utilize
Sentiment Analysis and available tools
– May need to learn a new language?
– Maybe hire a guide?
– Plan to visit again!
Questions?
Thank you!
Resources
A Sentimental Education: Sentiment Analysis Using Subjectivity Summarization Based on
Minimum Cuts Bo Pang and Lillian Lee
http://www.cs.cornell.edu/home/llee/papers/cutsent.pdf
LingPipe Sentiment Tutorial
http://alias-i.com/lingpipe/demos/tutorial/sentiment/read-me.html
Professor Dan Jurafsky & Chris Manning are offering a free online course on Natural
Language Processing.
http://www.nlp-class.org/
https://www.coursera.org/course/textanalytics
Resources
•
http://www.online-utility.org/text/analyzer.jsp
Free software utility which allows you to find the most frequent phrases and frequencies of words. Non-English
language texts are supported. It also counts number of words, characters, sentences and syllables. Also calculates
lexical density.
•
Text Mining package in R: http://cran.r-project.org/web/packages/tm/vignettes/tm.pdf
•
Data Mining (including Text) Examples: http://cran.rproject.org/doc/contrib/Zhao_R_and_data_mining.pdf
•
MAXQDA, free 30-day trial available at http://www.maxqda.com/downloads/demo
•
UCINET, free 30-day trial available at http://www.analytictech.com/downloaduc6.htm
Resources - Word Cloud Tools
•
www.wordle.net
TIP: Double click on word to remove, can use “~” to combine words, customize color
•
www.tagcrowd.com
TIP: Does stemming
•
www.tagxedo.com
•
TIP: Does stemming, many shapes
•
https://tagul.com/
Free Sentiment Analysis Tools
1. Twitrratr – www.twitrratr.com
2. Sentiment 140 - http://www.sentiment140.com
3. Tweetfeel – www.tweetfeel.com
4. Opinmind – www.opinmind.com
5. Social Mention – www.socialmention.com