Transcript Semantic Information Processing of Spoken Language

Semantic Information
Processing
of Spoken Language
- How May I Help You?
Alicia Abella
AT&T Labs – Research
Florham Park, New Jersey
sm
Motivation

Goal is to provide automated customer
services via natural spoken dialog.

Natural means what people actually say,
rather than what we’d like them to.

Shift burden from user to machine
Why Spoken Language Understanding?

User interface as the bottleneck to exploiting
speech and language processing technological
advancement

Spoken language as focus of this work

Machine Initiative

•
•
Menus (please say collect, calling card …)
Classes (please say credit card number, destination
city, person name, date, …)
User Initiative
•
How may I help you?
Stroustrup on programs
which communicate with people
"... it must cope with that person's whims,
conventions and seemingly random errors.
Trying to force the person to behave in a
manner more suitable for the machine is
often (rightly) considered offensive.”
from "The C++ Programming Language”(1987) pp. 76
A History of Applications

Department Store Call-Routing (1989-1991)

Almanac Data Retrieval (1992)

Airline Travel (1993)

Multimodal Blocks World (1993)

Operator Services (1995-9)

Customer Care (2000+)

Enterprise Customers (2002+)
How May I Help You?




SM
Prompt is “AT&T. How may I help you?”
User responds with unconstrained fluent speech
System recognizes and determines the meaning of users’
speech, then routes the call
Dialog technology enables task completion
HMIHY
Account
Balance
Calling
Plans
Local
Unrecognized
Number
...
Extracting Meaning from Speech



Extracting meaning is primary in speech
understanding systems.
How to quantify the information content
of a natural language message?
Such theory is crucial to engineering
devices which understand and act upon
such messages.
Communication Paradigm

Goal of communication is to induce the machine
to
• perform some action
• undergo some internal transformation

Communication is successful if the machine
responds appropriately

Contrast with traditional communication theory
Shannon (1948)
The fundamental problem of
communication is that of reproducing at
one point either exactly or approximately
a message selected at another point.
Frequently the messages have meaning,
… These semantic aspects of
communication are irrelevant to the
engineering problem.
Architecture for Natural Spoken Dialog
Voice reply to customer
Speech
Text-to-Speech
Synthesis
TTS
ASR
Automatic Speech
Recognition
Data
Words to be
Words
synthesized
Spoken Language
Generation
Speech
Words spoken
Words
SLG
Action
SLU
DM
Dialogue
Management
Meaning
Meaning
Spoken Language
Understanding
Architecture for
Natural Spoken Dialog
Play
prompt
User
speech
Acoustic
Models
Spoken
Language
Understanding
ASR
Language
Models
Salient
Grammar
Fragments
Dialog
Manager
Inheritance
Hierarchy
Technology Component Traits

Robustness
•
ASR
•
SLU
• Large vocabulary > 10,000 words
• Dialects (Nationwide deployment)
• Real-time
• Tolerance of varied phraseology
•
•
•
Many ways of saying the same thing
Similar way of saying different things
• Real-time
Dialog
• Confirmation, Re-prompting, Context switching
• Say anything, anytime, anyway
Examples of Customer Utterances
Account Balance
Other
General Billing
Rates and Calling Plans
Charge on Bill
Change Customer Info
Example Dialogs






Rate Plan
Account Balance
Local Service
Unrecognized Number
Threshold Billing
Billing Credit
The technology is in use today

AT&T Customer Care organizations
•
•
Consumer service live since Nov. 2000
• Decreased servicing costs; reduced time spent in
automation; reduced repeat calls and customer defections
Small Business service pilot underway
• Supports 800#s used for billing inquiries and corporate
•

calling card transactions.
Determines caller intent to perform activities such as
making payments, requesting bill adjustments, ordering
cards, reporting stolen cards
AT&T Enterprise customers
•
•
Several in beta trials
Delivers this functionality in a networked, managed
environment.
Industry Specific Applications
Check account balances,
Apply for mortgage,
Request credit report,
Locate branch
Benefit enrollment, Get
a referral, Obtain test
results, Pre-admissions
procedures
Get instructions, Report
a problem, Obtain
problem status, Order
Financial Insurance
Services
Healthcare
Tech
Retail
Travel
Verify coverage,
Inquire about a claim,
Check claim status
Get store hours, Locate
nearest store, Directions,
Check inventory availability
and order status
Obtain a price quote, Make
reservations, Get a seat
assignment, Check flight
status, Redeem miles
And Applications Common to All Industries:
Password Reset, PIN Reset, FAQs,
Help Desk, Locator Services, Order Entry and Status
Key Value Determinants
Enhanced Customer
Experience
Decrease Costs
•Reduced wait and call times
even at peak times
•Natural efficient and
personalized dialog
•Properly fulfills/routes
request the first time
•Increased use of automation
reduces servicing costs
•Liberate agent headcount
•Reduce handling time and
hang-ups and call-backs
Enhanced Business
Results
•Implement new applications
to drive revenue
•No capital investment to
build or support
•Dynamic, customizable
resource sharing in secure
environment
Observed Benefits

Increased automation

Improved customer satisfaction
• Improved routing
• Ability to add more functionality
• Decreased repeat calls (37%)
• Decreased customer defection rate (18%)
• Decreased rep time per call (10%)
• Decreased customer complaints (78%)
Research References at
www.research.att.com/~algor/hmihy
e.g. IEEE Computer Magazine, April 2002