Transcript Nick Gentile

Introduction to Neural Networks
and Example Applications in HCI
Nick Gentile
Overview
• Introduction to neural networks
• Example Applications in HCI
–
–
–
–
NETtalk
Cognitive Text-editing
Filtering Messages
Learning a User’s Interests by
Observing their behavior
• Discussion
Neural Networks
• The brain
– Cerebral cortex
• Memory, perceptual awareness, thinking, language and
consciousness
• Made up of billions of neurons
• Motivation
– “Understanding human behavior and brain
construction…”
• Computers have the power
– Using the power of computers we can better
understand the underlying processes of the brain
Neural Networks
• Three key elements
– Network architecture
• Typically described in terms of layers
• Connectionist model (connection strengths)
– “Each connection has associated with it a numerical
weight. Each neuron's output is a single numerical activity
which is computed as a monotonic function of the sum of
the products of the activity of the input neurons with their
corresponding connection weights.“
– Learning algorithm
• How the weights are set
– Data representation
• Inputs and outputs
Neural Networks
• Some useful terms
– Feedforward network - “A layered network in which each
layer only receives inputs from previous layers.”
– Target vector - “The desired output vector for a given input
vector.”
– Pattern recognition - “The task performed by a network
trained to respond when an input vector close to a learned
vector is presented. The network “recognizes” the input as
one of the original target vectors.”
– Error vector - “The difference between a network’s output
vector in response to an input vector and an associated
target output vector.”
NETtalk
• Training
– Sequences and pronunciations
• Gives good representation, but not good enough
• Further training handles exceptions and special cases
• How it works
– Seven letter window
• Looks at middle letter and uses rest as
“[context]” to determine output (phonetic symbols)
• Window moves through whole document generating a
sequence of phonetic symbols
NETtalk
• Results
– Network learned to abstract exceptions
and special cases to produce more
accurate speech
– Eventually was able to produce accurate
speech from words it had never seen
before
NETtalk
• Architecture
– Three-layered feed-forward network
• Representation
– Inputs - Sequence of letters
– Outputs - Sounds
• Learning
– Backpropagation
Cognitive Text-editing
• Goal
– Predict text editing strategies
• How
– By taking into account keystrokes and the pauses
between them
– “Robertson and Black have shown that the pauses
which occur during text editing are indicators of
the formulation of planning strategies.”
Cognitive Text-editing
• Training
– Subjects asked to write a memo using the ‘vi-editor’
– Inputs - 36 vi commands and three types of pauses (long,
short and intermediate)
– Outputs - Editing goals (Address Memo, Puncture Memo,
Organise Memo, Enhance Memo, Review Memo and Error
Correction). Manually input for training purposes.
• Outcome
– 36 memos written, 12 used for training 24 used for testing
– Network was able to recognize 96% of the test data
Cognitive Text-editing
• Architecture
– Feed-forward network
• Representation
– Inputs - vi commands and pauses between
keystrokes
– Outputs - Editing goals
• Learning
– Backpropagation
Filtering Messages
• The problem
– Eliminate irrelevant messages from newsgroup
• The solution
– Generate two dictionaries
• “Common Dictionary”
• “Deference Dictionary”
– Construct a neural net
• # input nodes = # of words the dictionary
• # output nodes = # of subtopics or categories
Filtering Messages
Message - Hi Nick what’s up
Dictionary
hi
nick
s
up
what
do
hci
hi
network
nick
what
Input vector - [0.1, 0.1, 0.9, 0.1, 0.9, 0.9]
Filtering Messages
• Architecture
– Three-layered, feed-forward network
• Representation
– Input - email and newsgroup messages
– Output - relevant messages
• Learning
– Backpropagation
Learning a User’s Interests by
Observing their behavior
• Task
– To predict weather or not a particular page is of
interest, based on the user’s profile
• Methodology
– Obtain training examples by recording user
navigation behavior
– Use this information to predict user interest in a
page
Learning a User’s Interests by
Observing their behavior
• Implementation
– Three output neurons
• # of hyperlinks clicked
– Fraction of hyperlinks clicked on a page
• Scrolling activity and mouse activity
– Counts scaled by 100
• Result
– Based on page content
• # of hyperlinks user will click
• Amount of scrolling
• Amount of mouse activity
Learning a User’s Interests by
Observing their behavior
• Architecture
– Feed-forward network
• Representation
– Input - IE logfile
– Output - User’s interest in a page
• Learning
– Backpropagation
Final Thought
• One of the main goals of HCI is to model user
behavior in order to gain a better understanding of
how they interact with computers. They can then
take that understanding and apply it to new and
existing applications to make them more usable.
So what better way to understand human behavior
than to exploit the very mechanism that guides
that behavior?
References
•
•
•
Anderson, J. A. (1995), An Introduction to Neural Networks, Cambridge,
MA: MIT Press.
Yasdi, R. (2000). "A Literature Survey on Applications of Neural
Networks for Human-Computer Interaction." Neural Computing &
Applications 9(4): 245-258.
Cool site to learn about and play with various types of networks.
http://diwww.epfl.ch/mantra/tutorial/english/