Transcript Slide 1

Haptic Interface
April 2006
Prof. Ed Colgate
Northwestern University
Evanston, IL USA
Today’s Class


Course overview
Introduction to Haptics







Haptics overview
History
Applications/Motivations
How to design effective haptic interfaces
Current challenges
Break
Psychophysics
© J. Edward Colgate, 2006
Chicago
Northwestern University
© J. Edward Colgate, 2006
Northwestern University





Founded in 1851 in Evanston, IL
Two campuses today: Evanston and Chicago
~8000 undergraduates and ~6000 graduates in 9
schools
~1400 undergraduates and ~700 graduates in the
McCormick School of Engineering and Applied
Science
9 Departments in McCormick

Applied Math; Biomedical; Chemical; Civil &
Environmental; Computer; Electrical; Industrial; Materials
Science; Mechanical
© J. Edward Colgate, 2006
Northwestern Scenes
© J. Edward Colgate, 2006
Northwestern>Dept of
Mechanical Engineering>LIMS
Prof. Kevin Lynch
Prof. Mitra Hartmann
Prof. Michael Peshkin
Not shown: Prof. Malcolm
MacIver
© J. Edward Colgate, 2006
LIMS Research

Human-Robot Interaction
 Haptic
(touch) interface
 Assistive robots
Prototype variable friction haptic
display

Robot Motion Planning
 Underactuated

Developed by John Glassmire
systems
Biologically-Inspired Robotics
Robotic Ribbon Fin
 Robotic
fish
 Active sensing
Developed by Michael Epstein
© J. Edward Colgate, 2006
Goals of this course

Gain familiarity with key ideas in haptics








Haptic perception
Psychophysics
Design and control of haptic interfaces
Passivity, Z-width
Haptic Rendering
Hands-on experience with haptics
Gain some familiarity with current research and
literature
Identify opportunities for research in haptics
© J. Edward Colgate, 2006
Grading

30% Class participation/contribution



40% Homework


Ask questions!
Offer opinions, insights, etc.
3 assignments, due Wed, Thu, Fri
30% Paper presentations


Each student gives a 15 minute presentation of a
paper
All papers are from 2006 Haptics Symposium
© J. Edward Colgate, 2006
Website
http://othello.mech.northwestern.edu/~colgate/UPC/
© J. Edward Colgate, 2006
hap·tic ('hap-tik)
adj.
Of or relating to the sense of touch; tactile.
[Greek haptikos, from haptesthai, to grasp, touch. (1890)]
Location/configuration
Motion
Force
Compliance
Cutaneous
Kinesthesia
Temperature
Texture
Slip
Vibration
Force
© J. Edward Colgate, 2006
Our focus: programmable
haptic interfaces

Mainly: kinesthetic
interface to virtual
environments
Phantom - kinesthetic

Also: tactile interface to
virtual environments
Pin Array – low
frequency cutaneous
© J. Edward Colgate, 2006
More generally: human-robot
interaction







Telemanipulation
Exoskeletons
Physical Rehabilitation and Exercise
machines
Intelligent Assist Devices (IADs)
Advanced prosthetics
“Near-field” telerobotics
Human-robot-human
© J. Edward Colgate, 2006
A Little History

Ray Goertz, Argonne National Lab, 1940s
© J. Edward Colgate, 2006
Computer simulation replaces
the slave manipulator





Fred Brooks, UNC Chapel
Hill, 1970s
Developed to study
molecular docking
User feels interaction force
between molecules
Master was one of Goertz’s
Didn’t work very well…
© J. Edward Colgate, 2006
~1990 – Haptic Interface Emerges
as an Engineering Discipline


Margaret Minsky’s “virtual sandpaper” system
developed at the MIT Media Lab
Dov Adelstein’s force reflecting joystick
developed in the MIT Biomechanics Lab
Force
Minsky, 1990
© J. Edward Colgate, 2006
LIMS has been involved since
the early days (~1991)

Paul Millman’s 4DOF
Haptic Interface


Originally developed with
telemanipulation in mind
Never got around to
developing the slave!
© J. Edward Colgate, 2006
LIMS continues to be active in
haptics
© J. Edward Colgate, 2006
Three-rotational virtual spring
and damper
© J. Edward Colgate, 2006
Ball in a box
© J. Edward Colgate, 2006
Haptics has many applications

Blind Persons






Programmable Braille
Access to GUIs
Training





Medical Procedures
Astronauts


Assembly-Disassembly
Human Factors



BMW “iDrive”
Haptic Touchscreens
Mobile Phones


Arcade (steering wheels)
Home (game controllers)
Automotive

Education
Computer-Aided Design

Entertainment
Immersion “Vibetonz”
Animation/Modeling
Art
Material Handling

Virtual Surfaces
© J. Edward Colgate, 2006
Training

Visual display alone is not sufficient for
certain types of virtual environments. To learn
physical skills, such as using complicated
hand tools, haptic information is a
requirement
Applications
© J. Edward Colgate, 2006
Virtual Prototyping
Applications
McNeeley et al. (Boeing
Corp.)
© J. Edward
Colgate, 2006
Rehabilitation
Applications
© J. Edward Colgate, 2006
Teleoperation
Applications
© J. Edward Colgate, 2006
Computer interface for blind
users



Text-based computers can easily be enhanced to
include a speech synthesizer
Graphical user interfaces are inherently visual
A haptic display can help a blind computer user
interact with graphics-based operating systems
© J. Edward Colgate, 2006
Entertainment
Applications
© J. Edward Colgate, 2006
Underlying motivations for
haptics

Looking across applications, we find common
motivations:

Haptics is required to solve the problem




Haptics improves realism and sense of immersion



Interfaces for the blind
Phlebotomy training (task is mainly “feel”)
Vibetonz – a private communication channel
Entertainment
Animation/modeling
Haptics provides constraint


Assembly/disassembly
Virtual surfaces
© J. Edward Colgate, 2006
How to design effective haptic
interfaces
A simple three-step program…

A.
B.
C.
Understand how the human sensory and
perceptual systems work
Use this information to develop performance
metrics
Understand how to build/control machines that
display haptic percepts and meet performance
metrics
© J. Edward Colgate, 2006
A. How haptic sensing works

Let’s see it in action…
© J. Edward Colgate, 2006
Some terminology




“Haptic” refers to the perceptual system that draws
information from the skin and kinesthesis
“Proprioception” is the unconscious perception of
movement and spatial orientation arising from
stimuli within the body itself
“Kinesthesia” is the sense that detects bodily
position, weight, or movement of the muscles,
tendons, and joints
“Tactual Stereognosis” is the perception of the form
of an object by means of touch
© J. Edward Colgate, 2006
Sensors that contribute to
haptic perception




4 types of
mechanoreceptors
2 types of
thermoreceptors
2 types of nociceptors
(free nerve endings
for pain)
3 types of kinesthetic
receptors
© J. Edward Colgate, 2006
Mechanoreceptors
Mechanoreceptors
differ according to:
-frequency response
-receptive field
-location
Ruffini Endings
Merkel’s
Meissner’s
Pacinian
Disk
Corpuscle
Corpuscle
(FA II)
(SA
I)
I)
II)
• 0.4
2 Hz
40
Hz
Hz
- -200
-800
100
Hz
Hz
Hz
• 59
11 mm222receptive
13
101mm
receptivefield
field
• deep
shallow
••Curvature,
flutterstretch,
vibration
skin
vibration;
shape,
force
tickle;
pressure
texture?
© J. Edward Colgate, 2006
The skin is an important
organ!


Large: approximately 2 m2
Abundant sensors:

~500,000 mechanoreceptors spread across the
body

~17,000 in the glabrous (non-hairy) skin of the hand
© J. Edward Colgate, 2006
Sensory Homunculus
© J. Edward Colgate, 2006
Kinesthetic receptors



Muscle Spindles
provide muscle length
and velocity information
Golgi Tendon Organs
provide tension
information
Joint Afferents provide
joint angle and angular
velocity information


Ruffini endings and
Pacinian corpuscles
located in joint capsule
Note that people with
artificial joints have almost
normal sense of joint
position
© J. Edward Colgate, 2006
Bilateral Nature of Kinesthetic
Sensing
Human
Hand/Arm
effort
flow
Environment
effort  flow = Power


Unlike vision & audition, kinesthetic sensing is twoway
There is also the prospect for significant power
exchange with the environment as part of a haptic
interaction
© J. Edward Colgate, 2006
Conclusions: how haptic
sensing works

A vast number of sensors in both the skin and
musculoskeletal system work in conjunction
with the motor control system to enable
sensing of mechanical stimuli



Haptic sensing is bilateral
Perception clearly involves the CNS as well
as the peripheral nervous system, but that is
the subject of another lecture…
We’ve just barely scratched the surface!
© J. Edward Colgate, 2006
B. Performance metrics for
haptics

Performance can be assessed at various
levels:




Peripheral sensors
From sensors to CNS
Perceptual
Functional
© J. Edward Colgate, 2006
Pressure thresholds


Weinstein, 1968
Pressure
measured with
precisely
calibrated nylon
filaments pressed
into skin
© J. Edward Colgate, 2006
Point localization thresholds



Weinstein, 1968
Distance between
body point
stimulated and
subject’s
impression of
where stimulation
took place
Two-point
discrimination data
are similar
© J. Edward Colgate, 2006
Frequency response
thresholds

Bolanowski, 1988
© J. Edward Colgate, 2006
Just Noticeable Differences



DI is the increment in intensity that, when
added to stimulus intensity I, produces a just
noticeable difference.
DI/I = k is the “Weber fraction”
Weber hypothesized that k would remain
constant across all values of I for a given
modality.

Not true, but often a reasonable approximation
© J. Edward Colgate, 2006
JNDs
Vision (brightness, white light)
1.5%
Audition (middle pitch & moderate loudness)
10%
Smell (odor of India rubber)
25%
Taste (table salt)
33%
Kinesthesis (lifted weights)
2%
Pressure (cutaneous pressure “spot”)
14%
Length
10% or less
Velocity
10%
Acceleration
20%
Force on skin
7%
Compliance
23%
Viscosity
34%
Sources: Biggs and Srinivasan “Haptic Interfaces”
Schiffman “Sensation and Perception”
© J. Edward Colgate, 2006
Perceptual Measures


Channel capacity in bits/sec
Max information flow at receptor level:




~107 bits/sec for eye
~106 bits/sec for hand
~105 bits/sec for ear
Post-processing rate for tactile information is
~2-56 bits/sec

Compare to ~40 bits/sec for speech and ~30
bit/sec for reading Kaczmarek, K.A., and P. Bach-y-Rita, “Tactile
Displays”, in W. Barfield, and T.A. Furness (Eds.), Virtual
Environments and Advanced Interface
© J.Design,
EdwardOxford
Colgate, 2006
University Press, New York, 1995, pp. 349-414.
Information Transmission
Rates
Reading
~ 30 bits/sec
Kinesthetic Morse Code
2.7 bits/sec
Tadoma
12-14 bits/sec
Optacon (vibrotactile)
5.4 bits/sec (40 wpm)
Tadoma
Optacon
© J. Edward Colgate, 2006
Example of a Functional
Measure

Object identification via tactual stereognosis



Klatzky, Lederman and Metzger 1985
96% correct identification of 100 common objects
94% within first 5 seconds; 68% within first 3
seconds
© J. Edward Colgate, 2006
Conclusions: haptic
performance

High bandwidth



Extraordinary sensitivity to certain stimuli



Better for signals (e.g. force) than impedances (e.g. compliance)
Haptics does not excel as a high bandwidth channel for
structured information (characters, words, text)


300 Hz vibration (.1 mm), raised edges (<1 mm)
Huge dynamic range: forces from ~0.5N to ~500N
JNDs generally consistent with other senses


Temporal to 1 kHz
Spatial discrimination to 1 mm
But Tadoma illustrates the power of a highly parallel approach
Haptics does excel at 3D object recognition
© J. Edward Colgate, 2006
Displaying haptic percepts




Ground-based devices
Body-based devices
Inertial reaction devices
Tactile displays
© J. Edward Colgate, 2006
Ground-based devices




Phantom
Haptic Master
Cobot Hand Controller
List goes on…
© J. Edward Colgate, 2006
Body-based devices



Cybergrasp
Rutgers Hand Master
Again, many others
© J. Edward Colgate, 2006
Inertial Reaction Devices

Game controllers
motors
shafts
eccentric masses
© J. Edward Colgate, 2006
Tactile Displays




Pin arrays (Optacon, Harvard displays, many
others)
Lateral stretch
Electrocutaneous displays
Haptic field displays
© J. Edward Colgate, 2006
Some current challenges in
haptics

Low power, embedded haptics


Exploiting tactual stereognosis (e.g., for
automobile instrument panels)


Exoskeletal devices haven’t been the answer
Haptics over the internet (e.g., for
telesurgery)


Mobile electronics
Latencies are a big issue
Haptic feedback for amputees
© J. Edward Colgate, 2006
Haptics for prosthetics

“Sensory reinnervation” provides a possible
means for restoring the sense of touch to
amputees
Haptic
display
Kinesthetic
and tactile
sensors
© J. Edward Colgate, 2006