Transcript Low-Cost Multi-Touch Sensing through Frustrated Total

Low-Cost Multi-Touch
Sensing through Frustrated
Total
Internal Reflection
Jefferson Y. Han,
New York University
Presented by: Cody Boisclair
Introduction



Touch sensitivity is fairly common in
electronics today…
…but only for a single point of contact at
any given time.
Touch-sensitive devices that allow for
multiple points of contact have thus far
been expensive or difficult to produce.
Why multi-touch?

Allows a user to interact with a system
with more than one finger at once



Chording
Operations using both hands
Also allows multiple users to interact on
the same touch-sensitive platform


Interactive walls
Interactive tabletops
Prior approaches to the
problem

Matrix of smaller sensors



Requires many connections, which
severely limits resolution
Visually opaque - no touch-screen
Video cameras



Measuring of brightness
Stereo cameras
Identifying tracking markers in gel
A new approach…
F rustrated
T otal
I nternal
R eflection
Total Internal Reflection


When light encounters a medium with a
lower index of refraction (e.g., going
from glass to air), its refraction depends
on the angle at which it hits the border.
Beyond a certain critical angle, light is
not refracted, but instead reflects
entirely within the material.

This is the basis for fiber optics and other
optical wave guides.
Frustrated Total Internal
Reflection


If another material touches that within
which the light is reflecting, the reflection
is frustrated, causing the light to escape.
This has been used in the past:



fingerprint imaging
early touch sensors (1970s!)
tactile sensors for robotic grips
A Schematic of FTIR
(Kasday, 1984)
Using FTIR for touch
sensitivity

A clear acrylic sheet is used as the
touch surface.




16 inches x 12 inches in prototype
Edges of surface lit by infra-red LEDs to
produce total internal reflection.
A video camera is mounted under the
surface and facing it.
When the surface is touched, the light
escapes and registers on the camera.
Using FTIR for touch
sensitivity



Basic image-processing techniques are
performed on the camera output to
identify the points of contact.
Computer-vision techniques are used to
interpret the motion of contact points as
discrete touches or strokes.
Processing easily handled in real-time
by a 2 GHz Pentium IV processor
Advantages to this approach

High capture rate and resolution






30 frames per second
640x480
True zero-force touch sensitivity
Inexpensive to construct
Scalable to much larger (even wallsized!) surfaces
Transparent: can be combined with rearprojection display
How is projection achieved?
Disadvantages


Requires significant space behind touch
surface for camera
Gloves, certain types of styluses, and
even dry skin may not register


a function of refractive index
Residues on surface (e.g., sweat) also
produce FTIR effect that may build up
Some applications…
Video Clip at NYU (Local Copy)


Uses a 36x27-inch rear-projection
screen.
Touch information is sent to programs
using OSC (Open Sound Control)
protocol
Any Questions?