Transcript 3DInteractionx

3D User Interfaces
Using the Kinect and Beyond
Expressive Intelligence Studio // Center for Games and Playable Media
http://eis.ucsc.edu // http://games.soe.ucsc.edu
John Murray
John Murray
Expressive
Intelligence Studio
Title
Here (Arial)
3D User Interfaces
Introduction to Interfaces
• User interface design (HCI) focuses on use
cases and user studies
• Have only begun investigating 3D interfaces
with the same level of rigor
• But have been actively researching them for
the past two decades
• Divided into input devices, manipulation
techniques, navigation techniques, and
Expressive Intelligence Studio
http://eis.ucsc.edu
3D User Interfaces
Input Devices
•
•
•
•
•
•
Joysticks
3D Mice
2D Mice
Buttons
Brain Input
Gloves
Expressive Intelligence Studio
http://eis.ucsc.edu
3D User Interfaces
Special Purpose Game Controllers
• Playstation Move
• Wiimote & Nunchuk
• Steel Battalion
controller
• Wii Fit
• 360 Controller
Expressive Intelligence Studio
http://eis.ucsc.edu
3D User Interfaces
Playstation Move
• Uses optical tracking to locate sphere
• Color distinguishes from environment
• Uses 6dof (accelerometer & gyroscope) to
provide rotation around point
• Up to 4 controllers
• Very fast
Expressive Intelligence Studio
http://eis.ucsc.edu
3D User Interfaces
Wiimote
• Began it all
• Uses IR LEDs in a tracker to handle exact
position, but must be pointed toward “sensor”
bar.
• Has accelerometer, but not gyroscope
• Extension adds additional resolution/detail to
rotation
Expressive Intelligence Studio
http://eis.ucsc.edu
3D User Interfaces
Steel Battalion
Expressive Intelligence Studio
http://eis.ucsc.edu
3D User Interfaces
New wave of input devices
•
•
•
•
Touche $?
Leap $70
Sparkfun $400+
Emotiv EPOC neuroheadset $300
Expressive Intelligence Studio
http://eis.ucsc.edu
3D User Interfaces
Output
•
•
•
•
Augmented reality (Foresight, Google Glass)
3D TVs and Monitors getting cheaper
Tactile (Vibration, haptic feedback)
Heat/cold
Expressive Intelligence Studio
http://eis.ucsc.edu
3D User Interfaces
3D MANIPULATION
Expressive Intelligence Studio
http://eis.ucsc.edu
3D User Interfaces
3D Selection
& Manipulation
• Nonstandard
manipulation
– Deform
– Puppet
– Meshes
Expressive Intelligence Studio
• 3D Manipulation
• “Spatial Rigid Object
Manipulation”
– Selection
– Positioning:
– Rotation: Changing
orientation of the object
http://eis.ucsc.edu
3D User Interfaces
• Precision Grasp
• Small muscle groups
(thumbs, fingers
• Thumbsticks on 360 and
wiimote
• Better performance in
3d, faster, more
accurate
• Reduce “clutching”
Expressive Intelligence Studio
• Attached to hand
(Kinect)
• Provides kinesthetic
sense
• Uses larger muscle
groups
• Exercise & imitates
actual actions
http://eis.ucsc.edu
3D User Interfaces
Isomorphic vs Isometric
• Think thumbpad vs mouse or Kinect.
• Measuring force vs position
• Relative vs absolute mapping of rotation and
transform
• Think its more natural, but isomorphic can be
limited by our own physical limitations.
• We do not have extendable arms, for instance.
Expressive Intelligence Studio
http://eis.ucsc.edu
3D User Interfaces
Selection Technique
• Indication of object
–
–
–
–
Occlusion
Object touching (list, voice set, automatic, iconic)
Pointing (2d, 3d gaze, 3d hand)
Indirect selection (1-1, pos->pos, vel->pos, pos->vel)
• Confirmation of selection (Event, gesture, voice
command, no explicit command)
• Feedback (Text/Symbolic,Aural, visual,
force/tacticle)
Expressive Intelligence Studio
http://eis.ucsc.edu
3D User Interfaces
Manipulation Techniques
• Exocentric Metaphors
– World in miniature
– Automatic scaling
• Egocentric Metaphors
– Virtual Hand Metaphors
• “Classical”
• Go-Go
• Indirect Go-Go
– Virtual Pointer Metaphors
•
•
•
•
Expressive Intelligence Studio
Ray Casting
Aperture
Flashlight
Image plane
http://eis.ucsc.edu
3D User Interfaces
TRAVEL AND WAYFINDING
Expressive Intelligence Studio
http://eis.ucsc.edu
3D User Interfaces
Maneuvering
• Similar to search task: destination known.
• Need to have precise local changes to reach
destination
• How far do you need to go?
• How many turns in path?
– Turning torse when less frequent, other method
when they are frequent
• Visibility of target from start location
Expressive Intelligence Studio
http://eis.ucsc.edu
3D User Interfaces
Travel Techniques
• Active vs Passive
– Active is when the user moves their viewport
consciously
– Passive is when the user directs travel
• Direction/Target Selection
– Gaze, pointing, orientation, Stop moving
• Velocity/Acceleration Selection
– Constant, gesture, explicit, adaptive
• Input Conditions
– Constant, continuous input, start and stop inputs,
automatic start/stop
Expressive Intelligence Studio
http://eis.ucsc.edu