Transcript Diapositiva 1

University of Siena
Robotics for Rehabilitation and
Medical Applications
Domenico Prattichizzo
Robotics & Systems Lab
University of Siena
www.dii.unisi.it/prattichizzo
[email protected]
Centro per la ricerca scientifica e tecnologica
ITC - IRST Trento, December 2003
0
University of Siena
Outline
Visuo-Haptic Interaction
1) Grasp Rehabilitation
2) the FeTouch Project
Eye Tracking
3) ASTIDET
1
University of Siena
Haptic Interaction ?
2
University of Siena
Touch is very important in medicine
HAPTIC interfaces
from Greek haptesthai : to
touch
Increasing telepresence
through video-haptic
interfaces
3
University of Siena
Anthromorphic Robot Arm (kinematics)
• sensors to measure end-tip position
• actuators to apply end-tip force
4
University of Siena
Haptic Interface Principle
• measure finger positions• compare with the virtual obj. (collision detectio
• get finger position
• if the hand-avatar is inside, apply a force …
5
University of Siena
Ungrounded Haptic Interfaces
joint angles are
measured and
forces applied
relative to mount
point on the
body
Cybergrasp by Immersion inc.
6
University of Siena
Visio-Haptic Interaction
bidirectional flow
Haptic Interface
VR
Visual Interface
7
University of Siena
Outline
Visuo-Haptic Interaction
1) Grasp Rehabilitation
2) the FeTouch Project
Eye Tracking
3) ASTIDET
8
University of Siena
Grasp Rehabilitation
University of Siena & University Vita-Salute San-Raffaele
Other partners
Service de Reeducation, Hopitaux Universitaires, Switzerland
Institut National de la Sante et de la Recherche Medicale, France
Institut Guttmann, Spain
FCS Control Systems B.V, Netherlands
9
University of Siena
Grasp rehabilitation goals
1) Rehabilitation units equipped with robotic systems helping
patients with motor deficits to relearn basic manual skills
such as to open doors, use forks and knives, drink a glass of
water, or button a shirt.
2) Virtual Reality Environments that allow pre-surgery
patients to face the expected deficits after a surgical
intervention, by mimicking its effects on fine manual skills.
10
University of Siena
Robotic upper-limb rehabilitation
HapticMasterTM with the Endo-Ring (project GENTLE/S).
11
University of Siena
Upper-limb rehabilitation @ MIT
• MIT-MANUS (1990s) - The robot provided guidance, assistance or
resistance while the patient executed a pointing movement.
Robot-aided patients improved more than the control patients.
The gains obtained with this technique remain significant after 3 years.
{{{{{{{{{{{{{{{{{{{{{{
Burke Hospital-MIT group
Palo Alto Veteran's Affairs Hospital / Stanford group
Spaulding Rehabilitation Hospital (1999)
Baltimore VA Hospital (2000)
12
University of Siena
Upper-limb rehabilitation in Europe
• MULOS
Motorised Upper Limb Orthotic System is a 5 dofs robot
mounted on a wheel chair that can move the joints of the upper
limb under joystick control.
• REHAROB
This project used industrial robots to passively manipulate the
arm of patients.
• GENTLE project
…
13
University of Siena
Robotic rehabilitation @ Siena
Grasp rehabilitation
Simpler robotic devices
14
University of Siena
Robotic rehabilitation @ Siena/Milan
The Grip Analyzer. The Grip Analyzer
is
essentially
a
real
object
instrumented with several six degreeof-freedom (DOF) transducers for
measuring contact forces.
15
University of Siena
Robotic rehabilitation @ Reachin
The Virtual Grasping Environment. Simulation of a double
point contact interaction using two degrees of freedom devices.
ReachinTM setup
16
University of Siena
Robotic rehabilitation @ Siena
Multi finger haptic interaction.
Stability analysis
Performance evaluation
Deformable objects
Stabilizing a multi contact haptic intercation:
Problem overview
17
University of Siena
References
D.Prattichizzo, J.K. Salisbury, and A. Bicchi, “Contact and grasp robustness
measures: Analysis and experiments.’’ in Experimental Robotics IV}, Lecture
Notes in Control and Information Sciences, pages 83--90. Springer-Verlag,
1997.
A.Bicchi and D.Prattichizzo, in \Analysis and optimization of tendinous actuation
for biomorphically designed robotic systems\ in Robotica 18:23--31, 2000.
Workshop ``Multi-point interaction: from robotic hands to haptic interfaces'‘
organized by F. Barbagli, D. Prattichizzo and K. Salisbury at the IEEE
International Conference on Robotics and Automation}, 2004, (submitted). To
be published on a Springer Verlag Serie.
F. Barbagli, D. Prattichizzo, J.K. Salisbury, ``Multi-contact haptic
interaction with deformable objects: amulti-rate approach'' in ISRR
2003, the 11th International Symposium of Robotics Research, October
19-22, 2003, Siena, Italy. To be published on a Springer Verlag Serie.
18
University of Siena
Outline
Visuo-Haptic Interaction
1) Grasp Rehabilitation
2) the FeTouch Project
Eye Tracking
3) ASTIDET
19
University of Siena
The FeTouch project @ Siena
Robotics & Systems Lab
Dip. di Pediatria, Ostetricia e Medicina della Riproduzione
University of Siena
www.fetouch.org
University of Siena
Medical Applications
• training & simulation, for:
– diagnosis
– tissue palpation
• Minimally Invasive Surgery (MIS)
– training through simulation
– the real thing: “fly by wire” to improve interaction
• telesurgery
demonstrated on a human in 2002
(at the EITS in Strasburg from New York)
21
University of Siena
Confluence of Ultrasound and Haptic
Interfaces
real patient data
Haptic Interface (1KHz)
(tactile rendering)
Siemens Sonoline Elegra
Dipartimento di Pediatria,
Ostetricia e Medicina della
Riproduzione.
Vision (25fps)
Isosurface
22
University of Siena
The FeTouch System
23
University of Siena
Volume and Surface Rendering
Direct Volume Rendering
(Ray-Casting)
NEW !
(geometric)
Isosuphase Extraction
(Marching Cubes)
Marching Cubes
(triangles)
24
University of Siena
Segmentation and Surface Extraction
Ultrasound Image
Segmentation
Marching Cubes
(triangles)
25
University of Siena
Haptic Principle
1. [Avatar] Measure hand position (encoders)
2. [Collision Detection] Localize the avatar in VR
3. [Force Rendering] Apply force to the operator(motor)
4. [Visual Feedback] Modify VR (deformable objects)
F
FR
Problems
 fast force feedback loop
 1KHz
 slow deformable object dynamics …
 Internal organ modeling
26
University of Siena
Proxy and force rendering
27
University of Siena
US3D & US3DTouch Session
US3D:
US3DTouch:
Fetus face (baby.vtk)
VIDEO
Heartbeat
to know more
www.dii.unisi.it/fetouch
www.fetouch.org
28
University of Siena
Neonatology
in cooperation with
29
University of Siena
References
B. La Torre, D. Prattichizzo, F. Barbagli, and A. Vicino.``The fetouch
project, a tool for augmenting the fetus perception.\ in The Inter. Journal
of Medical Robotics and Compurer Assisted Surgery, Jan.2004.
B.~la~Torre, D.~Prattichizzo, F.~Barbagli, and A.~Vicino. “The fetouch
project.\ in Proc. IEEE Int. Conf. on Robotics and Automation, 2003.
B. La Torre, D. Prattichizzo, F. Barbagli, and A.Vicino, ``The fetouch
system: Visual-haptic rendering of fetuses.\ in Proc. Euro{Haptics 2003,
Dublin, Ireland, 6-9 July 2003.
30
University of Siena
Outline
Visuo-Haptic Interaction
1) Grasp Rehabilitation
2) the FeTouch Project
Eye Tracking
3) ASTIDET
31
University of Siena
ASTIDET, Advanced STImuli Design for Eye
Tracking Easy Applicability for Ocular
Movement Study in Neurology
Robotics & Systems Lab
Dep. of Neurological and Behavioural Sciences
University of Siena
University of Siena
Eye Tracking
PC
Eye movement data
33
University of Siena
Eye-Tracking: Setup
Eye Movements
Pan/Tilt Infrared camera (ASL)
observed by
controlled by
Pupil, CR, Line of gaze
Control Unit
34
University of Siena
Infrared Video-Based Eye-Tracker
•CCD camera-IR emitting diodes with
light source coaxial with optical axis
•Detection of light reflections from cornea
(CR) and bright pupil.
•Real time processing of eye image
sequences and extraction of pupil and
CR centroids.
•Line of gaze computed using pupil and
CR centroids.
35
University of Siena
ASTIDET
©
Stimulus generator
36
University of Siena
Experimental Results: Saccades Study
Tg=20cs Ta=180cs
12 °
37
University of Siena
vertical movement
horizontal movement
38
University of Siena
Eye tracking for medical
diagnosis
• Disease causes abnormal oscillations of the eyes
• Images of stationary objects move excessively
on the retina (blurring of vision and oscillopsia)
39
University of Siena
Ocular rehabilitation
Generated scene can change with respect to
the subject’s eye movements!
40
Scene Viewing-Fixation
41
University of Siena
References
A. Rufa, D. Prattichizzo, G.L. Mariottini, A. Vicino, and A. Federico,
``Advanced STImuli Design for Eye Tracking - ASTIDET: a software tool
for scene generation, data acquisition and analysis in ocular
movements study,'‘ in Proc. of the American Academy of Neurology
Conference, San Francisco, USA, April 2004,
A. Rufa, D. Prattichizzo, G.L Mariottini, A. Patrichi, A. Vicino, and A.
Federico, ``Advanced STImuli Design for Eye Tracking (ASTIDET):
easy applicability for ocular movement study in neurology,'' in Atti del
Simposio Italiano di Neurologia, Rome, Italy, October 2003.
42
University of Siena
Concluding
Visuo-Haptic Interaction
Grasp Rehabilitation
the FeTouch Project
Eye Tracking
ASTIDET
Task Driven Robotic Devices
Towards Simplicity (home tools)
Modularity
rslab.dii.unisi.it
www.dii.unisi.it/prattichizzo
43