Transcript Open Source Surgical Simulation

Open Source / Open
Architecture Software
Development Framework for
Surgical Simulation
M.C. Çavuşoğlu, T. Göktekin,
S.S. Sastry, F. Tendick, R. Fearing
Department of Electrical Engineering and
Computer Sciences
University of California, Berkeley
http://robotics.eecs.berkeley.edu/~mcenk/simulation/
PI Meeting – May 22-24, 2002 – Washington, DC
The Digital Human Project
Vision
Digital Human Project is an initiative which aims to build a
complete functioning library of interactive views and simulations
of human anatomy, physiology, pathology, histology and
genomics
Technical Objectives
Simulate all relevant physical scales, timescales, stages of
development
Allow collaborative, worldwide development and sharing by
many individuals and teams
Permit easy integration of research tools into education and
training
PI Meeting – May 22-24, 2002 – Washington, DC
The Digital Human Project
Applications
• Aid research by tying together large amounts of
information available about biological systems
•
•
•
•
gene expression
cell models
organ models
Eventually serve as an experimental platform to predict
the response of the human body to new therapies
• Improve Education and Training at All Levels
• Reduce the cost and error of medical training
• Provide a platform for medical certification and
accreditation
• New opportunities for biomimetics (including
computing, assembly)
PI Meeting – May 22-24, 2002 – Washington, DC
Project Focus
• Surgical simulation
– Training
– Preoperative planning
– Intraoperative assistance
• Organ level simulation
– Heterogeneous physical processes within the organs
– Multiple organ interactions
– Hierarchy of models
PI Meeting – May 22-24, 2002 – Washington, DC
National Security Relevance
• Tie BioSpice and other cell level modeling
and simulation efforts to organ and system
level modeling and simulation
– Study the organ and system level effects of
chemical and biological agents
• Simulators for Training of Medical Personnel
PI Meeting – May 22-24, 2002 – Washington, DC
Open Source / Open Architecture
Software Development for Organ
Simulation
PI Meeting – May 22-24, 2002 – Washington, DC
Need for an Open Source Framework
• Currently each research group independent: models
are neither interoperable nor independently verified
• Challenge of developing a useful simulation is too
great for one group
– Expertise required from moleculer and cell biology, anatomy,
physiology, computer science, system theory
– Multiple sources of experimental data
• Different applications have different requirements on
accuracy, spatio-temporal scale, and computational
requirements
• Reusable, Interoperable, Verifiable Models
PI Meeting – May 22-24, 2002 – Washington, DC
Technical Issues
• Abstraction
• Heterogeneous Physical Mechanism and Models of
Computation
• Customization with Patient Specific Models
• Verifiability
• Modularity through Encapsulation and Data Hiding
PI Meeting – May 22-24, 2002 – Washington, DC
Physiology
Electrical Activity
Mechanical Behavior
Blood Dynamics
Haptics
..…
Blood Vessels
Lungs
Test Bed: Heart Model for
Surgical Simulation
HEART MODEL
PI Meeting – May 22-24, 2002 – Washington, DC
Heart Model
Heart
Physiology
Cell Level
Physiological
Processes
Discrete
Event
Abstraction
Biochemical
Model
Mechanical
Behavior
Blood
Dynamics
Material
Transport
Lumped Model
Lumped
Model
Lumped
Transmission
Model
Finite Element
Model
Individual
Muscle
Fibers
Computational
Fluid Dyn.
Model
Fluid
Dynamics
Model
Blood Vessels
Mechanical
Behavior
Lumped
Model
Individual
Layers
Of Vessels
Material
Transport
Lumped
Model
Cell Level
Diffusion &
Transport
Heterogeneous Processes
PI Meeting – May 22-24, 2002 – Washington, DC
Level of Detail
Discrete
Event
Abstraction
Electrical
Activity
Blood
Simulator Architecture
Modeling Tools
NL Finite Element Models
Linear Finite Element Models
Mass-Spring-Damper Models
…
Models
Physiology of
Heart Beat
Regulation
Heart Tissue
Deformation
Computational Tools
Ordinary Differential Eq.
Differential-Algebraic Eq.
NL Algebraic Eq.
Discrete Event Systems
Hybrid…
Systems
Simulation
Kernel
Input/Output
Haptic
Interface
Visualization
Blood Dynamics
…
Collision Detection
…
Parameters
Support/Utility
Functions
PI Meeting – May 22-24, 2002 – Washington, DC
Organ Level Interconnections
PI Meeting – May 22-24, 2002 – Washington, DC
Organ Interface
• Regulatory Input / Output :
– Nervous system (neural signals)
– Blood circulation (hormones)
• Material Input / Output :
– Blood circulation
• Interaction with the External World
PI Meeting – May 22-24, 2002 – Washington, DC
Case Study: Heart Model
PI Meeting – May 22-24, 2002 – Washington, DC
Circulatory and Nervous Systems
Central Nervous System
Peripheral Nervous System
Sensory Somatic
Motor
Neurons
Autonomic
Sensory Motor
Neurons Neurons
External
Environment
Sensory
Neurons
Internal
Environment
Homeostasis
Circulatory System (Material Transport)
Regulatory
Control Feedback Loops
PI Meeting – May 22-24, 2002 – Washington, DC
Visualization
Sim. Object
Sim. Object
Sim. Object
Particle Model
Volume Model
Surface Model
Sim. Object
Functional Info.
Triple Graphics Buffer
Visualization Engine
PI Meeting – May 22-24, 2002 – Washington, DC
Haptics
Sim. Object
Local Physical and
Geometric Model
Instrument
Constraints
Slow update rate:
Global mechanical
deformation computation.
Haptic
Interface
User
Fast update rate:
Local collision detection
and force feedback
computation.
PI Meeting – May 22-24, 2002 – Washington, DC
Haptic Interfacing to Deformable
Objects in Virtual Environments
PI Meeting – May 22-24, 2002 – Washington, DC
Constructing a Local Model in
Real Time
PI Meeting – May 22-24, 2002 – Washington, DC
Stability Implications
• Update rate of simulation
is a critical factor for
stability of interaction.
• Fast update rate haptic
simulation instead of
visual rates improves
stability.
• Oscillatory behavior
present in low frequency
simulation is not
observed.
PI Meeting – May 22-24, 2002 – Washington, DC
Sample Object Model
class Heart : SimObject
Geometry
NonLinearFEM
LumpedFluidModel
{
HeartGeometry;
Muscle;
Blood;
Integrate ();
LocalHapticModel();
Display();
}
PI Meeting – May 22-24, 2002 – Washington, DC
Application:
Virtual Environment Based
Surgical Training Simulator
PI Meeting – May 22-24, 2002 – Washington, DC
Current Training Methods in
Surgery
•Apprenticeship
Limitations due to risks to patients
Difficulty in diffusion of knowledge
•Textbooks
Two Dimensional
•Training mannequins Not very realistic
Limited variation in pathologies
•Animal experiments
Excessive cost
Anatomical differences
PI Meeting – May 22-24, 2002 – Washington, DC
Virtual Environment Based
Surgical Training Simulator
Concept
• Arbitrary anatomies
and pathologies
• New techniques
• No risk to a patient
• Standardization of
training and
accreditation
PI Meeting – May 22-24, 2002 – Washington, DC
Training Simulator - Hardware
PI Meeting – May 22-24, 2002 – Washington, DC
Training Simulator –
Teaching Tasks and Procedures
PI Meeting – May 22-24, 2002 – Washington, DC
Application:
Off-Pump Coronary Artery
Bypass Surgery
PI Meeting – May 22-24, 2002 – Washington, DC
Coronary Artery Bypass Graft Surgery
On-Pump CABG
• Complications
On-pump CABG results in a persistent
20% drop in cognitive skills in 42% of the
patients
• Expensive Procedure
Beating Heart (Off-Pump) CABG
• Fewer Complications
• Less Expensive Procedure
24% reduction in expenditure vs. on-pump CABG
• Shorter Hospital Stay and Faster Recovery
40% shorter hospital stays vs. on-pump CABG
Sources: New England Journal of Medicine
and Annals of Thoracic Surgery
PI Meeting – May 22-24, 2002 – Washington, DC
Limitations of Current Off-Pump
Stabilizer Limitations
• Location of Blockages
• Number of Blockages
• Diameter of Recipient
Coronary Artery
Result
Only 15-20% of the
patients are candidates
PI Meeting – May 22-24, 2002 – Washington, DC
Surgery on the Beating Heart
Active Relative Motion Canceling (ARMC)
• Sensors to measure heart
motion and biological signals
• Algorithms to predict and track
heart motion
• Computer controls the camera
and the instruments to cancel
the heart’s cyclic motion
• Surgeon sees a stationary heart
• Surgeon’s motion is added to
the instrument motion
PI Meeting – May 22-24, 2002 – Washington, DC
Surgery on the Beating Heart
EKG Signal
Model Based Prediction Algorithm
A
Arterial and Ventricular
Blood Pressures
F
B
E
C
D
Feedforward
Position Measurements
+
Feedback
-
Controller
+
+
Surgical
Robot
Surgeon’s
Instrument Motion
PI Meeting – May 22-24, 2002 – Washington, DC
Next Steps
•
Definition of a software API for open source
development of surgical simulators
–
–
•
Development of a heart simulation for surgery
using the specified API
–
•
Release alpha version of the open source surgical
simulator test-bed
Release preliminary version of the organ level API
Stand alone physiological and mechanical heart models
Verification of the simulation with readily available
animal data
PI Meeting – May 22-24, 2002 – Washington, DC
Team and Collaborators
• Project Team:
– Prof. Ron Fearing (PI)
– Dr. M. Cenk Cavusoglu
– Tolga Goktekin
( 10 %)
(100 %)
( 50 %)
• Collaborators:
– UC Berkeley
Prof. Shankar Sastry
Xunlei Wu
Jeff Ustin, MD
– UC San Francisco
Prof. Frank Tendick
PI Meeting – May 22-24, 2002 – Washington, DC
Framework for Open Source Software Development for Organ
Simulation in the Digital Human
Objectives
Organ level modeling for surgical simulation
• Interconnection of organ level models
• Interfacing of organ models with the higher and lower
levels in a hierarchical simulation
Program Schedule
Expected Contributions
API Design:
• Definition of a software API for open source
development of surgical simulators
Surgical Simulation:
• Development of a heart simulation for surgery
API Design
Applications
Heart Model
Organ level modeling interface for BioSpice
Digital Human Project
2001 2002
SOND
2003
JFMAMJJASOND JFMAMJJA
Simulation I/O Interface API
Inter-organ Interface API
Intra-organ Interface API
Simulation Framework
Initial Simulator Design
Support for the API
Heart Model Application
Stand-alone physiological mdl
Stand-alone mechanical mdl
Integrated Model
University of California, Berkeley
PI Meeting – May 22-24, 2002 – Washington, DC
PI Meeting – May 22-24, 2002 – Washington, DC