Transcript BrainJ3D - University of Washington

3-D Visualization of Functional
Brain Map Data
A.V. Poliakov; E.B. Moore; J.F. Brinkley,
Structural Informatics Group
Department of Biological Structure
University of Washington
UW Human Brain Project
Structural Informatics Group, Dr. Brinkley
Collaborators
Neurosurgery, Dr. Ojemann
Psychiatry, Dr. Corina
Radiology, Dr. Maravilla
Medical Procedure
Neurosurgery for intractable epilepsy
Objective
Remove epileptic foci while sparing the patient’s language function
Method
Language mapping using electrical stimulation
Information System
Help researches organize and analyze the data
Unique data
Study language
Evaluate non-invasive imaging techniques for surgical planning
Information System for Language Mapping
Primary Sources of Data
Data acquired prior to surgery
MR scans of cortical anatomy, veins, arteries
Functional MRI
Data acquired during surgery
Electrical stimulation map
Phonograms
Information System for Language Mapping
Primary Sources of Data
Data acquired prior to surgery
MR scans of cortical anatomy, veins, arteries
Functional MRI
EEG in grid patients
Data acquired during surgery
Electrical stimulation map
Phonograms
Single unit recording
Information System for Language Mapping
Brain Mapper
• Alignment
• Masking
• Reconstructing
3D models
• Mapping CSM
Sites
Information System for Language Mapping
Information System for Language Mapping
Data Management
Talairach Daemon Query
Query the Brain Mapping Repository using XQuery
In development
fMRI DB and Automated Data Analysis
Single Unit DB
SPM
Web Interface
Web Interface
Web Interface
Web Interface
Web Interface
Web Interface
Web Interface
Web Interface
Web Interface
3D Visualization
System Architecture
– Client-server
– Server-based rendering and data storage
– Web-based client (e.g. Java or forms-based)
Client
Client #1
CGI script
Experiment Management System
Client #2
Java Applet
Client #3
...
Web Interface
snapshot
Graphics
Server
snapshot
Relational
Database
3-D Image
Volumes
3-D
Models
Internet
Server
Data
Brain
Mapper
Stimulation
Sites
2-D
Images
3D Visualization
Requests
New Users
Different data modalities
Various software packages
Spectroscopy
ERP Source Signal
fMRI
Collaborators
New features
Remote collaboration
Transcranial Magnetic Stimulation
3D Visualization
Limitations of current system
Relatively slow software rendering
Non-portable architecture
SGI Irix and Linux only
Large code base
Needs to be maintained and developed
Reliance on outdated software tools
3D Visualization
Other possible approaches
OpenGL, DirectX (Windows)
VTK, Java3D
Higher level toolkits
Java3D
Cross Platform
Supports hardware acceleration
Off-screen rendering
Java
Pilot studies



Performance
Off-screen rendering
Client-server
Proof of concept
BrainJ3D – Next Generation
Live Demo
BrainJ3D
BrainJ3D
Features Seen in Demo
• Cutaway
• Mapping
• fMRI Overlays
• Workspace
BrainJ3D
Other Features
• Supports common formats and data types
(Analyze, Minc, data created with SPM, FSL, etc.)
– Leverages functionality of other programs
– Increases user base
– Use Java utilities
• 3D model loaders
• NetCDF library
• 2D image API
• Configuration and internal files in XML
– More portable
– Easy to customize
BrainJ3D
Other Features
• Client/Server
– Client/Server mode looks and behaves exactly
like standalone mode
– Reduced performance
• Sending the image is a bottleneck
– Takes advantage of built in Java security
– Thin client only requires installation of Java
• Java3D and accelerated graphics card not required
BrainJ3D
Architecture
BrainJ3D
Java and Java3D provide viable development
environment for scientific visualization
applications
+
+
+
+
+
Cross Platform ( Linux, Windows and Solaris)
Supports hardware acceleration and scene optimization
Supports hardware accelerated off-screen rendering
Mature software package that provides needed functionality
Java
Modern object-oriented language
Numerous libraries and utilities
Web-based and remote technologies are well supported
BrainJ3D
Future Development
Different way to access program
Java Web Start
HTML GUI
Advanced Visualization Techniques
Time series (movie) mode for Volumes
Show functional activation on Surface model
Other Features
Analysis and visualization features requested by users
Current and Future Work
Integration with Information System for Language Mapping
Improving Interoperability
Support for other data modalities
Spectroscopy, ERP source signal, Transcranial Magnetic Stimulation
Importing data from other Brain Imaging software packages
FreeSurfer and SureFit
* Interoperability is a fast moving target, often beyond our control
Refining Client/Server mode
Security
Performance
UW Human Brain Project
Structural Informatics Group
http://sig.biostr.washington.edu/
Acknowledgements
Human Brain Project grant DC02310
National Institute of Deafness and Other Communication Disorders and
National Institute for Mental Health