Transcript Advances in the Use of Neurophysiologycally-based Fusion for Visualization and Pattern

Advances in the Use of
Neurophysiologycally-based
Fusion for Visualization and Pattern
Recognition of Medical Imagery
M. Aguilar, J. R. New and E. Hasanbelliu
Knowledge Systems Laboratory
MCIS Department
Jacksonville State University
Jacksonville, AL 36265
1Knowledge Systems Laboratory
Outline






Introduce Med-LIFE.
Revisit 3D image fusion architecture.
Compare 2D and 3D fusion results.
Fusion for segmentation and pattern
recognition.
Contextual zoom tool.
Segmentation results.
2Knowledge Systems Laboratory
Med-LIFE: Learning, Image Fusion,
and Exploration System
3Knowledge Systems Laboratory
3D Shunt Equation
Shunting Neural Network
Equation:
Grossberg (1968),
Elias & Grossberg (1972)
xijk   Axijk  ( B  xijk )C[Gc * I c ]ijk
 ( D  xijk )[Gs * I S ]ijk
Where:
A – decay rate
B – maximum activation level (set to 1)
D – minimum activation level (set to 1)
IC – excitatory input
IS – lateral inhibitory input
C, Gc and Gs are as follows:
G ( x, y, z ) 
1
4 2

e
3
( x2  y2  z 2 )
2 2
3D Shunt Operator Symbol
4Knowledge Systems Laboratory
2-Band 3D Fusion Architecture
5Knowledge Systems Laboratory
4-Band 3D Fusion Architecture
6Knowledge Systems Laboratory
2D vs. 3D Fusion Results
MRI-T1
MRI-PD
2D Fusion
MRI-T2
SPECT
3D Fusion
7Knowledge Systems Laboratory
4-Band Hybrid Fusion
Architecture
.
.
T1
Images
SPECT
Images
Q
Color
I
Remap
Y
T2
Images
.
..
.
PD
Image
Noise cleaning &
registration if needed
Color Fuse
Result
+
_
Contrast
Enhancement
Between-band Fusion
and Decorrelation
8Knowledge Systems Laboratory
Hybrid Fusion Results
2D Fusion
3D Fusion
9Knowledge Systems Laboratory
User-Driven Learning for
Segmentation & Pattern Recognition
10Knowledge Systems Laboratory
Contextual Zoom Visualization
Zoom in place:
1. occludes information
2. reduces efficiency by
forcing user to maintain
context
Zoom in place supports:
1. focused attention
2. improved screen realestate usage
11Knowledge Systems Laboratory
Contextual Zoom Visualization
12Knowledge Systems Laboratory
Contextual Zoom Visualization
• Developed based on COTS
software developed by
Idelix
• Supports visualization of
fused imagery at multiple
details levels
• Supports detailed analysis
and selection for user-driven
pattern learning…
13Knowledge Systems Laboratory
User-Driven Pattern Learning




Supervised learning where training data is
selected by user/expert (Waxman et al).
Results assessed and corrected by user.
Fuzzy ARTMAP neural network for fast and
stable learning.
Address order sensitivity by introducing N
voters trained with alternate ordering of
the training data.
14Knowledge Systems Laboratory
Pattern Recognition Results
15Knowledge Systems Laboratory
Heterogeneous Voting


Train 3 Fuzzy ARTMAP systems with
parameters as before (different data
orderings)
Train remaining 2 systems with all
parameters as in the 3rd system except for
Vigilance (which is a threshold measure
that controls the sensitivity of the system).
16Knowledge Systems Laboratory
Homogeneous vs. Heterogeneous
Voters
5 Homogeneous Voters
5 Heterogeneous Voters
17Knowledge Systems Laboratory
2D vs. 3D Fusion
Segmentation Results
2D Fusion-based
Segmentation
3D Fusion-based
Segmentation
18Knowledge Systems Laboratory
Generalization
Training
Results
Slice 11
Testing
Results
Slice 10
19Knowledge Systems Laboratory
Conclusions




Modified fusion approach combines benefits of
2D and 3D fusion.
Preliminary learning segmentation results
indicate robustness across slices and cases.
Demonstrated superior performance of 3D fusion
for both visualization and pattern recognition.
Heterogeneous voting scheme improves learning
performance.
20Knowledge Systems Laboratory
BACK-UPS
21Knowledge Systems Laboratory
2D vs. 3D Generalization
2D Fusion
3D Fusion
Slice 10
Testing
Results
22Knowledge Systems Laboratory
Image Fusion
23Knowledge Systems Laboratory