Transcript Near-Earth Asteroid Tracking Summary of NEAT Results 12/95

Mirage -- Interactive Pattern Discovery
with Large Imaging Databases
http://www.cs.bell-labs.com/who/tkh/mirage
Tin Kam Ho
Department of Scientific Computing Research
Computing Sciences Research Center
Bell Labs, Lucent Technologies
In collaborations with
David Wittman, J. Anthony Tyson of UC Davis
Samuel Carliles, William O’Mullane, Alex Szalay of JHU
Mining Large Imaging Databases
Basic needs:
• Hierarchical data structures & indexing
• Sophisticated navigation tools
Also,
• Joint usage of data, meta-data, extracted features & catalogs
• Automatic pattern discovery algorithms to compute layers of abstraction
from observation, concepts, to theory
Mirage uses visualization to help
• Track horizontal correlations across different types of attributes for the
same objects
• Track vertical correlations across layers of abstraction from signal to the
result of analysis
• Integrate human and machine pattern recognition capabilities
Horizontal Correlations:
Similarity of Objects from Different Perspectives
• Objects can be described by many types of attributes:
position, morphology, color, spectra, temporal variability, motion …
• Meaningful similarity metric exists only for attributes of
the same type
• Similar groups found from one perspective need to be
correlated to those from others
e.g. Are the objects similar in color also similar in shape?
Color groups
Shape groups
Vertical Correlations Across Layers of Analysis
Raw Images
Processed Images
Numerical Features
Classes and Groups
Validation in Input Domain
Relationship between Groups
Interpretation in Context
Human / Machine Interaction
in Pattern Discovery
Domain expertise
Hypotheses
Decisions in algorithmic choices
Interpretation in context
Visualized data geometry
Systematic exploration control
Computed features & data structures
Tentative classifications
Mirage
in action …
A simple way to start :
java –jar Mirage0.3.jar
Loading in a Data Matrix
Making a histogram plot on any attribute
Selecting some interesting bars
Selected bars highlighted
Changing the plot to a different attribute
Making a scatter plot with two attributes
Highlighting an interesting trend
Following it to a different pair of attributes
Making a plot in parallel coordinates
Selecting objects of interest
Highlighting the selection
Bringing up a table view to read the details
When the plots are combined …
Selection from one plot …
Can be broadcast to all others
Opening a new page of plots
Configuring it as you wish
Relating computed spectral classes to other views
Selecting one computed class
Broadcasting it to see member spectra
Many cool features waiting for you to explore …
Challenges for the Analysis Tool
A good tool should support
• separate treatment of non-comparable groups of variables
• versatile visualization utilities allowing many perspectives
• exploration across data types & levels of abstraction
• feedback between manual & automatic pattern recognition methods
A good tool should also
• leverage existing visualization, analysis methods
• enable continuous growth: new visualization, analysis tools
• support seamless interface with data archives
• be scalable in data volume and processing speed
Towards Extensibility
VO
Data Archives
Data Access Clients
Cone Search, CAS
External Rendering Code
Custom Data Views
FITS Viewer, …
Mirage
Core
Extinction Calculator
Data Analysis
Methods
Web Services
Message Based Updates
Data Exchange Pipes
Other Analysis Platforms
Data Access, Custom Views:
VO Enabled Mirage
(with Samuel Carliles, William O’Mullane, and Alex Szalay)
http://skyservice.pha.jhu.edu/develop/vo/mirage/
Data Analysis Functions: Extinction Web Service
(with Chris Miller, Simon Krughoff)
Using DIRBE/IRAS Dust Maps by Schlegel et al.
Mirage
Core
Object selection
Extracts RA,DEC,[mag]
from Mirage data set
Positions, mags
Positions,
mags, filterIDs
SOAP client calls
Extinction server
Enhanced
data set
Result stream
Extinction
Service
E(b-v),
dered_mags
Merges results
with Mirage data set
Continuous Data Updates: SEQUIN experiment
(With Marina Thottan, Ken Swanson)
Network Poller
Obtains statistics from each node
Monitored
Network
Health Checker
Computes health indicators
Mirage Monitor
Record Keeper
Stores statistics and indicators
in relational database
Messenger
Broadcasts messages about
database updates
Retrieves data, updates displays
when message arrives
Open Questions
• What questions do scientists want to ask about their data?
• How can they be translated into graphical operations and
answers?
• Which automatic algorithms are reliable for the tasks?
• Which visualization techniques can help where it matters?
• How can we handle large data volume, variable demands
on speed, disperse archives, and bandwidth constraints?
• What are the best ways to support continuous and
collaborative explorations?
• …
Mirage
can be downloaded at
http://www.cs.bell-labs.com/who/tkh/mirage
Publicly released on the web since late 2002
Development ongoing …
Open source soon to be available