Transcript PPT - SSTD 2011

Tipping Points, Butterflies, and
Black Swans: A Vision for
Spatio-temporal Data Mining
Analysis
Dr. James M. Kang and Daniel L. Edwards
InnoVision Basic and Applied Research Office
National Geospatial-Intelligence Agency
August 24, 2011
1
Approved for Public Release 11-412
Vision
The development of data mining and
spatio-temporal analytical techniques
to discover tipping-points, butterflies,
and black swans.
2
Approved for Public Release 11-412
What are Tipping Points?
“the moment of critical mass, the threshold, the
boiling point” – M. Gladwell
Climate Tipping Point, Upsala
Glacier, Patagonia, Argentina
(Source: http://www.changeclimate.org/)
3
Approved for Public Release 11-412
What is the Butterfly Effect?
Behavior of dynamic systems
• Highly sensitive to initial conditions – J. Gleck
• Involve topologically mixing – B. Hasselblatt
Source: http://www.guardian.co.uk/world/interactive/2011/mar/22/middle-east-protest-interactive-timeline
Approved for Public Release 11-412
4
What are Black Swans?
• Unpredictable patterns that do not appear to
be Gaussian with an exponential diminishing
tail, but a flatter curve with tails that are fatter
• Have the following characteristics:
• The event is a surprise (to the observer).
• The event has a major impact.
• After its first recording, the event is rationalized
by hindsight, as if it could have been expected.
– N. Taleb
5
Approved for Public Release 11-412
Challenges
6
Approved for Public Release 11-412
Tipping Point Challenges
• Assumptions about
a dataset may
change before and
after a tipping point
event
• Tobler’s Law vs.
Teleconnections
7
Approved for Public Release 11-412
Butterfly Effect Challenges
Bounding problem with sufficiency
1. Depth – sufficient data to
mine vs. scope of problem?
2. Breadth - breadth of data
sufficient to sample problem?
3. Missing – key data/meta
data missing?
3. Stability - of mined patterns?
8
Approved for Public Release 11-412
Black Swan Challenges
• As a Black Swan unfolds,
• Mined patterns over populations
and time may not become
“interesting”
• May not be prevalent or
anomalous
• After a Black Swan is
recognized (hindsight),
• Bounding sufficiency may be
too complex to overcome
• May not generalize to other
known black swans
9
Approved for Public Release 11-412
First and Next Steps
• Tipping Points
• Existing literature in
abrupt changes,
transitions, etc.
• Transient vs. Persistent
• Butterflies and Black
Swans
• Can these be generalized?
• Are these even possible?
• How can we begin
quantifying these events?
• Example Datasets
Source: https://www.cia.gov/library/publications/the-world-factbook/
• Guardian’s event dataset of
middle-east
• CIA World Factbook dataset
10
Approved for Public Release 11-412
www.nga.mil
11
Approved for Public Release 11-412